Finding the following in a CS research article:
… understanding the brain’s computing paradigm has the potential to produce a paradigm shift in current models of computing.
almost surely would qualify the respective article as crackpot, right? Wrong, for historical and contemporary reasons, which I shall mention further.
The brain differs from modern computing systems in many ways. The first striking difference is its use of heterogeneous components: unlike the components of a modern computer, the components of the brain (ion channels, receptors, synapses, neurons, circuits) are always highly diverse – a property recently shown to confer robustness to the system . Second, again unlike the components of a computer, they all behave stochastically – it is never possible to predict the precise output they will produce in response to a given input; they are never “bit-precise”. Third, they can switch dynamically between communicating synchronously and asynchronously. Fourth, the way they transmit information across the brain is almost certainly very different from the way data is transmitted within a computer: each recipient neuron appears to give its own unique interpretation to the information it receives from other neurons. Finally, the brain’s hierarchically organised, massively recurrent connectively, with its small-world topology, is completely different from the interconnect architecture of any modern computer. For all these reasons, understanding the brain’s computing paradigm has the potential to produce a paradigm shift in current models of computing.
Part of the efforts made by HBP are towards neuromorphic computing. See the presentation Real-time neuromorphic circuits for neuro-inspired computing systems by Giacomo Indiveri, in order to learn more about the history and the present of the subject.
2. As you can see from the presentation, neuromorphic computing is rooted in the article “A logical calculus of the ideas immanent in nervous activity” by Warren Mcculloch and Walter Pitts,1943, Bulletin of Mathematical Biophysics 5:115-133. This brings me to the “history” part. I shall use the very informative article by Gualtiero Piccinini “The First Computational Theory of Mind and Brain: A Close Look at McCulloch and Pitts’s ‘Logical Calculus of Ideas Immanent in Nervous Activity'”, Synthese 141: 175–215, 2004. From the article:
[p. 175] Warren S. McCulloch and Walter H. Pitt’s 1943 paper, ‘‘A Logical Calculus of the Ideas Immanent in Nervous Activity,’’ is often cited as the starting point in neural network research. As a matter of fact, in 1943 there already existed a lively community of biophysicists doing mathematical work on neural networks. What was novel in McCulloch and Pitts’s paper was a theory that employed logic and the mathematical notion of computation – introduced by Alan Turing (1936–37) in terms of what came to be known as Turing Machines – to explain how neural mechanisms might realize mental functions.
About Turing and McCulloch and Pitts:
[p. 176] The modern computational theory of mind and brain is often credited to Turing himself (e.g., by Fodor 1998). Indeed, Turing talked about the brain first as a ‘‘digital computing machine,’’ and later as a sort of analog computer. But Turing made these statements in passing, without attempting to justify them, and he never developed a computational theory of thinking. More importantly, Turing made these statements well after the publication of McCulloch and Pitts’s theory, which Turing knew about. Before McCulloch and Pitts, neither Turing nor anyone else had used the mathematical notion of computation as an ingredient in a theory of mind and brain.
[p. 181] In 1936, Alan Turing published his famous paper on computability (Turing 1936–37), in which he introduced Turing Machines and used them to draw a clear and rigorous connection between computing, logic, and machinery. In particular, Turing argued that any effectively calculable function can be computed by some Turing Machine – a thesis now known as the Church–Turing thesis (CT) – and proved that some special Turing Machines, which he called ‘‘universal,’’ can compute any function computable by Turing Machines. By the early 1940s, McCulloch had read Turing’s paper. In 1948, in a public discussion during the Hixon Symposium, McCulloch declared that in formulating his theory of mind in terms of neural mechanisms, reading Turing’s paper led him in the ‘‘right direction.’’
On McCulloch and “the logic of the nervous system”:
[p. 179] In 1929, McCulloch had a new insight. It occurred to him that the all-or-none electric impulses transmitted by each neuron to its neighbors might correspond to the mental atoms of his psychological theory, where the relations of excitation and inhibition between neurons would perform logical operations upon electrical signals corresponding to inferences of his propositional calculus of psychons. His psychological theory of mental atoms turned into a theory of ‘‘information flowing through ranks of neurons.’’ This was McCulloch’s first attempt ‘‘to apply Boolean algebra to the behavior of nervous nets.’’ The brain would embody a logical calculus like that of Whitehead and Russell’s Principia Mathematica, which would account for how humans could perceive objects on the basis of sensory signals and how humans could do mathematicsand abstract thinking. This was the beginning of McCulloch’s search for the ‘‘logic of the nervous system,’’ on which he kept working until his death.
On Pitts, McCulloch and logic:
[p. 185-186] In the papers that Pitts wrote independently of McCulloch, Pitts did not suggest that the brain is a logic machine. Before McCulloch entered the picture, neither Pitts nor any other member of Rashevsky’s biophysics group employed logical or computational language to describe the functions performed by networks of neurons. The use of logic and computation theory to model the brain and understand its function appeared for the first time in McCulloch and Pitts’s 1943 paper; this is likely to be a contribution made by McCulloch to his joint project with Pitts. […]
Soon after McCulloch met Pitts, around the end of 1941, they started collaborating on a joint mathematical theory that employed logic to model nervous activity, and they worked on it during the following two years. They worked so closely that Pitts (as well as Lettvin) moved in with McCulloch and his family for about a year in Chicago. McCulloch and Pitts became intimate friends and they remained so until their death in 1969. According to McCulloch, they worked largely on how to treat closed loops of activity mathematically, and the solution was worked out mostly by Pitts using techniques that McCulloch didn’t understand. To build up their formal theory, they adapted Carnap’s rigorous (but cumbersome) formalism, which Pitts knew from having studied with Carnap. Thus, according to McCulloch, Pitts did all the difficult technical work.
A citation from McCullogh and Pitts paper [p. 17 from the linked pdf]
It is easily shown: first, that every net, if furnished with a tape, scanners connected to afferents, and suitable efferents to perform the necessary motor-operations, can compute only such numbers as can a Turing machine; second, that each of the latter numbers can be computed by such a net; and that nets with circles can be computed by such a net; and that nets with circles can compute, without scanners and a tape, some of the numbers the machine can, but no others, and not all of them. This is of interest as affording a psychological justification of the Turing definition of computability and its equivalents, Church’s -definability and Kleene’s primitive recursiveness: If any number can be computed by an organism, it is computable by these definitions, and conversely.
Comment by Piccinini on this:
[p. 198] in discussing computation in their paper, McCulloch and Pitts did not prove any results about the computation power of their nets; they only stated that there were results to prove. And their conjecture was not that their nets can compute anything that can be computed by Turing Machines. Rather, they claimed that if their nets were provided with a tape, scanners, and ‘‘efferents,’’ then they would compute what Turing Machines could compute; without a tape, McCulloch and Pitts expected even nets with circles to compute a smaller class of functions than the class computable by Turing Machines.
I have boldfaced the previous paragraph because I find it especially illuminating, resembling the same kind of comment as the one on currying I gave in the post “Currying by using zippers and an allusion to the Cartesian Theater“.
[p. 198-199] McCulloch and Pitts did not explain what they meant by saying that nets compute. As far as the first part of the passage is concerned, the sense in which nets compute seems to be a matter of describing the behavior of nets by the vocabulary and formalisms of computability theory. Describing McCulloch–Pitts nets in this way turned them into a useful tool for designing circuits for computing mechanisms. This is how von Neumann would later use them (von Neumann 1945).