# Ideology in the vision theater

Thanks to Kenneth Olwig for suggesting that ideology may be related to the argument from  the post  On the exterior homunculus fallacy . More precisely, Olwig points to the  following quote from The German Ideology by Karl Marx and Friedrich Engels:

If in all ideology men and their circumstances appear upside-down as in a camera obscura, this phenomenon arises just as much from their historical life-process as the inversion of objects on the retina does from their physical life-process. In direct contrast to German philosophy which descends from heaven to earth, here we ascend from earth to heaven. That is to say, we do not set out from what men say, imagine, conceive, nor from men as narrated, thought of, imagined, conceived, in order to arrive at men in the flesh. We set out from real, active men, and on the basis of their real life-process we demonstrate the development of the ideological reflexes and echoes of this life-process. The phantoms formed in the human brain are also, necessarily, sublimates of their material life-process, which is empirically verifiable and bound to material premises.

One of the first posts of this blog was The Cartesian Theater: philosophy of mind versus aerography, where I use the article “All that is landscape is melted into air: the aerography’ of ethereal space”, Environment and Planning D: Society and Space 2011, volume 29, pages 519 – 532 by Olwig in order to argue that the Cartesian Theater notion of Dennett is showing only one half of the whole homunculus fallacy. Indeed, Dennett’s theater is a theater in a box, the invention of Inigo Jones, already designed around the king (or homunculus, in Dennett argument), using geometrical perspective for giving an appearance of reality to an artificial construct, the scenic space.

With any homunculus, I argue, comes also a scenic space, which has to be taken into account in any theory of mind, because it is as artificial, it leads to the same kind of fallacy as the homunculus. In the posts   Towards aerography, or how space is shaped to comply with the perceptions of the homunculus   and  Theatron as an eye  I further develop the subject by trying to see what becomes the homunculus fallacy if we use not the theater in a box, but the old greek theater instead (and apparently it seems that it stops to be a fallacy, as homunculi and designed scenic spaces melt into oblivion and the gnomon, the generator of self-similarity, comes to the attention). Finally, in   the post On the exterior homunculus fallacy  I argue that the original homunculus fallacy is not depending on the fact that the homunculus is inside or outside the brain, thus leading me to suppose that the neuroscientist which studies a fly’s vision system is an exterior homunculus with respect to the fly and the lab is the scenic space of this homunculus. It means that any explanation of the fly vision which makes use of arguments which are not physically embedded in the fly brain (like knowledge about the euclidean structure of the space) makes sense for the experimenter, but cannot be the real explanations, because the fly does not have a lab with a small homunculus inside the head.

Which brings me to the relation with ideology, which is more than a given point of view, is a theater in a box which invites the infected host to take the place of the homunculus, watch the show and make an opinion based on the privileged position it occupies. But the opinion can be only one, carefully designed by the author of the ideology, the scenographer.
The scenic space needs an Inigo Jones, Inigo Jones is the ignored dual of the homunculus-king. He does not use magic in order to organize the show for the king, but he adds meaning. In the case of an ideology (a word which has as root a greek word meaning “to see”, thanks again to Olwig for this) the added meaning is intentional, but in the case of a neuroscientist which experiments on the vision system of a fly (what a king) it is unintended, but still present, under the form of assumptions which lead the experimenter to an explanation of the fly vision which is different from what the fly does when seeing (likely  an evolving graph with neurons as nodes and synapses as edges, which modifies itself according to the input, without any exterior knowledge about the experimenter’s lab and techniques).

# On the exterior homunculus fallacy

If we think about a homunculus outside the brain, the homunculus fallacy still functions.

This post continues the Vision theater series  part I, part II, part III, part IV, part V, part VI  and also links to the recent Another discussion about computing with space .

The homunculus argument is a fallacy arising most commonly in the theory of vision. One may explain (human) vision by noting that light from the outside world forms an image on the retinas in the eyes and something (or someone) in the brain looks at these images as if they are images on a movie screen (this theory of vision is sometimes termed the theory of the Cartesian Theater: it is most associated, nowadays, with the psychologist David Marr). The question arises as to the nature of this internal viewer. The assumption here is that there is a ‘little man’ or ‘homunculus‘ inside the brain ‘looking at’ the movie.

The reason why this is a fallacy may be understood by asking how the homunculus ‘sees’ the internal movie. The obvious answer is that there is another homunculus inside the first homunculus’s ‘head’ or ‘brain’ looking at this ‘movie’. But how does this homunculus see the ‘outside world’? In order to answer this, we are forced to posit another homunculus inside this other homunculus’s head and so forth. In other words, we are in a situation of infinite regress. The problem with the homunculus argument is that it tries to account for a phenomenon in terms of the very phenomenon that it is supposed to explain.

Suppose instead that the homunculus is outside the brain. Why, for example think about the experimenter doing research on your vision. The fallacy functions as well, because now we have another homunculus (outside the brain) who looks at the movie screen (i.e. the measurements he performed on your visual system, in the medium controlled by him). “But how does this homunculus see the ‘outside world’?” Infinite regression again.

If you think that is outrageous, then let me give you an example. The exterior homunculus (experimenter) explains your vision by interpreting the controlled space  he put you in (the lab) and the measurements he performed. When he does this interpretation he relies on:

• physical laws
• geometrical assumptions
• statistical assumptions

at least. Suppose that the experimenter says: “to the subject [i.e. you] was presented a red apple, at distance $d$, at coordinates $x,y,z$. By the physical laws of opticks and by the geometrical setting of the controlled lab we know that the sensor $S$ of the retina of the left eye was stimulated by the light coming from the apple. We recorded a pattern of activity in the regions $A, B, C$ of the brain, which we know from other knowledge (and statistical assumptions) that   $A$ is busy with recognition of fruits, $B$ is involved in contour recognition and $C$ with memories from childhood.” I agree that is a completely bogus simplification of what the real eperimenter will say, but bear with me when I claim that the knowledge used by the experimenter for explaining how you see the apple has not much to do with the way you see and recognize the apple. In the course of the explanation, the experimenter used knowledge about the laws of optics, used measurements which are outside you, like coordinates and geometric settings in the lab, and even notions from the experimenter’s head, as “red”, “apple” and “contours”.

Should the experimenter not rely on physical laws? Or on geometrical asumptions (like the lab is in a piece of euclidean 3d space)? Of course he can rely on those. Because, in the case of physical laws, we recognize them as physical because they are invariant (i.e. change in a predictable way) on the observer. Because in the case of geometrical assumptions we recognize them as geometrical because they are invariant on the parametrization (which in the lab appears as the privilege of the observer).

But, as it is the case that optics can explain only what happens with the light until it hits the retina, not more, the assumptions in the head of the experimenter, even physical and geometrical, cannot be used as an explanation for the way you see. Because, simply put, it is much more likely that you don’t have a lab in the head which is in a euclidean space, with an apple, a lamp and rules for measuring distances and angles.

You may say that everybody knows that apples are not red, that’s a cheap shot because apples scatter light of all frequencies and it just happen that our sensors from the retina are more sensible at some frequencies than other. Obvious. However, it seems that not many recognize that contours are as immaterial as colors, they are in the mind, not in reality, as Koenderink writes in  Theory of “Edge-Detection”  JJ Koenderink – Analysis for Science, Engineering and Beyond, 2012.

The explanation of vision which uses an exterior homunculus becomes infinite regression unless we also explain how the exterior homunculus thinks about all these exterior facts as lab coordinates, rational deductions from laws of physics and so on. It is outrageous, but there is no other way.

Let’s forget about experiments on you and let’s think about experiments on fly vision. Any explanation of fly vision which uses knowledge which is not, somehow, embodied in the fly brain, falls into the (exterior)  homunculus fallacy.

So what can be done, instead? Should we rely on magic, or say that no explanation is possible, because any explanation will be issued by an exterior homunculus? Of course not. When studying vision, nobody in the right mind doubts about the laws of optics. They are science (i.e. reproducible and falsifiable). But they don’t explain all the vision, only the first, physical step. Likewise, we should strive for giving explanations of vision which are scientific, but which do not make appeal to a ghost, in the machine or outside the machine.

Up to now, I think that is the best justification for the efforts of understanding space not in a passive way.

# Neural darwinism, large scale homunculus

Neural darwinism, wiki entry:

Neural Darwinism, a large scale theory of brain function by Gerald Edelman, was initially published in 1978, in a book called The Mindful Brain (MIT Press). It was extended and published in the 1989 book Neural Darwinism – The Theory of Neuronal Group Selection.”

“The last part of the theory attempts to explain how we experience spatiotemporal consistency in our interaction with environmental stimuli. Edelman called it “reentry” and proposes a model of reentrant signaling whereby a disjunctive, multimodal sampling of the same stimulus event correlated in time leads to self-organizing intelligence. Put another way, multiple neuronal groups can be used to sample a given stimulus set in parallel and communicate between these disjunctive groups with incurred latency.”

Here is my (probably very sketchy) understanding of this mysterious “reentry”.

Say $X$ is the collection of neurons of the brain, a discrete set with large cardinality $N$. At any moment the “state” of the brain is partially described by a $N \times N$ matrix of weights: the number $w_{ij}$ is the weight of the connection of the neuron $i$ with the neuron $j$ (a non-negative real  number).

We may imagine such a state of the brain as being the trivial groupoid $X \times X$ with a weight function defined on arrows with values in $[0,+\infty)$.

Instead of neurons and weights of connections we may easily imagine more complex situations (for example take the trivial groupoid generated by connections; an arrow between two connections is the neuron incident to both connections, and so on; moreover, weights could be enriched,…), so let’s just say that a state of  the brain is a weighted groupoid.

With a dynamics of weights.

Define a “neuronal group” as being a sub-groupoid with a weight function.  Take now two neuronal groups $(A,w_{A})$ and $(B,w_{B})$.  How similar are they, in the brain?

For this we need a cost function which applies to any “weighted relation” (in the brain, i.e. in the big weighted groupoid) from $A$ to $B$ and spills a non-negative number. The similarity between two neural groups (with respect to a particular state of the brain) is the minimum of these numbers over all the possible connectomes between $A$ and $B$.

My feeble understanding of this reentry is that, as time passes, the state of the brain evolves in a way which increases (in fact decreases the cost of) similarity of neuronal groups “encoding” aspects of the same “stimulus” which are correlated in time.

We may the imagine a “large scale homunculus” as being a similar but strict neural sub-group of the whole brain. The reentry weighted relation will then have a structure of an emergent algebra.

Indeed, there is a striking similarity between this formalization (probably very naive w.r.t. the complexity of the problem, and also totally ignoring dynamical aspects) and the characterization of emergent algebras as being related to the problem of exploring space and matching collections of maps, as described in  “Maps of metric spaces“, see also  these slides.

Just replace distances with matrices of weights, or equivalently, think about the previous image as:

I shall come back to this with all details later.

# A discussion about the structure of visual space

In August I discovered the blog The structure of visual space group and I was impressed by the post The Maya theory of (visual) perception.  A discussion with Bill Rosar started in the comments, which I consider interesting enough to rip it from there and put it here.

This discussion may also help to better understand some of the material from my blog. Several  links were added in order to facilitate this. Here is the exchange of comments.

Me: “I just discovered this blog and I look forward to read it in detail.

Re: “What I am calling into question is the ontological status of a physical world existing beyond the senses.”

Also in relation to the mind-body dualism, I think there is a logical contradiction in the “Cartesian Theater” argument by Dennett, due to the fact that the fact that Dennett’s theater is a theater in a box, already designed for a dualist homunculus-stage space perception (in contradistinction with the older, original Greek Theater).”

Bill Rosar: “Thank you for your posting, Marius Buliga, and welcome! It is great to have a mathematician join us, for obvious reasons, especially since you are interested in problems in geometry.

Your idea of the eye as a “theatron” is interesting, though I do not believe that the brain is computing anything, for the simple reason that it is not a computer, and doesn’t behave like one, as some neuroscientists are now publicly saying. It is people who perform computations, not brains.

Raymond Tallis, who posted “The Disappearance of Appearance” here two years ago, went to some pains to articulate the fallacious reasoning behind the computational metaphor of mind and brain in his marvelous little book WHY THE MIND IS NOT A COMPUTER.

It has long been a truism in cognitive psychology that we do not see our retinal images, and the “function” or process of vision is probably very different from the creation of images, because there is no image in the brain, nor anything like one. If anything, the pattern of stimulation on the retinae is “digested” by the visual system, broken down rather like food is into nutrients (as an alternative, think of chemical communication among insects).

To my knowledge, Descartes did not invoke the analogy of a theater for vision (or perception in general), so for Dennett to construe his ideas on such an analogy is dubious at the outset and, in this instance, just seems to make for a straw man. For that matter, Dennett does not seem to understand the reasons for dualism very fully, and as nearly as I can determine, never bothered to acquaint himself with the excellent volume edited by John Smythies and John Beloff, THE CASE FOR DUALISM (1989). His ill-informed refutations just strike me as facile and unconvincing (and his computational theory of mind has been roundly rejected by Ray Tallis as being fallacious).

My own invoking of theater here as an analogy is to reality itself, not just perception, and is therefore quite different from the view Dennett imputes to Cartesian dualism, though. I propose that physics studies the stagecraft of a reality that only (fully) exists when perceived–which is closer to Berkeley than Descartes, and is a view consistent with John Wheeler’s “observer-participant” model of the universe.

Theoretical physicist Saul-Paul Sirag advanced a “many realities” alternative to the Everett-Wheeler “many worlds” hypothesis, arguing that other realities are mathematically possible. That is why I have tendered the provocative notion that the reality we know is a sort of construction, one that is maintained by the physical constants–or so it seems. Sirag argued that it is not the only possible reality for that reason, and that the constants are comparable to the “chains” that hold the cave dwellers captive to the shadow play on the wall.

I propose instead that the senses are part of the reality-making “mechanism,” and that vision has more the character of a resolving mechanism than a picture-making one (not quite like the Bohm-Pribram holographic reality/brain analogy, though). That gets rid of the homunculus problem, because it turns the perception process inside out: The person and homunculus are one and the same, and visual space is just where it appears to be, viz. in front of us, not a picture made by the visual system in the brain. The forerunner of this view was James Culbertson. The flaw is that it requires a rejection or modification of the causal theory of perception, as we have discussed here. But causality is a metaphysical principle, not a physical one, and perhaps in this context at least requires some close scrutiny, just as Culbertson gave it.”

Me: “…”…for Dennett to construe his ideas on such an analogy is dubious at the outset and, in this instance, just seems to make for a straw man.” This is my impression also, but what can we learn from this about vision?

As a mathematician, maybe, I am quite comfortable with vagueness. What I get from the greek theater/theater in a box argument is that the homunculus is as artificial as the scenic space, or the outer, physical space. These two notions come in pairs: either one has both, or none. The positive conclusion of the argument is that we have to go higher: there is a relation, akin to a map-territory relation, which has on one side the homunculus and on the other side the space.

Let me elaborate a bit on the map-territory relation. What is a map of a territory? It is the outcome of a collection of procedures agreed by the cartographer and the map reader. The cartographer wanders through the territory and constructs a map by some procedure, say by measuring angles and distances using some apparatus. The cartographer follows a convention of representation of the results of his experiments on a piece of paper, let us call this convention “euclidean geometry” (but it might be “quantum mechanics” as well, or “relativity theory”…). The map reader knows that such convention exists and moreover, at least concerning basic facts, he knows how to read the map by applying the convention (for example, the reader of the map of a city, say, knows that straight lines are shortest on the maps as well as across the city). We may say that the map-territory relation (correspondence between points from the territory – pixels from the map) IS the collection of agreed procedures of representation of experiments of the cartographer on the map. The relation between the particular map and the particular territory is just an outcome of this map-territory relation.

Looking at this level, instead of speaking about the perception of the exterior by the homunculus, it is maybe more reasonable to speak, like in “The structure of visual spaces” by J.J. Koenderink, A.J. van Doorn, Journal of mathematical imaging and vision, Volume: 31, Issue: 2-3 (2008), pp. 171-187, about the structure of the visual space as being the result of a controlled hallucination, based on prior experiences which led to coherent results.

Bill Rosar: “Thank you, Marius! What can we learn from Dennett’s faulty analysis of vision, you ask? The “moral of the story” IMO is that any model based on computation presupposes that we know how people perform computations–or how the human minds does–which is something we presently unknown, because we don’t really know what the “mind” really is–it’s just a name. All a computer does is automate a procedure we humans perform. To assume that Nature makes computers strikes me as a classic example of anthropormorphism, and Ray Tallis would agree. How then to get beyond that fallacy? Or, in the case of vision, to echo John Wheeler’s style of formulating foundational problems in physics, “How do you get vision without vision?”–that is, how to understand vision without presupposing it? That’s quite a feat!

A few months ago when Bob French and I were last debating some of these points I suggested that we turn to the evolution of the eye and see what that tells us. Conveniently the evolution of the eyes has been one of Richard Dawkins’ favorite examples to refute the idea of “intelligent design”.

In light of all the questions the account Dawkins raises but leaves unanswered, intelligent design seems to make more sense (I offer no opinion on that myself). So it is a question of what the simplist eyes do and how the organisms possessing them use them. There is a nice little video on YouTube that highlights all that Dawkins does not explain in his simplist account of the evolution of the eye.

As for the map-territory analogy you suggest, it is comparable to the idea of “cortical maps” but shares the same conceptual pitfall as that of the perspective projection analogy I gave above, because as I noted, unlike being able to compare the flat perspective projection (map) with the 3-D *visual space* of which it is (supposedly) a projection, we cannot do that with visual space in relation to putative physical space, which lies beyond our senses. It seems to me that we are to some extent each trapped solipsistically within our own perceptual world.

Koenderink’s idea just seems like nonsense to me, because we don’t even really know what hallucinations are any more than how a hallucinatory space is created relative to our “normal” waking visual space (BTW we invited Koenderink to join the blog a few years ago, but he never replied). The *concept* of a hallucination is only useful when one has some non-hallucinatory experience to which to compare it–thus the same problem as the projection analogy above.

Trouble is we seem to be *inside* the system we are trying to understand, and therefore cannot assume an Archimedean point outside it from which to better grasp it (one of the fundamental realizations Einstein had in developing the theory of relativity, i.e., relativity is all *within* the system = universe).

As for visual space being non-Euclidean or not, I called into question many years ago the interpretation of the data upon which all theories of the geometry of visual space are based, because the “alley experiments” never took into account changes of projection on the retinae as a function of eye movement, i.e., the angles of objects projected on the retina are constantly changing as the eyes move. This has never been modeled mathematically, but it should be. Just look at the point where a wall meets the ceiling an run your eyes along its length, back and forth. You will notice that the angle of the line changes as you move your eyes along it.

Yes, the space and homunculus are an inseparable pair IMO–just look at Wheeler’s symbolic representation of the observer-participant universe (the eye looking at the U).”

Bill Rosar: “I should hasten to emend my remarks above by stating that when we speak of “eyes” and “brains” such objects are only known to us by perception. So like any physical object, we cannot presuppose their existence as such separate from our perception of them–except by an act of a kind of faith (belief), much as we believe that the sun will rise every morning. Therefore talking about their “function” etc. is still all resting upon perceptions, without which we would have no knowledge of anything, ergo, something like Aristotle’s dictum “There is nothing in the mind that was not first in the senses.” Are there eyes and brains that exist independently of perceptions of them?”

Me: “Dear Bill, thank you for the interesting comments! I have several of my own (please feel free to edit the post if it is too long, boring or otherwise repellent for the readers of this blog):

1. It looks to me we agree more than my faulty style of exposition shows: one cannot base an explanation of how the space is “re-constructed” in the brain on the structure of the physical space, point. It may be that what we call structure of physical space is formed by features selected as significant by our brain, in the same way as a wind pipe extracts a fundamental note from random noise (thank you Neal Stephenson).

2. We both agree (as well as Koenderink, see his “Brain a geometry engine”) that, as you write, “the senses are part of the reality-making “mechanism,” and that vision has more the character of a resolving mechanism than a picture-making one”.

3. Concerning “computing”, is just a word. In the sense that “computing” is defined as something which could be done by Turing machines, or expressed in lambda calculus, etc, I believe too that the brain is not computing in this sense. With efforts and a lot of dissipation, it seems that the brain is able to compute in this sense, but naturally it does not. (It would be an interesting project to experimentally “measure” this dissipation, starting for example from a paper by Mark Changizi “Harnessing vision for computation”, here is the link to a pdf.

4. But if we enlarge the meaning of the word “computing” then it may as well turn out that the brain does compute. The interesting question for a mathematician is: find a definition of “computation in enlarged sense” which fits with what the brain does in relation to vision. This is a project dear to me, I don’t want to bother you with this (unless you are interested), which might have eventual real world applications. I started it with the paper “Computing with space, a tangle formalism for chora and difference” and I reached the goal of connecting this (as a matter of proof of principle, not because I believe that the brain really computes in the classical sense of the word) with lambda calculus in the paper “Local and global moves on locally planar trivalent graphs, lambda calculus and lambda-Scale“.
(By the way, I cannot solve the problem of where to submit a paper like “Computing with space…”)

5. Concerning “hallucination”, as previously, is just a word. What I think is likely to be true is that, even if the brain does not have direct access to the physical space, it may learn a language of primitives of this space, by some bayesian or other, unknown, procedure, which is akin to say that we may explain why we see (suppose, for the sake of the discussion) an euclidean 3d space not by using as hypothesis the fact that the physical space has this structure, but because our brains learn a family of primitives of such a structure and then lay in front of our eyes a “hallucination” which is constructed by the consistent use of those primitives.”

Bill Rosar: “Thanks for these stimulating thoughts and ideas, Marius. Not to worry about the length of your blog postings. Mine are often (too) long, too. My remarks will be in two parts. This is part I.

When John Smythies and I started this blog (which was really intended to be a “think tank” rather than a blog), we agreed that, following the lead of Einstein, it may be necessary to re-examine fundamental concepts of space and geometry (not to mention time), thus John’s very first posting about Jean Nicod’s work in this regard, and a number of mine which followed.

One of these fundamental concepts that calls for closer scrutiny is space itself, or, to be more precise, the nature of *spatial extension,* both of which are abstractions, especially in mathematics (in this regard see Graham Nerlich’s excellent monograph, “The Shape of Space”).

We need to better understand the basis of those two abstractions–space and extension–IMO if we are to make progress on the nature of visual space, or the other sensory modalities that occupy perceptual space as a whole (auditory, tactile, olfactory, gustatory). Abstractions reflect both what they omit and what they assume, and it is the assumptions that we especially need to examine here. While clearly visual space is extended, what about smell? Are smells extended in space?

What we find is that there is a *hierarchy* in perceptual space, one that in man is dominated by visual sensation–what has been called the “dominant visual matrix” by psychologists studying perception. Even sounds are referred to visual loci (“localized”), and I think that can be said of smells, too. But in of themselves it is not clear that even auditory sensations are extended in the same way that visual sensations are, because it is as if when a sound is gone, that part of the “soundscape” is also gone, but that which remains is visual space. In visual space an object may disappear, but the locus it occupied does not also disappear. For example, though we can point to the *visual* source of a sound we hear, we do not point to a sound–even the phrase sounds strange, and ordinary language reveals much about the nature of the perceptual world–or what the man of the street calls the “physical world.””

Bill Rosar: “Part II.

If that is so, why should we assume that physical space has all the properties of visual space and is perhaps not more like smell? Physics is making one big assumption!

I will always remember what Caltech mathematician Richard M. Wilson told me when I consulted him many years ago on ideas I had about how the geometry of visual space reflects changing perspective projections on the retinas. He said, “Keep it simple!” By that he meant being parsimonious and not jumping into fancy mathematical formulations without necessity. I am suggesting that we need to keep the mathematical apparatus here to a minimum, lest its elegance obscure the deeper truth we are seeking–just as Einstein cautioned.

So when we talk about the brain, I think we need to be mindful of what Ray Tallis says about it in his posting “The Disappearance of Appearance,” and just *how* we know about the brain, because we cannot talk about the extended world of physical space and exclude the brain itself from that as a (presumably) physically extended biophysical object. It is not that there is the physical world and there is the brain apart from it.

This ultimately becomes question-begging, because in talking about the brain, we are presupposing physical space, rather than explaining how we have arrived at the notion of physical space and extension. Certainly physical science would deny that physical space is created by the brain. Yet David Bohm would say that physics is largely based on an optical lens-like conception of the physical world, but that physical reality may be more like a hologram (now once again a popular analogy in cosmology because of Leonard Susskind’s theory).

Of course when Karl Pribram then talks about the brain being a mechanism that resolves the holonomic reality (“implicate order”) into a hologram or holographic image (“explicate order”), he forgets that the brain itself would presumably be part of the same holonomic implicate order, and would therefore be resolving itself. By what special power can it perform that trick?

So the very “picture” we have of the brain itself is no different from any other physical entity, as Ray Tallis has been at pains to show.

For now, I’m going to rest with just these rejoinders, and return to your other points later.”

# Geometry of imaginary spaces, by Koenderink

This post is about the article “Geometry of imaginary spaces“,   Journal of  Physiology – Paris, 2011, in press, by Jan Koenderink.

Let me first quote from the abstract (boldfaced  by me):

“Imaginary space” is a three-dimensional visual awareness that feels different from what you experience when you open your eyes in broad daylight. Imaginary spaces are experienced when you look “into” (as distinct from “at”) a picture for instance.

Empirical research suggests that imaginary spaces have a tight, coherent structure, that is very different from that of three-dimensional Euclidean space.

[he proposes the structure of a bundle $E^{2} \times A^{1} \rightarrow E^{2}$, with basis the euclidean plane, “the visual field” and fiber the 1-dimensional affine line, “the depth domain”,]

I focus on the topic of how, and where, the construction of such geometrical structures, that figure prominently in one’s awareness, is implemented in the brain. My overall conclusion—with notable exceptions—is that present day science has no clue.

What is remarkable in this paper? Many many things, here are just three quotes:

–  (p. 3) “in the mainstream account”, he writes, “… one starts from samples of … the retinal “image”. Then follows a sequence of image operations […] Finally there is a magic step: the set of derived images turns into a “representation of the scene in front of you”. “Magic” because image transformations convert structures into structures. Algorithms cannot convert mere structure into quality and meaning, except by magic. […] Input structure is not intrinsically meaningful, meaning needs to be imposed (magically) by some arbitrary format.”

– (p. 4) “Alternatives to the mainstream account have to […] replace inverse optics with “controlled hallucination” [related to this, see the post “The structure of visual space“]

– (p. 5) “In the mainstream account one often refers to the optical structure as “data”, or “information”. This is thoroughly misleading because to be understood in the Shannon (1948) sense of utterly meaningless information. As the brain structures transform the optical structure into a variety of structured neural activities, mainstream often uses semantic terms to describe them. This confuses facts with evidence. In the case of an “edge detector” (Canny, 1986) the very name suggests that the edge exists before being detected. This is nonsensical, the so-called edge detector is really nothing but a “first order directional derivative operator” (Koenderink and van Doorn, 1992). The latter term is to be preferred because it describes the transformation of structure into structure, whereas the former suggests some spooky operation” [related to this, see the tag archive “Map is the territory“]

Related to my  spaces with dilations, let me finally quote from the “Final remarks”:

The psychogenetic process constrains its articulations through probing the visual front end. This part of the brain is readily available for formal descriptions that are close to the neural hardware. The implementation of the group of isotropic similarities, a geometrical object that can  easily be probed through psychophysical means, remains fully in the dark though.

# The gnomon in the greek theater of vision, I

In the post Theatron as an eye I proposed the Greek Theater, or Theatron (as opposed to the “theater in a box”, or Cartesian Theater, see further) as a good model for   vision.

Any model of vision should avoid the homunculus fallacy. What looks less understood is that any good model of vision should avoid the scenic space fallacy. The Cartesian Theater argument against the existence of the homunculus is not, by construction, an argument against the scenic space. Or, in the Cartesian Theater, homunculus and scenic space come to existence in a pair. As a conclusion, it seems that there could not be a model of vision which avoids the homunculus but is not avoiding the scenic space. This observation is confirmed by facts: there is no good, rigorous  model of vision up to date, because all proposed models rely on the a priori existence of a scenic space. There is, on the contrary, a great quantity of experimental data and theoretical partial models which show just how complex the problem of vision is. But, essentially, from a mathematician viewpoint, it is not known how to even formulate the problem of vision.

In the influent paper “The brain a geometry engine”  J. Koenderink proposes that (at least a part of) the visual mechanism is doing a kind of massively parallel computation, by using an embodiment of the geometry of jet spaces (the euclidean infinitesimal geometry of a smooth manifold)  of the scenic space. Jean Petitot continues along this idea, by proposing a neurogeometry of vision based essentially on the sub-riemannian geometry of those jet spaces. This an active mathematical area of research, see for example “Antropomorphic image reconstruction via hypoelliptic diffusion“, by Ugo Boscain et al.

Sub-riemannian geometry is one of my favorite mathematical subjects, because it  is just a  particular model of a metric space with dilations.  Such spaces are somehow fundamental for the problem of vision, I think. Why? because there is behind them a purely relational formalism, called “emergent algebra“, which allow to understand “understanding space” in a purely relational way. Thus I hope emergent algebras could be used in order to formulate the problem of vision as the problem of computing with space, which in turn could be used for getting a good model of vision.

To my surprise, some time ago I have found that this  very complex subject has a respectable age, starting with Pythagora  and Plato!  This is how I arrived to write this blog, as an effort to disseminate what I progressively understand.

This brings me back to the theater and, finally, to gnomon. I cite from previous wiki link:

Hero defined a gnomon as that which, added to an entity (number or shape), makes a new entity similar to the starting entity.

In the greek theater, a gnomon sits in the center of the orchestra (which is the circular place where things happen in the greek thater, later replaced by the scene in the theater in a box). Why?

# Three problems and a disclaimer

In this post I want to summarize the list of problems I am currently thinking about. This is not a list of regular mathematical problems, see the disclaimer on style written at the end of the post.

Here is the list:

1. what is “computing with space“? There is something happening in the brain (of a human or of a fly) which is akin to a computation, but is not a logical computation: vision. I call this “computing with space”. In the head there are a bunch of neurons chirping one to another, that’s all. There is no euclidean geometry, there are no a priori coordinates (or other extensive properties), there are no problems to solve for them neurons, there is  no homunculus and no outer space, only a dynamical network of gates (neurons and their connections). I think that a part of an answer is the idea of emergent algebras (albeit there should be something more than this).  Mathematically, a closely related problem is this: Alice is exploring a unknown space and then sends to Bob enough information so that Bob could “simulate” the space in the lab. See this, or this, or this.

Application: give the smallest hint of a purely relational  model of vision  without using any a priori knowledge of the (euclidean or other) geometry of outer space or any  pre-defined charting of the visual system (don’t give names to neurons, don’t give them “tasks”, they are not engineers).

2. non-commutative Baker-Campbell-Hausdorff formula. From the solution of the Hilbert’s fifth problem we know that any locally compact topological group without small subgroups can be endowed with the structure of a “infinitesimally commutative” normed group with dilations. This is true because  one parameter sub-groups  and Gleason metrics are used to solve the problem.  The BCH formula solves then another problem: from the infinitesimal structure of a (Lie) group (that is the vector space structure of the tangent space at the identity and the maniflod structure of the Lie group) and from supplementary infinitesimal data (that is the Lie bracket), construct the group operation.

The problem of the non-commutative BCH is the following: suppose you are in a normed group with dilations. Then construct the group operation from the infinitesimal data (the conical group structure of the tangent space at identity and the dilation structure) and supplementary data (the halfbracket).

The classical BCH formula corresponds to the choice of the dilation structure coming from the manifold structure of the Lie group.

In the case of a Carnot group (or a conical group), the non-commutative BCH formula should be trivial (i.e. $x y = x \cdot y$, the equivalent of $xy = x+y$ in the case of a commutative Lie group, where by convention we neglect all “exp” and “log” in formulae).

3. give a notion of curvature which is meaningful for sub-riemannian spaces. I propose the pair curvdimension- curvature of a metric profile. There is a connection with problem 1: there is a link between the curvature of the metric profile and the “emergent Reidemeister 3 move” explained in section 6 of the computing with space paper. Indeed, at page 36 there is this figure. Yes, $R^{x}_{\epsilon \mu \lambda} (u,v) w$ is a curvature!

Disclaimer on style. I am not a problem solver, in the sense that I don’t usually like to find the solution of an already formulated problem. Instead, what I do like to do is to understand some phenomenon and prove something about it in the simplest way possible.  When thinking about a subject, I like to polish the partial understanding I have by renouncing to use any “impure” tools, that is any (mathematical) fact which is strange to the subject. I know that this is not the usual way of doing the job, but sometimes less is more.

# Spacebook: a facebook for space

I follow the work of Mark Changizi on vision. Previously I mentioned one of his early papers on this subject, “Harnessing vision for computation” .

One of the applications of computing with space  could be to SHARE THE SPATIAL EXPERIENCE ON THE WEB.

Background. When I was writing the paper on the problem of computing with space, I stumbled upon this article by Mark in Psychology Today

The Problem With the Web and E-Books Is That There’s No Space for Them

The title says a lot. I was intrigued by the following passage

“My personal library serves as extension of my brain. I may have read all my books, but I don’t remember most of the information. What I remember is where in my library my knowledge sits, and I can look it up when I need it. But I can only look it up because my books are geographically arranged in a fixed spatial organization, with visual landmarks. I need to take the integral of an arctangent? Then I need my Table of Integrals book, and that’s in the left bookshelf, upper middle, adjacent to the large, colorful Intro Calculus book.”

So I posted the following comment:  Is your library my library?

“Good point, but you have converted a lot of time into understanding, exploring and using the space of your library. To me the brain-spatial interface of your library is largely incomprehensible. I have to spend time in order to reconstruct it in my head.

Then, your excellent suggestion may give somebody the idea to do a “facebook” for our personal libraries. How to share spatial competences, that is a question!”

In the section 2.7 (“Spacebook”) of the paper on computing with space I mention this as an intriguing application of this type of computing (the name itself was suggested by Mark Changizi after I sent him a first version of the paper).

What more?Again from browsing Mark Changizi site, I learned that in fact this problem of non-spatiality (say) of e-books has measurable effects. Indeed, see this article by Maia Szalavitz

Do E-Books Make It Harder to Remember What You Just Read?

Nice! But in order to do a spacebook we need first to understand the primitives of space (as represented in the human brain) and then how to “port” them by using the web.

# Theatron as an eye

I want to understand what “computing with space” might be. By making  a parallel with the usual computation, there are three ingredients which need to be identified: what are the computing with space equivalents of

1. the universal computing gate (in usual computing this is the transistor)

2. the universal machine (in usual computing this is the Turing machine)

3. what is the universal machine doing by using its arrangement of universal computing gates (in usual computing this is the algorithm).

I think that (3) is (an abstraction of) the activity of map making, or space exploration. The result of this activity is coded by a dilation structure, but I have no idea HOW such a result is achieved. Once obtained though, a mathematical model of the space is the consequence of  a priori assumptions (that we can repeat in principle indefinitely the map making operations) which lead to the emergent algebraic and differential structure of the space.

The universal gate (1), I think, is the dilation gate, or the map-territory relation.

Today I want to pave the way to the discovery of the universal machine (2). This is related to my previous posts The Cartesian Theater: philosophy of mind versus aerography and Towards aerography, or how space is shaped to comply with the perceptions of the homunculus.

My take is that the Greek Theater, or Theatron (as opposed to the “theater in a box”, or Cartesian Theater) is a good model for an universal machine.

For today, I just want to point to the similarities between the theatron and the eye.

The following picture represents the main parts of the theatron (the ancient greek meaning of “theatron” is “place of seeing). In black are written the names of the theatron parts and in red you see the names of the corresponding parts of the eye, according to the proposed similarity.

Let me proceed with the meaning of these words:

– Analemmata means the pedestal of a sundial (related with analemma and analemmatic sundial; basically a theatron is an analemmatic sundial, with the chorus as the gnomon). I suggest to parallel this with the choroid of the eye.

– Diazomata (diazoma means “belt”), proposed to be similar with the retina.

Prohedria (front seating) is a privilege to sit in the first few rows at the bottom of the viewing area. Similar with the fovea (small pit), responsible for sharp central vision.

Skene (tent), the stage building, meant to HIDE the workings  of the actors which are not part of the show, as well as the masks and other materials. When a character dies, it happens behind the skene. Eventually, the skene killed the chorus and  became the stage. The eye equivalent  of this is the iris.

Parodos (para – besides, counter, and ode – song) entrance of the chorus. Eye equivalent is the crystalline lens.

– Orchestra, the ancient greek stage, is the place where the chorus acts, the center of the greek theater. Here we pass to abstraction: the eye correspondent is the visual field.

# Combinatorics versus geometric…

… is like using roman numerals versus using a positional numeral system, like the hindu-arabic numerals we all know very well. And there is evidence that our brain is NOT using combinatorial techniques, but geometric, see further.

What is this post about? Well, it is about the problem of using knowledge concerning topological groups in order to study discrete approximate groups, as Tao proposes in his new course, it is about discrete finitely generated groups with polynomial growth which, as Gromov taught us, when seen from far away they become nilpotent Lie groups, and so on. Only that there is a long way towards these subjects, so please bear me a little bit more.

This is part of a larger project to try to understand approximate groups, as well as normed groups with dilations, in a more geometric way. One point of interest is understanding the solution to the Hilbert’s fifth problem from a more general perspective, and this passes by NOT using combinatorial techniques from the start, even if they are one of the most beautiful mathematical gems which is the solution given by Gleason-Montgomery-Zippin to the problem.

What is combinatorial about this solution? Well, it reduces (in a brilliant way) the problem to counting, by using objects which are readily at hand in any topological group, namely the one-parameter subgroups. There is nothing wrong about this, only that, from this point of view, Gromov’s theorem on groups with polynomial growth appears as magical. Where is this nilpotent structure coming from?

As written in a previous post, Hilbert’s fifth problem without one-parameter subgroups, Gromov’ theorem is saying a profound geometrical thing about a finitely generated group with polynomial growth: that seen from far away this group is self-similar, that is a conical group, or a contractible group w.r.t. any of its dilations. That is all! the rest is just a Siebert’ result. This structure is deeply hidden in the proof and one of my purposes is to understand where it is coming from. A way of NOT understanding this is to use huge chunks of mathematical candy in order to make this structure appear by magic.

I cannot claim that I understand this, that I have a complete solution, but instead, for this post, I looked for an analogy and I think I have found one.

It is the one from the first lines of the post.

Indeed, what is wrong, by analogy, with the roman numeral system? Nothing, actually, we have generators, like I, V, X, L, C, D, M, and relations, like IIII = IV, and so on (yes, they are not generators and relations exactly like in a group sense). The problems appear when we want to do complex operations, like addition of large numbers. Then we have to be really clever and use very intelligent and profound combinatorial arguments in order to efficiently manage all the cancellations and whatnot coming from the massive use of relations. Relations are at very small scale, we have to bridge the way towards large scales, therefore we have to approximate the errors by counting in different ways and to be very clever about these ways.

Another solution for this, historically preferred, was to use a positional number system, which is more geometric, because it exploits a large scale property of natural numbers, which is that their collection is (approximately) self-similar. Indeed, take (as another kind of generators, again not in a group sense), a small set, like B={0, 1, 2, …, 8, 9} and count in base 10, which goes roughly like this: take a (big, preferably) natural number $X$ and do the following

– initialize $i = 1$,

– find the smallest natural power $a_{i}$ of 10 such that $10^{-a_{i}} X$ has a norm smaller than 10, then pick the element $k_{i}$ of $B$ which minimizes the distance to $10^{-a_{i}} X$,

– substract (from the right or the left, it does not matter here because addition of natural numbers is commutative) $10^{a_{i}} k_{i}$ from $X$, and rename the result by $X$,

-repeat until $X \in B$ and finish the algorithm by taking the last digit as $X$.

In the end (remember, I said “roughly”) represent $X$ as a string which codes the string of pairs $(a_{i}, k_{i})$.

The advantage of this representation of natural numbers is that we can do, with controlled precision, the addition of big numbers, approximately. Indeed, take two very big numbers $X, Y$ and take another number, like $10$. Then for any natural $n$ define

$(X+Y)$ approx(n) $= 10^{n} ( [X]_{n} + [Y]_{n})$

where $[X]_{n}$ is the number which is represented as the truncation of the string which represents $X$ up to the first $n$ letters, counting from left to right.
If $n$ is small compared with $X, Y$, then $(X+Y)$ approx(n) is close to the true $X+Y$, but the computational effort for calculating $(X+Y)$ approx(n) is much smaller than the one for calculating $X+Y$.

Once we have this large scale self-similarity, then we may exploit it by using the more geometrical positional numeral system instead of the roman numeral system, that is my analogy. Notice that in this (almost correct) algorithm $10^{a} X$ is not understood as $X+X+....+X$ $10^{a}$ times.

Let me now explain why the positional numeral system is more geometric, by giving a neuroscience argument, besides what I wrote in this previous post: “How not to get bored, by reading Gromov and Tao” (mind the comma!).

I reproduce from the wiki page on “Subitizing and counting

Subitizing, coined in 1949 by E.L. Kaufman et al.[1] refers to the rapid, accurate, and confident judgments of number performed for small numbers of items. The term is derived from the Latin adjective subitus (meaning “sudden”) and captures a feeling of immediately knowing how many items lie within the visual scene, when the number of items present falls within the subitizing range.[1] Number judgments for larger set-sizes were referred to either as counting or estimating, depending on the number of elements present within the display, and the time given to observers in which to respond (i.e., estimation occurs if insufficient time is available for observers to accurately count all the items present).

The accuracy, speed, and confidence with which observers make judgments of the number of items are critically dependent on the number of elements to be enumerated. Judgments made for displays composed of around one to four items are rapid,[2] accurate[3] and confident.[4] However, as the number of items to be enumerated increases beyond this amount, judgments are made with decreasing accuracy and confidence.[1] In addition, response times rise in a dramatic fashion, with an extra 250–350 ms added for each additional item within the display beyond about four.[5]

This is a brain competence which is spatial (geometrical) in nature, as evidenced by Simultanagnosia:

Clinical evidence supporting the view that subitizing and counting may involve functionally and anatomically distinct brain areas comes from patients with simultanagnosia, one of the key components of Balint’s syndrome.[14] Patients with this disorder suffer from an inability to perceive visual scenes properly, being unable to localize objects in space, either by looking at the objects, pointing to them, or by verbally reporting their position.[14] Despite these dramatic symptoms, such patients are able to correctly recognize individual objects.[15] Crucially, people with simultanagnosia are unable to enumerate objects outside the subitizing range, either failing to count certain objects, or alternatively counting the same object several times.[16]

From the wiki description of simultanagnosia:

Simultanagnosia is a rare neurological disorder characterized by the inability of an individual to perceive more than a single object at a time. It is one of three major components of Bálint’s syndrome, an uncommon and incompletely understood variety of severe neuropsychological impairments involving space representation (visuospatial processing). The term “simultanagnosia” was first coined in 1924 by Wolpert to describe a condition where the affected individual could see individual details of a complex scene but failed to grasp the overall meaning of the image.[1]

I rest my case.

# Towards aerography, or how space is shaped to comply with the perceptions of the homunculus

In the previous post

The Cartesian Theater: philosophy of mind versus aerography

I explained why the Cartesian Theater is not well describing the appearance of the homunculus.

A “Cartesian theater”, Dennett proposes, is any theory about what happens in one’s mind which can be reduced to the model of a “tiny theater in the brain where a homunculus … performs the task of observing all the sensory data projected on a screen at a particular instant, making the decisions and sending out commands.”

This leads to infinite regression, therefore any such theory is flawed. One has to avoid the appearance of the homunculus in one’s theory, as a consequence.

The homunculus itself may appear from apparently innocuous assumptions, such as the introduction of any limen (or threshold), like supposing that (from Consciousness Explained (1991), p. 107)

“…there is a crucial finish line or boundary somewhere in the brain, marking a place where the order of arrival equals the order of “presentation” in experience because what happens there is what you are conscious of.”

By consequence such assumptions are flawed. There is no limen, boundary inside the brain (strangely, any assumption which supposes a boundary which separates the individual from the environment is not disturbing anybody excepting Varela, Maturana, or the second order cybernetics).

In the previous post I argued, based on my understanding of the excellent paper of Kenneth R Olwig

that the “Cartesian theater” model is misleading because it neglects to notice that what happens on stage is as artificial as the homunculus spectator, while, in the same time, the theater itself (a theater in a box) is designed for perception.

Therefore, while everybody (?) accepts that there is no homunculus in the brain, in the same time nobody seems to be bothered that always the perception data are modeled as if they come from the stage of the Cartesian theater.

For example, few would disagree that we see a 3-dimensional, euclidean world. But this is obviously not what we see and the proof is that we can be easily tricked by stereoscopy. These are the visual data (together with other, more subtle, auditory, posture and whatnot) which the brain uses to reconstruct the world as seen by a homunculus, created by our illusory image that there is a boundary between us (me, you) and the environment.

You would say: nobody in the right mind denies that the world is 3d, at least our familiar everyday world, not quantum or black holes or other inventions of physicists. I don’t deny it, just notice, like in this previous post, that the space is perceived as it is based on prior knowledge, that is because prior “controlled hallucinations” led consistently to coherent interpretations.

The idea is that in fact there are two things to avoid: one is the homunculus and the other one is the scenic space.

The “scenic space” is itself a model of the real space (does this exists?) and it leads itself to infinite regression. We “learn space” by relating to it and modeling it in our brains. I suppose that all (inside and outside of the brain) complies with the same physical laws and that the rational explanation for the success of the “3d scenic space” (which is consistent with our educated perception, but also with physical phenomena in our world, at least at human scale and range) should come from this understanding that brain processes are as physical as a falling apple and as mathematical as perspective is.

# How not to get bored, by reading Gromov and Tao

This is a continuation of the previous post Gromov’s ergobrain, triggered by the update of Gromov paper on July 27. It is also related to the series of posts by Tao on the Hilbert’s fifth problem.

To put you in the right frame of mind, both Gromov and Tao set the stage for upcoming, hopefully extremely interesting (or “beautiful” on Gromov’s scale: interesting, amusing, amazing, funny and beautiful) developments of their work on “ergosystems” and “approximate groups” respectively.

What can be the link between those? In my opinion, both works refer to the unexplored ground between discrete (with not so many elements) and continuous (or going to the limit with the number of elements of a discrete world).

Indeed, along with my excuses for simplifying too much a very rich text, let me start with the example of the bug on a leaf, sections 2.12, 2.13 in Gromov’s paper). I understand that the bug, as any other “ergosystem” (like one’s brain) would get bored to behave like a finite state automaton crawling on a “co-labeled graph” (in particular on a Cayley graph of a discretely generated group). The implication seems to be that an ergosystem has a different behaviour.

I hardly refrain to copy-paste the whole page 96 of Gromov’s paper, please use the link and read it instead, especially the part related to downsides of Turing modeling (it is not geometrical, in few words). I shall just paste here the end:

The two ergo-lessons one may draw from Turing models are mutually contradictory.
1. A repeated application of a simple operation(s) may lead to something unexpectedly complicated and interesting.
2. If you do not resent the command “repete” and/or are not getting bored by doing the same thing over and over again, you will spend your life in a “Turing loop” of an endless walk in a circular tunnel.

That is because the “stop-function” associated to a class of Turing machines

may grow faster than anything you can imagine, faster than anything expressible by any conceivable formula – the exponential and double exponential functions that appeared so big to you seem as tiny specks of dust compared to this monstrous stop-function. (page 95)

Have I said “Cayley graph”? This brings me to discrete groups and to the work of Tao (and Ben Green and many others). According to Tao, there is something to be learned from the solution of the Hilbert’s fifth problem, in the benefit of understanding approximate groups. (I am looking forward to see this!) There are some things that I understood from the posts of Tao, especially that a central concept is a Gleason metric and its relations with group commutators. In previous posts (last is this) I argue that Gleason metrics are very unlike sub-riemannian distances. It has been unsaid, but obvious to specialists, that sub-riemannian metrics are just like distances on Cayley graphs, so as a consequence Gleason metrics are only a commutative “shadow” of what happens in a Cayley graph when looked from afar. Moreover, in this post concerning the problem of a non-commutative Baker-Campbell-Hausdorff formula it is said that (in the more general world of groups with dilations, relevant soon in this post) the link between the Lie bracket and group commutators is shallow and due to the commutativity of the group operation in the tangent space.

So let me explain, by using Gromov’s idea of boredom, how not to get bored in a Cayley graph. Remember that I quoted a paragraph (from Gromov paper, previous version), stating that an ergosystem “would be bored to death” to add large numbers? Equivalently, an ergosystem would be bored to add (by using the group operation) elements of the group expressed as very long words with letters representing the generators of the group. Just by using “finite state automata” type of reasoning with the relations between generators (expressed by commutators and finitary versions of Gleason like metrics) an ergosystem would get easily bored. What else can be done?

Suppose that we crawl in the Cayley graph of a group with polynomial growth, therefore we know (by a famous result of Gromov) that seen from afar the group is a nilpotent one, more precisely a group with the algebraic structure completely specified by its dilations. Take one such dilation, of coefficient $10^{-30}$ say, and (by an yet unknown “finitization” procedure) associate to it a “discrete shadow”, that is an “approximate dilation” acting on the discrete group itself. As this is a genuinely non-commutative object, probably the algorithm for defining it (by using relations between growth and commutators) would be very much resource consuming. But suppose we just have it, inferred from “looking at the forrest” as an ergosystem.

What a great object would that be. Indeed, instead of getting bored by adding two group elements, the first expressed as product of 200034156998123039234534530081 generators, the second expressed as a product of 311340006349200600380943586878 generators, we shall first reduce the elements (apply the dilation of coefficient $10^{-30}$) to a couple of elements, first expressed as a product of 2 generators, second expressed as a product of 3 generators, then we do the addition $2+3 = 5$ (and use the relations between generators), then we use the inverse dilation (which is a dilation of coefficient $10^{30}$) to obtain the “approximate sum” of the two elements!

In practice, we probably have a dilation of coefficient $1/2$ which could simplify the computation of products of group elements of length $2^{4}$ at most, for example.

But it looks like a solution to the problem of not getting bored, at least to me.

# Braitenberg vehicles, enchanted looms and winnowing-fans

Braitenberg vehicles were introduced in the wonderful book (here is an excerpt which contains enough information for understanding this post):

Vehicles: Experiments in Synthetic Psychology [update: link no longer available]

In the introduction of the book we find the following:

At times, though, in the back of my mind, while I was counting fibers in the visual ganglia of the fly or synapses in the cerebral cortex of the mouse, I felt knots untie,  distinctions dissolve, difficulties disappear, difficulties I had experienced much earlier when I still held my first naive philosophical approach to the problem of the mind.

This is not the first appearance of knots (and related weaving craft) as a metaphor for things related to the brain. A famous paragraph, by Charles Scott Sherrington compares the brain waking from sleep with an enchanted loom

The great topmost sheet of the mass, that where hardly a light had twinkled or moved, becomes now a sparkling field of rhythmic flashing points with trains of traveling sparks hurrying hither and thither. The brain is waking and with it the mind is returning. It is as if the Milky Way entered upon some cosmic dance. Swiftly the head mass becomes an enchanted loom where millions of flashing shuttles weave a dissolving pattern, always a meaningful pattern though never an abiding one; a shifting harmony of subpatterns.

Compare with the following passage (Timaeus 52d and following) from Plato:

…the nurse of generation [i.e. space, chora] …  presented a strange variety of appearances; and being full of powers which were neither similar nor equally balanced, was never in any part in a state of equipoise, but swaying unevenly hither and thither, was shaken by them, and by its motion again shook them; and the elements when moved were separated and carried continually, some one way, some another; as, when grain is shaken and winnowed by fans and other instruments used in the threshing of corn, the close and heavy particles are borne away and settle in one direction, and the loose and light particles in another.

The winnowing-fan (liknon) is important in the Greek mythology, it means also cradle and Plato uses this term with both meanings.

For a mathematician at least, winnowing and weaving are both metaphors of computing with braids: the fundamental group of the configuration space of the grains is the braid group and moreover the grains (trajectories) are the weft, the winnowing-fan is the warp of a loom.

All part of the reason of proposing a tangle formalism for chora and computing with space.

Back to Braitenberg vehicles. Vehicles 2,3,4 and arguably 5 are doing computations with space, not logical computations, by using sensors, motors and connections (that is map-making operations). I cite from the end of Vehicle 3 section:

But, you will say, this is ridiculous: knowledge implies a flow of information from the environment into a living being ar at least into something like a living being. There was no such transmission of information here. We were just playing with sensors, motors and connections: the properties that happened to emerge may look like knowledge but really are not. We should be careful with such words. […]

Meanwhile I invite you to consider the enormous wealth of different properties that we may give Vehicle 3c by choosing various sensors and various combinations of crossed and uncrossed, excitatory and inhibitory, connections.

# Gordon Pask: An essay on the kinetics of language, behaviour and thought

It looks to me that there is a considerable quantity of mathematical structure hidden in the following internal paper from Gordon Pask at System Research

An essay on the kinetics of language, behaviour and thought

I am not impressed by any authority or fashion arguments, my question is the following: is there somebody who said interesting mathematical things about this work?

Thanks to Nick Green for sending the link to the file and for very interesting discussions during the writing of Computing with space.

# Gromov’s Ergobrain

Misha Gromov updated his very interesting “ergobrain” paper

Structures, Learning and Ergosystems: Chapters 1-4, 6

Two quotes I liked: (my emphasis)

The concept of the distance between, say, two locations on Earth looks simple enough, you do not think you need a mathematician to tell you what distance is. However, if you try to explain what you think you understand so well to a computer program you will be stuck at every step.  (page 76)

Our ergosystems will have no explicit knowledge of numbers, except may be for a few small ones, say two, three and four. On the contrary, neurobrains, being physical systems, are run by numbers which is reflected in their models, such as neural networks which sequentially compose addition of numbers with functions in one variable.

An unrestricted addition is the essential feature of “physical numbers”, such as mass, energy, entropy, electric charge. For example, if you bring together $\displaystyle 10^{30}$  atoms, then, amazingly, their masses add up […]

Our ergosytems will lack this ability. Definitely, they would be bored to death if they had to add one number to another $\displaystyle 10^{30}$ times. But the $\displaystyle 10^{30}$ -addition, you may object, can be implemented by $\displaystyle log_{2} 10^{30} \sim 100$ additions with a use of binary bracketing; yet, the latter is a non-trivial structure in its own right that our systems, a priori, do not have.  Besides, sequentially performing even 10 additions is boring. (It is unclear how Nature performs “physical addition” without being bored in the process.) (page 84)

Where is this going? I look forward to learn.

# Topographica, the neural map simulator

The following speaks for itself:

Topographica neural map simulator

“Topographica is a software package for computational modeling of neural maps, developed by the Institute for Adaptive and Neural Computation at the University of Edinburgh and the Neural Networks Research Group at the University of Texas at Austin. The project is funded by the NIMH Human Brain Project under grant 1R01-MH66991. The goal is to help researchers understand brain function at the level of the topographic maps that make up sensory and motor systems.”

From the Introduction to the user manual:

“The cerebral cortex of mammals primarily consists of a set of brain areas organized as topographic maps (Kaas et al. 1997Vanessen et al. 2001). These maps contain systematic two-dimensional representations of features relevant to sensory, motor, and/or associative processing, such as retinal position, sound frequency, line orientation, or sensory or motor motion direction (Blasdel 1992Merzenich et al. 1975Weliky et al. 1996). Understanding the development and function of topographic maps is crucial for understanding brain function, and will require integrating large-scale experimental imaging results with single-unit studies of individual neurons and their connections.”

One of the Tutorials is about the Kohonen model of self-organizing maps, mentioned in the post  Maps in the brain: fact and explanations.

# A difference which makes four differences, in two ways

Gregory Bateson , speaking about the map-territory relation

“What is in the territory that gets onto the map? […] What gets onto the map, in fact, is difference.

A difference is a very peculiar and obscure concept. It is certainly not a thing or an event. This piece of paper is different from the wood of this lectern. There are many differences between them, […] but if we start to ask about the localization of those differences, we get into trouble. Obviously the difference between the paper and the wood is not in the paper; it is obviously not in the wood; it is obviously not in the space between them .

A difference, then, is an abstract matter.

Difference travels from the wood and paper into my retina. It then gets picked up and worked on by this fancy piece of computing machinery in my head.

… what we mean by information — the elementary unit of information — is a difference which makes a difference.

(from “Form, Substance and Difference”, Nineteenth Annual Korzybski Memorial
Lecture delivered by Bateson on January 9, 1970, under the auspices of the Institute of General Semantics, re-printed from the General Semantics Bulletin, no.
37, 1970, in Steps to an Ecology of Mind (1972))

This “difference which makes a difference” statement is quite famous, although sometimes considered only a figure of speach.

I think it is not, let me show you why!

For me a difference can be interpreted as an operator which relates images of the same thing (from the territory) viewed in two different maps, like in the following picture:

This figure is taken from “Computing with space…” , see section 1 “The map is the territory” for drawing conventions.

Forget now about maps and territories and concentrate on this diagram viewed as a decorated tangle. The rules of decorations are the following: arcs are decorated with “x,y,…”, points from a space, and the crossings are decorated with epsilons, elements of a commutative group (secretly we use an emergent algebra, or an uniform idempotent right quasigroup, to decorate arcs AND crossings of a tangle diagram).

What we see is a tangle which appears in the Reidemeister move 3 from knot theory. When epsilons are fixed, this diagram defines a function called (approximate) difference.

Is this a difference which makes a difference?

Yes, in two ways:

1. We could add to this diagram an elementary unknot passing under all arcs, thus obtaining the diagram

Now we see four differences in this equivalent tangle: the initial one is made by three others.
The fact that a difference is selfsimilar is equivalent with the associativity of the INVERSE of the approximate difference operation, called approximate sum.

2. Let us add an elementary unknot over the arcs of the tangle diagram, like in the following figure

called “difference inside a chora” (you have to read the paper to see why). According to the rules of tangle diagrams, adding unknots does not change the tangle topologically (although this is not quite true in the realm of emergent algebras, where the Reidemeister move 3 is an acceptable move only in the limit, when passing with the crossing decorations to “zero”).

By using only Reidemeister moves 1 and 2, we can turn this diagram into the celtic looking figure

which shows again four differences: the initial one in the center and three others around.

This time we got a statement saying that a difference is preserved under “infinitesimal parallel transport”.

So, indeed, a difference makes four differences, in at least two ways, for a mathematician.

If you want to understand more from this crazy post, read the paper 🙂

# Maps in the brain: fact and explanations

From wikipedia

Retinotopy describes the spatial organization of the neuronal responses to visual stimuli. In many locations within the brain, adjacent neurons have receptive fields that include slightly different, but overlapping portions of the visual field. The position of the center of these receptive fields forms an orderly sampling mosaic that covers a portion of the visual field. Because of this orderly arrangement, which emerges from the spatial specificity of connections between neurons in different parts of the visual system, cells in each structure can be seen as forming a map of the visual field (also called a retinotopic map, or a visuotopic map).

The existence of retinotopic maps is a fact, the problem is to explain how they appear and how they function without falling into the homunculus fallacy, see my previous post.

One of the explanations of the appearance of these maps is given by Teuvo Kohonen.

Browse this paper (for precise statements) The Self-Organizing map , or get a blurry impression from this wiki page. The last paragraph from section B. Brain Maps reads:

It thus seems as if the internal representations of information in the brain are generally organized spatially.

Here are some quotes from the same section, which should rise the attention of a mathematician to the sky:

Especially in higher animals, the various cortices in the cell mass seem to contain many kinds of “map” […] The field of vision is mapped “quasiconformally” onto the primary visual cortex. […] in the visual areas, there are line orientation and color maps. […] in the auditory cortex there are the so-called tonotopic maps, which represent pitches of tones in terms of the cortical distance […] at the higher levels the maps are usually unordered, or at most the order is a kind of ultrametric topological order that is not easy interpretable.

Typical for self-organizing maps is that they use (see wiki page) “a neighborhood function to preserve the topological properties of the input space”.

From the connectionist viewpoint, this neighbourhood function is implemented by lateral connections between neurons.

For more details see for example Maps in the Brain: What Can We Learn from Them? by Dmitri B. Chklovskii and Alexei A. Koulakov. Annual Review of Neuroscience 27: 369-392 (2004).

1. The use of a neighbourhood function is much more than just preserving topological information. I tentatively propose that such neighbourhood functions appear out of the need of organizing spatial information, like explained in the pedagogical paper from the post Maps of metric spaces.

2. Just to reason on discretizations (like hexagonal or other) of the plane is plain wrong, but this is a problem encountered in many many places elsewhere. It is wrong because it introduces the (euclidean) space on the back door (well, this and using happily an L^2 space).

# The Cartesian Theater: philosophy of mind versus aerography

Looks to me there is something wrong with the Cartesian Theater term.

Short presentation of the Cartesian Theater, according to wikipedia (see previous link):

The Cartesian theater is a derisive term coined by philosopher Daniel Dennett to pointedly refer to a defining aspect of what he calls Cartesian materialism, which he considers to be the often unacknowledged remnants of Cartesian dualism in modern materialistic theories of the mind.

Descartes originally claimed that consciousness requires an immaterial soul, which interacts with the body via the pineal gland of the brain. Dennett says that, when the dualism is removed, what remains of Descartes’ original model amounts to imagining a tiny theater in the brain where a homunculus (small person), now physical, performs the task of observing all the sensory data projected on a screen at a particular instant, making the decisions and sending out commands.

Needless to say, any theory of mind which can be reduced to the Cartesian Theater is wrong because it leads to the homunculus fallacy: the homunculus has a smaller homunculus inside which is observing the sensory data, which has a smaller homunculus inside which …

According to Dennett, the problem with the Cartesian theater point of view is that it introduces an artificial boundary (from Consciousness Explained (1991), p. 107)

“…there is a crucial finish line or boundary somewhere in the brain, marking a place where the order of arrival equals the order of “presentation” in experience because what happens there is what you are conscious of.”

As far as I understand, this boundary creates a duality: on one side is the homunculus, on the other side is the stage where the sensory data are presented. In particular this boundary acts as a distinction, like in the calculus of indications of Spencer-Brown’ Laws of Form.

This distinction creates the homunculus, hence the homunculus fallacy. Neat!

Why I think there is something wrong with this line of thought? Because of the “theater” term. Let me explain.

The following is based on the article of Kenneth R Olwig

but keep in mind that what is written further represents my interpretation of some parts of the article, according to my understanding, and not the author point of view.

There has been a revolution in theater, started by

“…the early-17th-century court masques (a predecessor of opera) produced by the author Ben Jonson (the leading author of the day after Shakespeare) together with the pioneering scenographer and architect Inigo Jones.
The first of these masques, the 1605 Masque of Blackness (henceforth Blackness ), has a preface by Jonson containing an early use of landscape to mean scenery and a very early identification of landscape with nature (Olwig, 2002, page 80), and Jones’s scenography is thought to represent the first theatrical use of linear perspective in Britain (Kernodle, 1944, page 212; Orgel, 1975).” (p. 521)Ben Johnson,

So? Look!

From the time of the ancient Greeks, theater had largely taken place outside in plazas and market places, where people could circle around, or, as with the ancient Greco-Roman theater or Shakespeare’s Globe, in an open roofed arena. Jones’s masques, by contrast, were largely performed inside a fully enclosed rectangular space, giving him control over both the linear-focused geometrical perspectival organization of the performance space and the aerial perspective engendered by the lighting (Gurr, 1992; Orrell, 1985).” (p. 522, my emphasis)

“Jonson’s landscape image is both enframed by, and expressive of, the force of the lines of perspective that shoot forth from “the eye” – notably the eye of the head of state who was positioned centrally for the best perspectival gaze.” (p. 523, my emphasis)

“Whereas theater from the time of the ancient Greeks to Shakespeare’s Globe was performed in settings where the actor’s shadow could be cast by the light of the sun, Jones’s theater created an interiorized landscape in which the use of light and the structuring of space created an illusion of three dimensional space that shot from the black hole of the individual’s pupil penetrating through to a point ending ultimately in ethereal cosmic infinity. It was this space that, as has been seen, and to use Eddington’s words, has the effect of “something like a turning inside out of our familiar picture of the world” (Eddington, 1935, page 40). It was this form of theater that went on to become the traditional theater in a box’ viewed as a separate imagined world through a proscenium arch.” (p. 526, my emphasis)

I am coming to the last part of my argument: Dennett’ Cartesian Theater is a “theater in a box”. In this type of theater there is a boundary,

“… scenic space separated by a limen (or threshold) from the space of the spectators – today’s `traditional’ performance space [on liminality see Turner (1974)]” (p. 522)

a distinction, as in Dennett argument. We may also identify the homunculus side of the distinction with the head of state.

But this is not all.

Compared with the ancient Greeks theater, the “theater in a box” takes into account the role of the spectator as the one which perceives what is played on stage.

Secondly, the scenic space is not “what happens there”, as Dennett writes, but a construction already, a controlled space, a map of the territory and not the territory itself.

Conclusion: in my view (contradict me please!) the existence of the distinction (limen) in the “Cartesian theater”, which creates the homunculus problem, is superficial. More important is the fact that “Cartesian theater”, as “theater in a box”, is already a representation of perception, having on one side of the limen a homunculus and on the other side a scenic space which is not the “real space” (as for example the collection of electric sparks sent by the sensory organs to the brain) but instead is as artificial as the homunculus, being a space created and controlled by the scenographer.

Litmus test: repeat the reasoning of Dennett after replacing the “theater in a box” preconception of the “theater” by the older theater from the time of ancient Greeks. Can you do it?

On the beautiful idea of “aerography”, later.