# The structure of visual space

Mark Changizi has an interesting post “The Visual Nerd in You Undestands Curved Space” where he explains that spherical geometry is relevant for the visual perception.

At some point he writes a paragraph which triggered my post:

Your visual field conforms to an elliptical geometry!

(The perception I am referring to is your perception of the projection, not your perception of the objective properties. That is, you will also perceive the ceiling to objectively, or distally, be a rectangle, each angle having 90 degrees. Your perception of the objective properties of the ceiling is Euclidean.)

Is it true that our visual perception senses the Euclidean space?

Look at this very interesting project

The structure of optical space under free viewing conditions

and especially at this paper:

The structure of visual spaces by J.J. Koenderink, A.J. van Doorn, Journal of mathematical imaging and vision, Volume: 31, Issue: 2-3 (2008), pp. 171-187

In particular, one of the very nice things this group is doing is to experimentally verify the perception of true facts in projective geometry (like this Pappus theorem).

From the abstract of the paper: (boldfaced by me)

The “visual space” of an optical observer situated at a single, fixed viewpoint is necessarily very ambiguous. Although the structure of the “visual field” (the lateral dimensions, i.e., the “image”) is well defined, the “depth” dimension has to be inferred from the image on the basis of “monocular depth cues” such as occlusion, shading, etc. Such cues are in no way “given”, but are guesses on the basis of prior knowledge about the generic structure of the world and the laws of optics. Thus such a guess is like a hallucination that is used to tentatively interpret image structures as depth cues. The guesses are successful if they lead to a coherent interpretation. Such “controlled hallucination” (in psychological terminology) is similar to the “analysis by synthesis” of computer vision.

So, the space is perceived to be euclidean based on prior knowledge, that is because prior controlled hallucinations led consistently to coherent interpretations.

# Maps in the brain: fact and explanations

From wikipedia

Retinotopy describes the spatial organization of the neuronal responses to visual stimuli. In many locations within the brain, adjacent neurons have receptive fields that include slightly different, but overlapping portions of the visual field. The position of the center of these receptive fields forms an orderly sampling mosaic that covers a portion of the visual field. Because of this orderly arrangement, which emerges from the spatial specificity of connections between neurons in different parts of the visual system, cells in each structure can be seen as forming a map of the visual field (also called a retinotopic map, or a visuotopic map).

The existence of retinotopic maps is a fact, the problem is to explain how they appear and how they function without falling into the homunculus fallacy, see my previous post.

One of the explanations of the appearance of these maps is given by Teuvo Kohonen.

Browse this paper (for precise statements) The Self-Organizing map , or get a blurry impression from this wiki page. The last paragraph from section B. Brain Maps reads:

It thus seems as if the internal representations of information in the brain are generally organized spatially.

Here are some quotes from the same section, which should rise the attention of a mathematician to the sky:

Especially in higher animals, the various cortices in the cell mass seem to contain many kinds of “map” […] The field of vision is mapped “quasiconformally” onto the primary visual cortex. […] in the visual areas, there are line orientation and color maps. […] in the auditory cortex there are the so-called tonotopic maps, which represent pitches of tones in terms of the cortical distance […] at the higher levels the maps are usually unordered, or at most the order is a kind of ultrametric topological order that is not easy interpretable.

Typical for self-organizing maps is that they use (see wiki page) “a neighborhood function to preserve the topological properties of the input space”.

From the connectionist viewpoint, this neighbourhood function is implemented by lateral connections between neurons.

For more details see for example Maps in the Brain: What Can We Learn from Them? by Dmitri B. Chklovskii and Alexei A. Koulakov. Annual Review of Neuroscience 27: 369-392 (2004).

1. The use of a neighbourhood function is much more than just preserving topological information. I tentatively propose that such neighbourhood functions appear out of the need of organizing spatial information, like explained in the pedagogical paper from the post Maps of metric spaces.

2. Just to reason on discretizations (like hexagonal or other) of the plane is plain wrong, but this is a problem encountered in many many places elsewhere. It is wrong because it introduces the (euclidean) space on the back door (well, this and using happily an L^2 space).

# The Cartesian Theater: philosophy of mind versus aerography

Looks to me there is something wrong with the Cartesian Theater term.

Short presentation of the Cartesian Theater, according to wikipedia (see previous link):

The Cartesian theater is a derisive term coined by philosopher Daniel Dennett to pointedly refer to a defining aspect of what he calls Cartesian materialism, which he considers to be the often unacknowledged remnants of Cartesian dualism in modern materialistic theories of the mind.

Descartes originally claimed that consciousness requires an immaterial soul, which interacts with the body via the pineal gland of the brain. Dennett says that, when the dualism is removed, what remains of Descartes’ original model amounts to imagining a tiny theater in the brain where a homunculus (small person), now physical, performs the task of observing all the sensory data projected on a screen at a particular instant, making the decisions and sending out commands.

Needless to say, any theory of mind which can be reduced to the Cartesian Theater is wrong because it leads to the homunculus fallacy: the homunculus has a smaller homunculus inside which is observing the sensory data, which has a smaller homunculus inside which …

According to Dennett, the problem with the Cartesian theater point of view is that it introduces an artificial boundary (from Consciousness Explained (1991), p. 107)

“…there is a crucial finish line or boundary somewhere in the brain, marking a place where the order of arrival equals the order of “presentation” in experience because what happens there is what you are conscious of.”

As far as I understand, this boundary creates a duality: on one side is the homunculus, on the other side is the stage where the sensory data are presented. In particular this boundary acts as a distinction, like in the calculus of indications of Spencer-Brown’ Laws of Form.

This distinction creates the homunculus, hence the homunculus fallacy. Neat!

Why I think there is something wrong with this line of thought? Because of the “theater” term. Let me explain.

The following is based on the article of Kenneth R Olwig

but keep in mind that what is written further represents my interpretation of some parts of the article, according to my understanding, and not the author point of view.

There has been a revolution in theater, started by

“…the early-17th-century court masques (a predecessor of opera) produced by the author Ben Jonson (the leading author of the day after Shakespeare) together with the pioneering scenographer and architect Inigo Jones.
The first of these masques, the 1605 Masque of Blackness (henceforth Blackness ), has a preface by Jonson containing an early use of landscape to mean scenery and a very early identification of landscape with nature (Olwig, 2002, page 80), and Jones’s scenography is thought to represent the first theatrical use of linear perspective in Britain (Kernodle, 1944, page 212; Orgel, 1975).” (p. 521)Ben Johnson,

So? Look!

From the time of the ancient Greeks, theater had largely taken place outside in plazas and market places, where people could circle around, or, as with the ancient Greco-Roman theater or Shakespeare’s Globe, in an open roofed arena. Jones’s masques, by contrast, were largely performed inside a fully enclosed rectangular space, giving him control over both the linear-focused geometrical perspectival organization of the performance space and the aerial perspective engendered by the lighting (Gurr, 1992; Orrell, 1985).” (p. 522, my emphasis)

“Jonson’s landscape image is both enframed by, and expressive of, the force of the lines of perspective that shoot forth from “the eye” – notably the eye of the head of state who was positioned centrally for the best perspectival gaze.” (p. 523, my emphasis)

“Whereas theater from the time of the ancient Greeks to Shakespeare’s Globe was performed in settings where the actor’s shadow could be cast by the light of the sun, Jones’s theater created an interiorized landscape in which the use of light and the structuring of space created an illusion of three dimensional space that shot from the black hole of the individual’s pupil penetrating through to a point ending ultimately in ethereal cosmic infinity. It was this space that, as has been seen, and to use Eddington’s words, has the effect of “something like a turning inside out of our familiar picture of the world” (Eddington, 1935, page 40). It was this form of theater that went on to become the traditional `theater in a box’ viewed as a separate imagined world through a proscenium arch.” (p. 526, my emphasis)

I am coming to the last part of my argument: Dennett’ Cartesian Theater is a “theater in a box”. In this type of theater there is a boundary,

“… scenic space separated by a limen (or threshold) from the space of the spectators – today’s `traditional’ performance space [on liminality see Turner (1974)]” (p. 522)

a distinction, as in Dennett argument. We may also identify the homunculus side of the distinction with the head of state.

But this is not all.

Compared with the ancient Greeks theater, the “theater in a box” takes into account the role of the spectator as the one which perceives what is played on stage.

Secondly, the scenic space is not “what happens there”, as Dennett writes, but a construction already, a controlled space, a map of the territory and not the territory itself.

Conclusion: in my view (contradict me please!) the existence of the distinction (limen) in the “Cartesian theater”, which creates the homunculus problem, is superficial. More important is the fact that “Cartesian theater”, as “theater in a box”, is already a representation of perception, having on one side of the limen a homunculus and on the other side a scenic space which is not the “real space” (as for example the collection of electric sparks sent by the sensory organs to the brain) but instead is as artificial as the homunculus, being a space created and controlled by the scenographer.

Litmus test: repeat the reasoning of Dennett after replacing the “theater in a box” preconception of the “theater” by the older theater from the time of ancient Greeks. Can you do it?

On the beautiful idea of “aerography”, later.

# Koenderink and Changizi

Jan Koenderink is a leading researcher in vision. He proposed the concept of
“scale-space representation” in relation to the understanding of how the front-end visual system works.

His paper “The brain a geometry engine” starts with:

According to Kant, spacetime is a form of the mind. If so, the brain must be a geometry engine. This idea is taken seriously, and consequently the implementation of space and time in terms of machines is considered. This enables one to conceive of spacetime as really embodied.

Later he writes:

There may be a point in holding that many of the better-known brain processes are most easily understood in terms of differential geometrical calculations running on massively parallel processor arrays whose nodes can be understood quite directly in terms of multilinear operators (vectors, tensors, etc).
In this view brain processes in fact are space.

This is a very interesting idea! As far as I understand, Koenderink is saying that somehow brain processes involved in vision and (external) space are similar!

In my opinion this is something to explore. However, my take is that this superb idea is clouded by his reliance on linear algebra and differential calculus of the exterior euclidean space (see “vectors, tensors, etc” as well as his derivation of the gaussian filter from invariance with respect to the same euclidean structure). If said brain processes are space and if those brain processes are a kind of computation (in a sense to be explained later) then space should appear as the result of a computation in the front-end visual system. No euclidean a priori!

Are those brain processes a kind of computation? The answer depends on what computation means. Anyway, nobody doubts that logical boolean computations are orthodox computations.

See then the following paper by Mark Changizi “Harnessing vision for computation” or check this Wired post

Scientists Build Visual Circuits to Harness your Brain’s GPU”

The abstract of the paper is:

Might it be possible to harness the visual system to carry out artificial computations, somewhat akin to how DNA has been harnessed to carry out computation? I provide the beginnings of a research programme attempting to do this. In particular, new techniques are described for building `visual circuits’ (or `visual software’) using wire, NOT, OR, and AND gates in a visual modality such that our visual system acts as `visual hardware’ computing the circuit, and generating a resultant perception which is the output

My conclusion: this is experimental proof that at least some brain processes related to vision can do something which simulates logical computation.

# Computing with space

This is the first in a series of postings concerning computing with space. I shall try to give a gentle introduction to – and later discussion around – the ideas from the paper

Computing with space: a tangle formalism for chora and difference