Visual perception – solving the inverse problem

Visual perception

By Carly Short

Propose a mechanism by which the visual system can deal with the inverse problem in regard to one of the following perceptual qualities: lightness, colour, geometry, depth, or motion.

Before I propose a mechanism by which the visual system can deal with the inverse problem in regard to lightness, let me first give a short explanation of exactly what the inverse problem actually is. Light is carried in elementary particles called photons. A photon is, therefore, a quantum of radiation energy. Photons of light travelling from a light source, such as the sun, do not directly enter the eye; instead they have a way to travel first and the atmosphere in which they travel directly acts upon them. By the time the photons have reached the retina (the light sensitive layer of tissue at the back of the eye onto which an image of the visual world is formed) some have been absorbed and some have been reflected. This means that the photons which fall onto the retina are in fact a combination of these three sources: illumination (light source), reflectance (from objects in the environment), and transmittance (through the atmosphere). The inverse problem refers to the fact that we cannot logically reverse this entanglement of stimuli at the retina to understand what is actually in the real world (Zygmunt, 2001).  Images are not something that exist in the ‘real external world’. They are, instead, biologically created. But if we cannot know what exactly is found in the ‘real physical world’, if the ‘real world’ is essentially hidden, then how can we carry out successful behaviours? Since we know that we can indeed operate successfully in our day to day lives, we can deduce that evolving a way of dealing with the inverse problem has determined both how we see and what we see.

Our visual percepts are not an accurate representation of the physical world (which can be physically measured). In other words, we don’t see physical qualities in the same way physical instruments measure them (Gilchrist et al, 1983). The lightness we perceive for example is dependent on the context in which we are seeing it. The ‘Cornsweet edge’ and the ‘Mach bands’ are both examples of the difference between measured luminance and the lightness we actually see. We see grey, not as grey, but as black or as white because of the context in which the grey is represented.  Why does this happen? It can be explained by thinking about a mechanism the visual system has evolved in order to circumvent the inverse problem. The significance of retinal luminance for behaviour in the real world is inherently uncertain due to the inverse problem; we cannot know the luminance of an object directly. And yet we must behave in terms of these object properties in the real world. So, lightness values are assigned a value based on past experience, based on reproductive success. Any scene can be divided into patches and it is likely that nearby image points will be the same or very similar. The role of the physical world is therefore to provide feedback, via reproductive rates, about the relative success of visual perceptions and visually guided behaviours. It is essentially a trial and error system but tracking reproductive success in this way aligns lightness percepts on a subjective scale according to their impact on reproductive success.


Gilchrist, Alan L.; Jacobsen, Alan. Lightness constance through a veiling luminance. Journal of Experimental Psychology:Human Perception and Performance, Vol 9(6), Dec 1983, 936-944.

Zygmunt Pizlo. Perception as an inverse problem. Elsevier. Vision Research, Vol 41(24), Nov 2001, 3145-3161.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s