A model of human vision based on sub-Riemannian geometry
We look for an explanation for certain visual illusions and for the capacity of human brain to fill in the gaps in images.
The anthropomorphic algorithm we propose is not (yet) more efficient that more standard methods (segmentation, wavelets,…).
Goes back to Hoffman, rediscovered by Petitot, refined by Citti and Sarti, Agrachev, Charlot-Gauthier-Rossi. I will also present results by Yuri Sachkov, discovered indepedently by Duits.
1. From neurophysiology to the model
In the V1 visual cortex, neurons are sensible to both position and direction. So the brain storeds an image as a set of points and directions, i.e. a subset of .
An image is reconstructed by minimizing the energy necessary to excite groups of neurons that are not excited by the image in .
Huber and Wiesel (Nobel prize 1981) observed that neurons split into groups each of which is sensitive to a specific direction. These groups are called orientation columns. They are in turn grouped into hypercolumns. There are both horizontal and vertical connections between orientation columns.
1.2. Geometry of
Every curve in has a canonical lift. The lifts of plane curves are in 1-1 correspondance with curves tangent to a contact structure on .
1.3. How V1 reconstructs an interrupted curve
By minimizing something. What ? When moving objects with the hands, the brain minimizes a compromise between energy and the stress of muscles (external cost). For reconstruction of images, the (internal) minimized cost is the energy necessary to activate neurons that are not naturally activated by the image of the interrupted curve. Given a neuron that is already active, it is easy to activate nearby neurons, where proximity in measured in brain geometry, i.e. by connections in hypercolumns.
The simplest cost is the Riemannian length of curves which are lifts of planar curves. This leads to a problem in sub-Riemannian geometry on a Lie group, the roto-translation group of the plane. There is only one left-invariant cost, up to planar dilations (Agrachev). One can replace length by energy (integral of squared derivative).
1.4. Advantages of this cost
In terms of planar geometry, this provides a compromise between length and curvature of the planar curve.
Minimizers exist in the function space of absolutely continuous horizontal curves.
There is a natural hypo-elliptic diffusion semi-group, which can be used for reconstruction.
1.5. Drawbacks of this cost
There are minimizers with cusps, which are not observed in psychological experiments.
1.6. The Citti-Sarti proposition to avoid cusps
Consider only curves in whose speed has always non-vanishing component. One looses existence of minimizers (observed by all the authors above). Minimizing sequences tend to converge to curves which do not statisfy the boundary condition any more.
Theorem 1 Classification of the boundary conditions for which existence of minimizers exist.
2. Reconstruction of curves
2.1. Computation of Sub-Riemannian distance
The Hamiltonian flow is closely connected to the phase portrait of the integrable pendulum. Closed trajectories of the pendulum correspond to fronts (alternating cusps). Other trajectories correspond to kinds of cycloids. The issue of optimality of trajectories is hard (Sachkov and Moisseev).
The cut locus of a point is genuinely -dimensional, unlike for the Heisenberg left-invariant metric.
2.2. Preliminary reconstruction results by Sachkov
Sachkov took an image made of level sets of a function, corrupted it by rubbing out big oval parts. His reconstruction manages to connect level sets corresponding ot the same level. How ?
2.3. Sachkov’s algorithm
Input is the image of level sets of a function.
1. Smooth the image with a Gaussian to get a Morse function (physiologists argue that such a smoothing indeed occurs in the human eye).
2. Lift the image to , get a smooth surface (resolution of singularities by blow-up), view it as a generalized function on . Use it as an initial condition for the hypo-elliptic heat equation (i.e. perform convolution with the heat kernel).
3. Project down onto by integrating along fibers. Output.
2.4. Underlying mathematical results
Theorem 2 Explicit expressions for the sub-Riemannian heat kernels on low dimensional Lie groups.
2.5. Numerical implementation
Attemps by Gauthier. There does not seem to be convergence results for the numerical computation of the hypo-elliptic diffusion. Methods like finite elements do not apply due to non-commutativity. Gauthier used Fourier analysis and the expression of the heat kernel.
The images produced look very good (like a hand-drawn picture), even when a large part of the original image was corrupted, but computation is very expensive at present.