U.S.-German Research Proposal: Neurocomputation in the Visual Periphery: Experiments and Models

Peripheral vision comprises over 99.99% of the visual field. Its strengths and limitations strongly constrain visual perception -- what humans can see at a glance, and the processes by which they move their eyes to piece together information about the world. Peripheral vision differs from foveal vision in complex and interesting ways, most importantly due to "crowding," in which identifying a peripheral stimulus can be substantially impaired by the presence of other, nearby stimuli. This project will examine the nature of the encoding in visual cortex, through development and testing of a set of models of peripheral vision. These models will be targeted at answering key questions about the neurobiological mechanisms. The collaborating investigators, in the US and Germany, will develop models and create a benchmark dataset of behavioral results to be explained. The models and dataset will be made freely available, to aid other researchers and to inform the development of applications such as heads up displays and user interfaces. This work will provide insight into what features are encoded in visual cortex, as well as what tradeoffs may have led the visual system to develop that encoding. Understanding those tradeoffs may inform computer vision which, like human vision, faces constraints on processing capacity.

The development of new model variants will be based on insights from neurophysiology, natural image statistics, sparse coding, and the recent success of convolutional neural networks in artificial intelligence. The investigators will gather benchmark behavioral phenomena far richer than existing crowding datasets, through a combination of studying natural image tasks and model-driven experiments. They will then compare predictions of the new models, as well as of Dr. Rosenholtz's existing high-performing model of peripheral vision, on the benchmark dataset. Doing so will identify the best-performing model(s), and answer key questions about the nature of pooling computations and of non-linear operators, and about the complexity, nature, and purpose of the features encoded by peripheral vision.