Gallant: Neural codes for natural vision

U. C. Berkeley

Much of our understanding of neural codes for vision stem from controlled studies that differ from natural vision in two important respects: natural images are more complex than the artificial images typically used, and eye movements are an integral part of natural vision but are usually prohibited in experiments. In order to examine the neural codes underlying natural vision, I have recorded from visual cortex of awake monkeys freely viewing natural scenes. The animal's eyes were tracked using a scleral search coil, so the location of a cell's classical receptive field (CRF) within the image could be determined continuously. In several controlled viewing experiments, artificial and natural stimuli were presented in and around the CRF while the animal performed a fixation task. The controlled stimuli included: (1) artificial patterns such as 2D gratings; (2) natural image patches which had fallen in the CRF of the cell during free viewing; (3) a review movie recreating the spatio-temporal sequence of natural image patches which had fallen in the CRF during free viewing.

Many V1 cells show substantial modulation of activity correlated with changes in fixation made during free viewing. V1 cells can also be driven by artificial and natural images flashed in the CRF. For some V1 cells, review movies produce reliable temporal responses across repeated presentations. In contrast, many extrastriate cells show little or no modulation correlated with changes in fixation made during free viewing, though natural and artificial images can produce robust responses when flashed under controlled viewing conditions.

These data suggest several important aspects of neural codes for vision:

1. Though cells in area V1 could support a mean-rate code, there is as yet little evidence for such a code in extrastriate cortex, suggesting the possibility of a more complex code in these areas.
2. For some V1 cells, review movies produce very reliable responses that are correlated with those obtained during free viewing. These data provide a lower bound on the potential reliability of a temporal code for natural vision in area V1.
3. More generally, the results emphasize the importance of nonlinear processes in vision, especially in extrastriate areas.