Researchers elucidate the mechanisms that facilitate accurate identification of moving images

An international collaboration elucidates the mechanisms that facilitate accurate identification of moving images. The findings have been published in Nature Communications

Imagine meeting a friend on the street, and imagine that with every step they take, your visual system has to process their image from scratch in order to recognize them. Now imagine if the same thing were to happen for every object and creature that moves around us. We would live in a constant state of uncertainty and inconsistency. Luckily, that is not the case.

Our visual system is able to retain information obtained in motion, thereby presenting us with a more consistent picture of our surroundings. These are the findings of a study conducted by SISSA, in collaboration with the University of Pennsylvania and Katholieke Universiteit Leuven and published in Nature Communications, which explains the neuronal underpinnings of this phenomenon.

One of the biggest challenges of all the sensory systems is to maintain a consistent representation of our surroundings, despite the constant changes taking place around us. The same holds true for the visual system. Just look around us: objects, animals, people, all on the move. We ourselves are moving. This triggers rapid fluctuations in the signals acquired by the retina, and until now it was unclear whether the same type of variations apply to the deeper layers of the visual cortex, where information is integrated and processed. If this was the case, we would live in tremendous confusion."

Davide Zoccolan, Director of the Visual Neuroscience Laboratory, Scuola Internazionale Superiore di Studi Avanzati

It has been known for a while that the signals produced by the retina following presentation of visual stimuli reach a set of consecutive processing stages within the visual cortex, arranged according to a finely-tuned hierarchy. It is this processing sequence that enables us to recognize an object or a face and to do so irrespective of its angle or position. This has been demonstrated in the case of static stimuli and can be explained by the invariance in the encoding of the images that is gradually built up along the cortical hierarchy.

To investigate the existence of a similar process in dynamic situations, researchers from SISSA, University of Pennsylvania (Penn), and Katholieke Universiteit Leuven (KU Leuven), led by Zoccolan, analyzed the signals produced by neurons across different visual cortical areas in rodents following presentation of dynamic visual stimuli. The findings have been published in Nature Communications.

"We used three distinct datasets: one collected by Liviu Soltuzu at SISSA, one collected by Kasper Vinken at KU Leuven in the group led by Hans Op de Beeck and one made freely available by the Allen Institute for Brain Science in Seattle," said the scientist. "The visual stimuli used in each were of different types. In SISSA, we created dedicated video clips showing objects moving at different speeds. The other datasets were acquired using various kinds of clips, including from films."

Next, the researchers analyzed the signals registered in the different areas of the visual cortex through a combination of sophisticated algorithms and models developed by Eugenio Piasini and Vijay Balasubramanian from Penn in collaboration with SISSA scientists (Liviu Soltuzu, Paolo Muratore and Riccardo Caramellino). The researchers developed a theoretical framework to help connect the images in the movies to the activity of specific neurons in order to determine how neural signals evolve over different time scales.

"The art in this science was figuring out an analysis method to show that the processing of visual images is getting slower as you go deeper and deeper in the brain," explains Balasubramanian. "Different levels of the brain process information over different time scales–some things could be more stable, some quicker. It's very hard to tell if the timescales are changing across the brain, so our contribution was to devise a method of doing this."

The results remained consistent, irrespective of the nature of the visual stimuli: "We observed an increased persistence of neuronal responses recorded in deeper stages of the visual system, a sort of 'perceptual constancy' that guarantees some amount of stability in the encoding of visual information and eliminates the fluctuations observed in earlier visual areas" explains Zoccolan. "And not only that. We also noticed a form of 'intrinsic' persistence that increased along the hierarchy of visual areas. In the deeper areas, the neural response remains for a few hundred milliseconds even after the stimulus disappears, which guarantees a minimum duration for the encoding of the images, and this in turn ensures that the information is properly processed and that the reaction to the stimulus is correctly calibrated."