How does the brain recognise images without colour? A recent collaborative study led by the Project Prakash research team in India and the United States has provided a potential explanation. In this study, the researchers report experimental results from newly-sighted Indian children and present a computational model, providing a candidate account for why some aspects of human visual development proceed in the way they do. 

Specifically, why do infants start with poor colour vision before being able to perceive the world in its full richness? 

In 2005, Project Prakash, a non-profit organisation for visually impaired children, was launched in New Delhi by Pawan Sinha, Professor, Massachusetts Institute of Technology (MIT), USA. Priti Gupta, an Indian research scientist at Project Prakash and one of the lead authors of this study, was involved in the behavioural analysis of experimental data on 10 children treated for congenital cataracts and 10 children with normal sight. Gupta says, ​“Working closely with these children have deeply inspired me and made me grow as a human being.”

Researchers in New Delhi observed that children treated for congenital cataracts had difficulties recognising images presented in black-and-white after surgery. Based on this experimental data, the researchers hypothesised that during the early development of vision in the immature retina, the brain learns to identify objects based on light intensity (greyscale) rather than relying primarily on colour. When this early visual experience is disrupted, as in children with congenital blindness, there is reduced resilience to the removal of colour compared to children with normal sight. This hypothesis was corroborated by computational simulations performed at Sinha’s lab in MIT. 

The researchers at MIT used a standard convolutional neural network, AlexNet as a computational vision model. This model was trained with different types of input to recognise objects. To simulate the normal developmental progression, the model was initially trained with only greyscale images, with colour images introduced later. This model mimics the developmental progression seen in newborns, where cone cells are not fully developed. In order to understand visual recognition among children with congenital cataracts, another training regimen involved colour images. 

Researchers found that the model mimicking normal development process could accurately recognise objects presented in either black-and-white or colour and was also resilient to other colour manipulations. But the model trained only on colour images showed poor generalisation to greyscale or hue-manipulated images. The strong generalisation observed in the greyscale-to-colour model stemmed from the sequential training of black-and-white images followed by colour images. Conversely, when the training order was reversed, coloured followed by black-and-white images — the model showed inferior generalisation to greyscale or hue-manipulated images.

Using both experimental data and computational simulations, the Project Prakash team have provided evidence that the temporal ordering of black-and-white followed by coloured images is important for normal visual development and the subsequent recognition of black-and-white patterns. Albert Yonas, Professor Emeritus at the University of Minnesota is involved in understanding perceptual development, layout perception in infants and also in the treatment of prosopagnosia (face blindness) and autism spectrum disorder in children.

He says, ​“What would seem to be deficits — low contrast sensitivity, and acuity in the early months of life — lead to the ability to use global information to recognise faces. The findings of the study suggest that the slow development of colour vision makes it possible to recognise objects under conditions where colour information is minimised”. 

In the context of late sight onset, an initial period of reduced colour exposure after surgery could improve later classification robustness. Incorporating these insights into training mechanism could improve the generalisation of machine vision systems. These findings also have broader implications for understanding the origin of some key organisational principles of the mammalian visual system. Narayanan Srinivasan, Professor, Indian Institute of Technology, Kanpur, comments:

The study allows us to understand the use of colour and greyscale information in categorisation in both typically developing and late-sighted individuals.