Video-to-Sound Tech Helps Visually Impaired Recognize Faces

Neuroscientists have demonstrated that blind individuals use the same brain regions as sighted people to recognize basic faces, even when the facial information is presented through audio rather than the visual cortex. This offers an intriguing insight into neuroplasticity.

The capacity to identify faces is a fundamental trait shared by humans and some distant, socially inclined primate relatives. Notably, certain regions in the brain, such as the fusiform face area (FFA) located at the lower back of the brain in the inferior temporal cortex, become active specifically when faces are observed.

Insights from FFA Activation

Interestingly, the fusiform face area (FFA), as discovered in a 2009 study, is activated not only when people see actual faces but also when they perceive things that somewhat resemble faces, contributing to the phenomenon of pareidolia where faces are perceived in non-living objects. Moreover, this same region becomes active when individuals gain expertise in a specific domain, aiding, for instance, car enthusiasts in visually distinguishing between different models or assisting chess experts in recognizing familiar board configurations.

In a remarkable finding, the FFA also responds in individuals blind from birth. A 2020 MIT study used fMRI scans on blind participants who explored 3D-printed shapes, including faces, hands, chairs, and mazes. Surprisingly, touching these miniature faces activated the FFA in a manner similar to visual stimulation.

So, in a way, the fusiform face area (FFA) appears indifferent to the sensory system providing facial information, and recent research from a neuroscience team at Georgetown University Medical Center supports this notion.

The team enlisted six blind and ten sighted participants and initiated training with a “sensory substitution device.” This device included a head-mounted video camera, blindfold eyepieces, a set of headphones, and a processing computer. The system would receive input from the video camera, translating it into audio. It segmented the field of view into a 64-pixel grid, assigning each pixel a distinct auditory pitch.

Translating Visual Data into Stereo Soundscapes

These pitches were also presented in a stereo soundstage, as detailed in the research paper. For instance, if the image appeared as a dot in the superior right corner of the camera’s field of view, the corresponding sound would be a high-frequency tone mainly delivered through the right headphone. If the dot was in the top middle of the field of view, the sound would be a high-frequency tone, distributed equally through both headphones. In the case of a line at the bottom left corner, the sound would be a blend of low frequencies primarily delivered through the left headphone.

Over ten one-hour sessions, the participants trained with these devices, adapting to “see” with their ears while moving their heads. They were presented with cards featuring simple shapes, including horizontal and vertical lines, various houses, geometric shapes, and basic emoji-style happy and sad faces. Although the training was challenging, by the end of it, all subjects were able to recognize simple shapes with an accuracy exceeding 85%.

During shape recognition testing in an fMRI machine, both the sighted and visually impaired participants exhibited activation of the fusiform face area (FFA) when presented with a basic face shape. Some blind individuals were even able to accurately discern whether the face displayed a happy or sad expression, as demonstrated in a 45-second audio clip from the study, providing an auditory representation of the device.

Geometric Exposure and Fusiform Face Area Development in the Blind

Josef Rauschecker, PhD, DSc, professor of Neuroscience and senior author of the study, remarked in a press release, “Our findings with individuals who are blind suggest that the development of the fusiform face area does not rely on exposure to actual visual faces but rather on exposure to the geometric configurations of faces, which can be conveyed through other sensory modalities.”

Furthermore, the research team observed that sighted subjects predominantly exhibited activation in the right fusiform face area, whereas blind subjects showed activation in the left FFA.

Read the original article on: New atlas

Read more: China’s Ambitious Mars Sample Return Mission and its Potential Impact on the Space Race