Selective Hearing Headphones: Hear Clearly in a Crowd

Researchers have integrated AI with standard headphones to isolate the voice of a single speaker in a noisy crowd simply by looking at them. The code for this advanced noise-cancelling system is freely available for anyone interested in building their own version.

Hearing one person in a crowded, noisy environment where many people are talking can be challenging, especially for those with hearing difficulties. Although modern hearing aids use noise-cancelling technology, they cannot completely eliminate background noise.

Enhancing Hearing in Noisy Environments

University of Washington (UW) researchers have created a solution for improving hearing in noisy settings. By equipping regular noise-cancelling headphones with AI, they developed a system that can focus on a speaker’s voice when the wearer looks at them.

“Nowadays, we often associate AI with web-based chatbots that respond to questions,” said Shyam Gollakota, a professor at UW’s Paul G. Allen School of Computer Science and Engineering and a senior author of the study. “However, in this project, we developed AI to enhance the auditory perception of anyone wearing headphones according to their preferences. Our devices allow you to clearly hear a single speaker even in a noisy environment with many people talking.”

The ‘target speech hearing’ (THS) system created by the researchers is straightforward yet highly effective. Standard headphones are equipped with two microphones, one on each earcup.

When the wearer looks at the person they want to hear and presses a button on the side of the headphones for three to five seconds, sound waves from that speaker’s voice reach both microphones simultaneously (within a 16-degree margin of error).

Signal Analysis and Voice Isolation

These signals are sent to an onboard computer, where machine learning software analyzes the speaker’s vocal patterns. The system then isolates the speaker’s voice and channels it through the headphones, even if they move around, while filtering out background noise.

The video below demonstrates the effectiveness of the headphones, showcasing their ability to quickly eliminate environmental noise and focus on the speaker. This includes removing noise from a nearby person talking on their phone indoors and the sound of a very noisy outdoor fountain.

How quickly can the AI process the speaker’s voice and eliminate unwanted sounds? In tests, the researchers discovered that their system exhibited an end-to-end latency of 18.24 milliseconds. For perspective, an eye blink typically lasts between 300 and 400 milliseconds.

This means that there is virtually no delay between looking at someone you want to hear and hearing only their voice through your headphones; the process occurs in real-time.

Evaluation by Participants

The researchers provided their THS system to 21 participants, who assessed the noise suppression capabilities of the headphones in real-world indoor and outdoor environments.

On average, participants rated the clarity of the speaker’s voice nearly twice as high as when it was not processed.

The THS system builds upon the ‘semantic hearing’ technology previously developed by the UW researchers. Similar to THS, this technology utilized an AI algorithm operating on a smartphone wirelessly connected to noise-cancelling headphones. The semantic hearing system could identify specific noises such as birdsong, sirens, and alarms.

At present, the new system can only filter one target speaker at a time and only when there is no other loud voice emanating from the same direction as the speaker. However, if the headphone user is dissatisfied with the sound quality, they have the option to resample the speaker’s voice to enhance clarity.

The researchers are actively working on expanding their system to include earbuds and hearing aids. Furthermore, they have made their THS code publicly accessible on GitHub to encourage further development. It is important to note that the system is not yet available for commercial purchase.

The researchers presented their findings earlier this month at the Association of Computing Machinery (ACM) Computer-Human Interaction (CHI) conference on Human Factors in Computing Systems in Honolulu, Hawai’i, where it received an Honorable Mention. The unpublished research paper is available for review here.

Read the original article on: New Atlas

Read more: Particle Accelerator Tech from CERN now Treats Brain Tumors