Brain Waves Transformed into Spoken Words in AI Breakthrough for Paralysis

Researchers from California have created an AI-driven system that enables paralyzed individuals to speak naturally in real time, using their own voices.

The innovative technology developed by teams from the University of California, Berkeley, and the University of California, San Francisco, utilizes brain-computer interfaces that can measure neural activity, combined with AI that learns to replicate the unique sound of a patient’s voice.

This advancement significantly surpasses earlier developments in brain-computer interfaces for speech synthesis, which were still in their infancy as recently as last year.

New Streaming Method Enables Natural Speech

“Our streaming method delivers the same rapid speech decoding capabilities found in devices like Alexa and Siri, but applied to neuroprosthetics,” said Gopala Anumanchipalli, an assistant professor of electrical engineering and computer sciences at UC Berkeley and co-principal investigator of the study published this week in Nature Neuroscience. “By using a similar algorithm, we decoded neural data and, for the first time, enabled near-instant voice streaming. This breakthrough leads to a more natural, fluent synthesis of speech.”

What makes this technology particularly exciting is its ability to function effectively with various brain-sensing interfaces. These include high-density electrode arrays that record neural activity directly from the brain’s surface (like the setup used in this study), as well as microelectrodes that penetrate the brain and non-invasive Surface Electromyography (sEMG) sensors placed on the face to track muscle movements.

Here’s how it works: First, the neuroprosthesis collects neural data from the motor cortex, the part of the brain that controls speech production. AI then decodes this data into speech. “What we’re decoding happens after a thought is formed, after the person decides what to say, what words to use, and how to move their vocal muscles,” explained co-author Cheol Jun Cho.

The AI system was trained using brain data captured while patients silently attempted to speak words displayed on a screen. This allowed the researchers to map the brain activity associated with specific words.

Using the Patient’s Own Voice for Speech Output

Additionally, a text-to-speech model, developed using the patient’s pre-injury voice, generates the audio you hear when the patient ‘speaks.’

In the demonstration of the proof of concept, the resulting speech isn’t yet flawless or perfectly paced, but it is remarkably close. The system begins decoding brain signals and generating speech almost instantly—within a second—compared to 8 seconds in a previous study conducted in 2023.

This breakthrough holds tremendous potential to improve the quality of life for people with paralysis, ALS, and similar conditions, allowing them to communicate everything from daily needs to complex ideas and engage with loved ones in a more natural way.

The researchers’ next steps will focus on enhancing the AI’s processing speed to generate speech even faster and explore ways to make the synthesized voice sound more expressive.

Read the original article on: New Atlas

Read more: Directly Transforming Skin Cells into Brain Cells Achieves a Tenfold Increase in Success