byKrystal Kasal, Medical Xpress

Framework of a real-time BCI for decoding Chinese sentences. Credit:Science Advances(2025). DOI: 10.1126/sciadv.adz9968

Researchers in Shanghai have reported in a study, recentlypublishedinScience Advances, that they've successfully decoded Mandarin Chinese language in real time with the help of a brain-computer interface (BCI) framework, a first for BCIs working with tonal languages. The participant involved in the study was also capable of controlling a robotic arm and digital avatar and interacting with a large language model using this new system.

While most people may not want a computer reading their mind, those who are unable to speak due to neurological conditions, like strokes or amyotrophic lateral sclerosis (ALS), need to find alternative ways to communicate. SpeechdecodingBCIs, capable of decoding neural signals, offer a promising way to restore communication in such individuals. In addition to communication, BCIs also offer ways to control devices directly through thought. This is particularly helpful for neurological conditions in which disabilities extend beyondspeechloss.

These types of devices are not exactly a novel technology, but most BCI speech decoding research has focused on English, a non-tonal language.

"A leading approach focuses on the ventral sensorimotor cortex, which encodes articulatory kinematic trajectories. Neural signals from this region can be transformed into discrete linguistic units or articulatory gesture parameters and subsequently synthesized into words, sentences, or sounds. This strategy is especially suitable for individuals with intact speech motor areas, aiming to re-enable functional communication.

"Recent advances in English language decoding have enabled real-time translation of brain activity into text or speech for patients with severe dysarthria caused by conditions such asamyotrophic lateral sclerosis(ALS) or brainstem stroke," the study authors write.

Uniqueness of Chinese and cortical electrodes distinguishing Chinese syllables and tones. Credit:Science Advances(2025). DOI: 10.1126/sciadv.adz9968

Advances in BCIs capable of decoding tonal languages, like Mandarin, have been more limited. Because Mandarin is a tonal, monosyllabic language with high homophone density, speech decoding is more challenging. Some previous studies have decoded small sets of Mandarin syllables or tones, but not the full range needed for practical use and not in real time.

However, aclinical studyon an epilepsy patient has enabled the researchers involved in the new study to take a different approach. The study, conducted on a 43-year-old woman, used an implanted 256-channel high-density electrocorticography (ECoG) array to monitor and recordneural signalsthrough a series of single-character and sentence reading tasks over 11 days. A 3-gram Mandarin language model was integrated to improve sentence decoding.

The team says analysis of the ECoG signals revealed distinct neural correlates for syllable and tone processing. The system achieved a median syllable identification accuracy of 71.2% in single-character tasks. Real-time sentence decoding reached 73.1% character accuracy with a language model, and a communication rate of 49.7 characters per minute.

"Our study demonstrates that combining high-density, ultraconformal ECoG grids with a syllablecentric decoding framework can yield substantial improvements. The ECoG arrays provided broad and stable cortical coverage, particularly over speech-related regions, and enabled us to decode a large set of 394 Mandarin tonal syllables with high accuracy—based primarily on neural features before any linguistic postprocessing," the study authors write.

While the study demonstrates marked improvement for BCIs decoding Mandarin, the authors do note some limitations and areas that could use improvement. The study only included one participant, limiting generalizability. And because the ECoG array was meant for clinical epilepsy monitoring, the electrode coverage did not include all tone-relevant brain regions. However, future studies can build upon this one, further increasing accuracy and generalizability.

The study authors hope to extend BCI applicability to a range of patients. They say, "Beyond improvements in decoding accuracy and hardware performance, expanding the neural targets of speech BCIs represents an exciting frontier.

"Although current approaches primarily leverage signals from motor and premotor cortices responsible for articulation, future systems may benefit from incorporating activity in higher-order language, such as the middle temporal gyrus, inferior frontal gyrus, and supramarginal gyrus. Integrating the semantic and syntactic information processed within these regions may help build more stable and accurate speech decoders."

© 2025 Science X Network

More information: Youkun Qian et al, Real-time decoding of full-spectrum Chinese using brain-computer interface, Science Advances (2025). DOI: 10.1126/sciadv.adz9968 Journal information: Science Advances