Communication is fundamental to the human experience. As a social species, we use speech and writing to convey our wants, needs, thoughts, and feelings to those around us. Disease or injury affecting communication can therefore impede both our ability to meet basic needs and to enjoy the profound fulfillment that comes from social connection.

Such is the case for individuals whose paralysis prevents them from engaging in traditional forms of communication. In the United States, over 150,000 people are paralyzed to this degree. These individuals have intact and highly active brains, but, due to damage or disease in the nervous system, cannot animate the muscles required for talking, writing, or typing.

The future of augmentative and alternative communication BCIs

Researchers have long recognized that brain-computer interfaces (BCIs) could offer a solution for these patients. By monitoring brain activity at its source, these neural interfaces translate the patterns of the brain’s electrical activity into control signals for assistive communication devices. The goal of these applications is to restore as much communication speed and accuracy as possible while also posing the lowest possible risk and burden to the patient.

The ideal system would enable effortless communication at the speed at which a person speaks; English speakers in the United States converse at an approximate rate of 150 words per minute (WPM).

Early BCIs for augmentative and alternative communication

Developed in the 1980s, the first assistive communication BCI prototypes measured brain activity via sensors on the scalp (i.e., electroencephalogram or EEG). Researchers reported that users of these “speller” systems could indeed spell out words, but at the frustratingly slow rate of 2.3 CPM. Still, these early experiments functioned as proof of concept for augmentative and alternative communication BCIs.

Over time, non-invasive BCI technology used in the research setting has demonstrated incremental improvement but has generally demonstrated communication speeds less than 10 words per minute. Further, use of these systems requires extremely focused engagement with the task and intense visual attention.

Similar performance for augmentative and alternative communication can be achieved without BCI by using gaze-tracking methods alone. In one study utilizing a novel gaze-tracking approach, five participants with amyotrophic lateral sclerosis (ALS) achieved an average typing speed of 9.51 WPM. Note that in addition to being rather slow, gaze-tracking presents other downsides including eye strain and double-vision.

Implanted BCIs driving advances in augmentative and alternative communication

Whereas early systems measured brain activity through sensors on the scalp, the new millennium brought assistive communication interfaces that either sit directly on the brain’s surface (ECoG) or are inserted into the cortical brain (microelectrodes). These applications have focused on three main communication modes:

• Computer cursor control-based typing
• Handwriting decoding
• Speech decoding​​

Cursor control-based typing with assistive communication BCI devices

In 2006, researchers announced the success of a “neural cursor”, an intracortical microelectrode BCI system that recorded signals from the brain’s motor cortex and interpreted those signals to guide a computer cursor. A 2015 study used the same type of intracortical device but with an improved algorithm interpreting the neural data. Subjects achieved substantially increased typing speeds than previously-reported cursor-controlled typing. Then, a more recent study, again using implanted microelectrodes, demonstrated further performance advances with an average typing speed of 5.6 WPM with a maximum of 7.84 WPM.

Another cursor-control study took a hybrid approach combining gaze-tracking and the Stentrode BCI from Synchron; rather than being inserted into the surface of the cortical brain, this stent-like device is placed into a blood vessel in the brain. In this study the two participants achieved 2.7 WPM and 4.02 WPM.

Though often faster and more accurate than EEG-based augmented and alternative communication interfaces, these systems remain dramatically slower than the speed at which a person may speak or type. An additional downside of these systems is that they occupy the user’s visual attention.

Imagined handwriting in assistive communication BCI devices

Researchers have also developed an interface that decodes imagined handwriting. A 2021 study utilized two microelectrode arrays (200 electrodes) implanted into the area of the motor cortex that controls the hand. The paralyzed subject imagined writing letters with a pen, and an autocorrect algorithm was applied to the resulting brain signals. This system achieved 18 WPM with greater than 99% accuracy, the fastest speed of any published study up to this time and far exceeding speeds of eye-gaze tracking and other non-invasive augmentative and alternative communication BCI applications. These results demonstrate an implantable BCI application that does not require the use of vision and has greater communication speed, providing an advantage over non-implantable BCIs.

Speech decoding with assistive communication BCI devices

In another approach to augmentative and alternative BCI communication, a user imagines that they’re talking while electrodes record from brain areas that control muscles involved in speech. An algorithm then decodes this brain data, with the goal of transforming it into a transcript of the imagined dialogue. This type of speech decoding is especially challenging because over 100 muscles from the mouth, tongue, and vocal tract articulators are utilized in natural speech production.

A recent study demonstrated the ability to decode intended speech in a paralyzed individual using output from an ECoG array at a rate of 15.2 WPM; this system had a 25.6% error rate when decoding sentences formed from a 50-word list. Comparing results from an additional ECoG speech decoding experiment and a study with intracranial electrodes directly suggests that neural interfaces that target individual neurons (i.e. intracortical microelectrodes) produce better results in speech decoding.

Toward practical augmentative and alternative communication BCIs

The studies summarized above demonstrate that it is feasible to translate the brain’s neural activity into some form of communication using cursor-controlled typing, imagined handwriting, and speech decoding. However, the practical performance of many of these devices have been limited by speed and accuracy.

Furthermore, these applications have been confined to the laboratory setting and make use of electrodes that are approved only for short-term implantations. As such, they remain impractical for use among the people who need them most. Paradromics is now working on technology capable of overcoming these limitations.

With over 1600 electrodes in direct contact with the cortical brain, our Connexus® Brain-Computer Interface will have the ability to collect a dramatically richer stream of brain data than previous systems. Researchers have posited that collecting more precise recording from more individual neurons will translate to augmentative and alternative communication applications that are both speedier and more accurate than their predecessors. Additionally, the Paradromics BCI platform is designed for long-term implantation. Together, these advancements will yield an assistive communication device that can produce results not just in a lab but improve the autonomy and quality of life of everyday users.

If you have any questions, please reach to media@paradromics.com.

References

Angrick M, Ottenhoff MC, Diener L, et al. Real-time synthesis of imagined speech processes from minimally invasive recordings of neural activity. Commun Biol. 2021; 4(1):1055.

Anumanchipalli GK, Chartier J, Chang EF. Speech synthesis from neural decoding of spoken sentences. Nature. 2019; 568: 493-510.

Farwell LA, Donchin E. Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. Electroencephalogr Clin Neurophysiol. 1988;70(6):510-523.

Hochberg LR, Serruya MD, Friehs GM, et al. Neuronal ensemble control of prosthetic devices by a human with tetraplegia. Nature. 2006;442(7099):164-171.

Jarosiewicz B, Sarma AA, Bacher D, et al. Virtual typing by people with tetraplegia using a self-calibrating intracortical brain-computer interface. Sci Transl Med. 2015;7(313):313ra179.

Kawala-Sterniuk A, Browarska N, Al-Bakri A, et al. Summary of over Fifty Years with Brain-Computer Interfaces-A Review. Brain Sci. 2021;11(1):43.

Moses DA, Metzger SL, Liu JR, Anumanchipalli GK, Makin JG, Sun PF, Chartier J. "Neuroprosthesis for Decoding Speech in a Paralyzed Person with Anarthria." The New England Journal of Medicine. 2021; 385 (3): 217-227.

Metzger SL, Littlejohn KT, Silva AB, Moses DA, Seaton MP, Wang R, Dougherty ME, Liu JR, Wu P, Berger MA, Zhuang C, Willsey K, Ganguly K, Anumanchipalli GK, Chang EF. A high-performance neuroprosthesis for speech decoding and avatar control. Nature. 2023;620(7976):1037-1046.

Mott ME, Williams S, Wobbrock JO, Morris MR. Improving Dwell-Based Gaze Typing with Dynamic, Cascading Dwell Times. Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. ACM. 2017; 2558-2570.

Pandarinath C, Nuyujukian P, Blabe CH, Sorice BL, Saab J, Willett FR, Hochberg LR, Shenoy KV, Henderson JM. High performance communication by people with paralysis using an intracortical brain-computer interface. Elife. 2017 Feb 21;6.

Willett FR., Avansino DT, Hochberg LR, Henderson JM, Shenoy KV. High-performance brain-to-text communication via handwriting. Nature. 2021; (593): 249-254.

Willett FR, Kunz EM, Fan C, Avansino DT, Wilson GH, Choi EY, Kamdar F, Glasser MF, Hochberg LR, Druckmann S, Shenoy KV, Henderson JM. A high-performance speech neuroprosthesis. Nature. 2023;620(7976):1031-1036.