Millions of people benefit from speech interfaces, such as Siri and Google Voice Search, which enable rapid speech-to-text input for electronic devices. However, these speech interfaces often require individuals to produce comprehensible speech, which may be hindered by environmental factors, such as loud noises or the inability to produce speech due to injury or certain health conditions. Recent research has indicated that it is possible to produce speech by imagining sounds through brain-computer interfaces (BCIs). This latest technology offers hope to amyotrophic lateral sclerosis (ALS) patients suffering from locked-in syndrome – a condition where the patient is fully conscious but is unable to move or communicate orally due to paralysis – to maintain a channel of communication with the world around them. Thinking words with brain-computer interfaces (BCIs) Traditional automatic speech recognition devices work by transforming spoken language into textual representation. With brain-computer interfaces, however, neural signals can now be translated into speech. BCI works by allowing communication based on brain activity, hence doing away with the need for voice production. “So instead of saying ‘Siri, what is the weather like today’ or ‘Ok Google, where can I go for lunch?’ I just imagine saying these things,” describes Christian Herff, a researcher specialising in cognitive systems at the University of Bremen. Herff published a review with Dr Tanja Schultz last year, in the journal Frontiers in Neuroscience. It explored the possibilities of using brain imaging techniques as input for speech recognition. Of all the methods used, electrocorticography (ECoG) was found to be the most ideal due to characteristics such as high temporal and spatial resolution, robustness of ECoG toward artefacts and being unfiltered by skull and scalp. A diagram depicting how speech decoding software works. Photo credit: Christian Herff/Frontiers In their study, epilepsy patients had their brain activity recorded via implanted electrode grids, while they read texts presented on a screen. A database containing neural signal patterns corresponding to their respective “phones” or speech elements was then developed. “For the first time, we could show that brain activity can be decoded specifically enough to use ASR technology on brain signals,” says Herff. “However, the current need for implanted electrodes renders it far from usable in day-to-day life.” His opinions are echoed by Stéphanie Martin, a doctoral student with the Chair in Brain-Machine Interface (CNBI), EPFL, who noted that electrocorticography is very “invasive” because “it involves implanting electrodes quite deep inside the patient's brain". Martin’s recent research, published in the journal Cerebral Cortex tries to provide more supportive understanding to develop BCIs. Her study demonstrated how music is imagined in the brain – how the auditory cortex and other parts of the brain process auditory information, such as high and low frequencies. Experimental task design. (A) The participant played an electronic piano with the sound of the digital keyboard turned on (perception condition). (B) In the second condition, the participant played the piano with the sound turned off and instead imagined the corresponding music in his mind (imagery condition). In both conditions, the sound output of keyboard was recorded in synchrony with the neural signals. Photo Credit: Ecole Polytechnique Federale de Lausanne Similar to Herff’s group, Martin’s group hopes to apply these findings to language for people who lost their ability to speak, including aphasia patients. "We are at the very early stages of this research. Language is a much more complicated system than music: linguistic information is non-universal, which means it is processed by the brain in a number of stages," explains Martin. Enabling advanced ALS patients to communicate This could also help patients suffering from motor neuron diseases such as ALS – where even eye movement is not possible in the later stages. Conventional BCIs rely on eye movement, which become useless, at advanced stages of ALS. Until now, researchers have only developed a novel way of identifying neural signals by measuring fluctuations in oxygen levels in the brain, called functional near-infrared spectroscopy (fNIRS). It utilises light to detect changes in blood oxygen levels and functions based on the principle that active brain regions consume more oxygen. In a study employing fNIRS to establish communication in complete locked-in state (CLIS), four ALS patients were able to communicate with their family members and caregivers albeit through simple “yes” or “no” answers. This ground-breaking research demonstrated for the very first time that it was possible for locked-in patients to interact reliably with the outside world. Although the level of accuracy for their responses was 70%, the patients claimed that they were “happy” and could express feelings pertaining to their condition. BCI technology a possibility in near future Brain-computer interface (BCI) technology has come a long way in striving to enable severely disabled or locked-in patients to “speak again”. Previous applications have enabled users to get messages across by spelling words but were often inhibited by low communication speed. Although the latest technology has progressed to a stage where locked-in patients are only able to provide simple yes or no responses, this already indicates “a big improvement in quality of life”, says Ana-Matran Fernandez, a researcher on biomedical signal processing. While challenges abound, it is the hope of enabling patients and their loved ones to communicate with each other again, that encourages researchers worldwide to push boundaries in the field of BCI. Source
