The technology will be able to help people who can not communicate, yet to communicate. Neuroscientists have developed a device that can convert brain signals to speech. It works using artificial intelligence. Article published in the journal Nature.
This technology is not yet sufficiently advanced for use outside the laboratory, although it can synthesize whole sentences that are mostly understandable. Scientists have previously used artificial intelligence to translate individual words, mostly consisting of a single syllable. Now there has been significant progress in technology. It is the transition from individual syllables to sentences that is one of the things that makes the current work so impressive. Technically it is quite difficult.
Many people who have lost the ability to speak communicate using technology that requires them to use tiny movements to control the cursor that selects letters or words on the screen. British physicist Stephen Hawking, who had a motor neuron disease, is one of the famous examples.
Since people who use such devices must spell words, this method can be very slow: only up to ten words per minute. Natural speaking, on average, 150 words per minute. That is why scientists are thinking about improving the system.
Researchers implanted electrodes like these into participants’ skulls to record their brain signals.
The researchers worked with five people whose electrodes were implanted onto the surface of the brain as part of the treatment of epilepsy. At first, the team recorded brain activity when participants read hundreds of sentences out loud. The scientists then combined these recordings with data from previous experiments, which determined how the movements of the tongue, lips, jaw and larynx create sound.
The device converts brain signals into the intended movements of the vocal tract and turns these movements into synthetic speech. People who listened to 101 synthesized sentences could understand on average 70% of words.
However, students identified synthesized speech by selecting words from a set of choices. As the number of options increased, it became increasingly difficult for people to understand them. So the device has a long way to perfection.