Using fMRI, scientists decoded what people were hearing and thinking. Scientists can now "decode" people's thoughts without even touching their heads, The Scientist reported (opens in new tab). Past mind-reading techniques relied on implanting electrodes deep in peoples' brains. The new method, described in a report posted Sept. 29 to the preprint database bioRxiv, instead relies on a noninvasive brain scanning technique (opens in new tab) called functional magnetic resonance imaging (fMRI). fMRI tracks the flow of oxygenated blood through the brain, and because active brain cells need more energy and oxygen, this information provides an indirect measure of brain activity. By its nature, this scanning method cannot capture real-time brain activity, since the electrical signals released by brain cells move much more quickly than blood moves through the brain. But remarkably, the study authors found that they could still use this imperfect proxy measure to decode the semantic meaning of people's thoughts, although they couldn't produce word-for-word translations. "If you had asked any cognitive neuroscientist in the world 20 years ago if this was doable, they would have laughed you out of the room," senior author Alexander Huth (opens in new tab), a neuroscientist at the University of Texas at Austin, told The Scientist. For the new study, which has not yet been peer-reviewed, the team scanned the brains of one woman and two men in their 20s and 30s. Each participant listened to 16 total hours of different podcasts and radio shows over several sessions in the scanner. The team then fed these scans to a computer algorithm that they called a "decoder," which compared patterns in the audio to patterns in the recorded brain activity. The algorithm could then take an fMRI recording and generate a story based on its content, and that story would match the original plot of the podcast or radio show "pretty well," Huth told The Scientist. In other words, the decoder could infer what story each participant had heard based on their brain activity. That said, the algorithm did make some mistakes, like switching up characters' pronouns and the use of the first and third person. It "knows what's happening pretty accurately, but not who is doing the things," Huth said. In additional tests, the algorithm could fairly accurately explain the plot of a silent movie that the participants watched in the scanner. It could even retell a story that the participants imagined telling in their heads. In the long term, the research team aims to develop this technology so that it can be used in brain-computer interfaces designed for people who cannot speak or type. Source
