Live Science Plus
Reading Minds: Brain Scans Create Pictures of What You See
Machines that can read people's minds are getting closer to reality: Scientists have revealed they can now use brain scans to read letters and images participants are viewing.
The results come from two studies, both detailed in the journal NeuroImage, that suggest machines could be taught to peer inside the mind to "see" pictures of what people see in near real time.
Even so, the researchers say the true contributions of such studies are in advancing the understanding of the brain.
"It's really nice to say that we can discern images based on brain data, but I think the potentials of the study are more fundamental — for example, understanding the dynamics of perception," said Marieke van de Nieuwenhuijzen, co-author of one of the studies.
BRAINS
In the first study, researchers used functional magnetic resonance imaging (fMRI) to record the activity of the visual cortex, the brain region that processes visual information, while study participants were viewing a series of handwritten characters: B, R, A, I, N and S. [See Mind-Reading Images from the Brain Scans]
By feeding parts of this data into mathematical models over and over again, the researchers were able to "teach" the machine which pattern of activity corresponded with which letter a person was viewing.
Get the world’s most fascinating discoveries delivered straight to your inbox.
The trained model was then tested on the rest of the data — that is, it had to reconstruct the letters from the activity of the brain. And it did so with strikingly high quality.
"We reconstruct the individual pixels of which perceived letters consist," said study researcher Marcel van Gerven, a neuroscientist at Radboud University Nijmegen in the Netherlands.
Tracking the information
In the second study, van de Nieuwenhuijzen and her colleagues set out to see the time lapse of visual information propagating across the brain — a process that happens in a small fraction of a second.
The researchers used magnetoencephalography (MEG) to record people's brain activity while they watched images of faces, scenes, bodies and tools.
The data was then used to teach a mathematical model how to differentiate between patterns of brain activity occurring about 100 milliseconds after the image was presented to the study participants. The model could then tell, with high accuracy, whether the image the study participant saw was a face, a body or an object.
MEG detects small changes in magnetic fields in the brain that are caused by neurons' electrical activity. Unlike fMRI, it can differentiate between changes in brain activity occurring just a few milliseconds apart.
Therefore, the researchers were able to pinpoint when the perceived visual information hit the first processing station in the brain and how that information evolved from there over time.
"We classify to which stimulus category a perceived stimulus belongs, and we can track the spatiotemporal dynamics of the involvement of different brain regions when perceiving different stimulus categories," van Gerven said.
Understanding the brain
The mind-reading studies could lead to a better understanding of how the brain "sees" the world. [Inside the Brain: A Photo Journey Through Time]
The human visual system consists of several distinct visual areas arranged in a highly interconnected and layered network. Scientists have been trying to build models that describe and predict how the brain will respond as we see the world.
"There's been an ongoing attempt to try to understand what it is about the visual world that drives activity in different visual areas of the brain," said Thomas Naselaris, a neuroscientist at the Medical University of South Carolina who wasn't involved in either study.
Visual reconstruction is a way of testing the different proposed models about how the brain encodes visual information, and it can be done using various algorithms.
"The studies are sophisticated and elegant," Naselaris said. "They are well-done explorations of what algorithms are most efficient for reconstructing visual information. That seems to be the real advance and the real contribution."
Next, the researchers will study other brain processes, such as memory, to see how retaining something in mind develops over time in the brain.
"Our next goal is to track these dynamics in more detail and examine these dynamics during working memory maintenance," van Gerven said.
Email Bahar Gholipour. Follow LiveScience @livescience, Facebook & Google+. Original article on Live Science.
