Israeli scientists have devised an AI model that can reconstruct and even predict what people are seeing with startling accuracy, likening the results to mind reading.
Their work still exists largely in lab settings, though the researchers are hopeful that advances down the line could be extended even as far as reading people’s dreams. More immediately, the technology could have potential for medical purposes, like helping people communicate who otherwise can’t due to injury or disability.
The AI model, called Brain-IT, was developed by professor Michal Irani and fellow researchers at the Weizmann Institute of Science in Israel. They’ve documented their work on GitHub and in a paper presented at a scientific conference earlier this year.
Brain-IT takes advantage of the thing that AI is best at, which is pattern recognition. It takes in data from functional MRI brain scans, which track changes in blood flow and oxygen in the brain to measure brain activity, and then uses that data to recreate the images the person was thinking about when the scans took place.
It also works in reverse, predicting what a brain scan would look like if shown a specific image.
According to Irani, other AI models can translate brain activity into images and preserve the general vibe of an image.
“However, they tend to make mistakes in basic features such as composition and color,” Irani said in a statement. “The new model we developed outperforms them in reconstructing both the content of the image and its details.”
Brain-IT is also substantially faster. The researchers say that it needs only 1 hour of fMRI data from a new subject to match the results achieved by other methods trained on 40 hours of recording.
The Brain-IT image reconstructions — the right side of each paired image — clearly aren’t perfect but they are largely true to the original.Weizmann Institute of ScienceHow does the Brain-IT AI model work?
The Weizmann Institute researchers trained the AI model on thousands of brain scans from the publicly available Natural Scenes Dataset that came from eight volunteers who were told to look at specific images while they were being scanned. This allowed the AI to identify patterns in brain activity.
The researchers noted that different regions of the brain light up on scans when people think about specific things and identified 128 “functional regions.” For example, Irani said, some regions light up in scans when someone looks at food and others when a person views sports. These patterns helped the AI figure out what the person being scanned was witnessing.
Some of these regions were already known by neuroscientists. Research has shown that dogs’ brains light up like Christmas trees in the presence of their owners and that dogs can differentiate between a human’s facial expressions. It’s also been observed that humans use the same neurons in recalling an image as they do looking at them.
As impressive as Brain-IT is, it’s also quite limited. It can currently only generate images from fMRI brain scans. This takes time and requires a person to willingly put themselves into an MRI machine.
Irani and other scientists are also looking into whether a simpler EEG device could deliver similar results more easily, according to MIT Technology Review.
So is it mind reading? Not really — nothing in this technology is capturing thoughts, memory or language. What the Weizmann Institute researchers have built is essentially a more efficient and reliable method of reconstructing a scene based on the brain activity of the person viewing it.
In an interview with MIT Technology Review, Irani acknowledged that “mind-reading” is a “cute, jazzy name.”
Irani and her team now want to see if they can achieve similar results with auditory information.
Beyond that lies a more mysterious realm: dreams. That, Irani said in a statement, would require conquering the challenges of decoding video.
“Dozens of images change every second while an fMRI scan takes about two minutes,” she said. “If we overcome all these obstacles, it’s possible that in the future, we may even be able to read dreams.”
Joe is a freelance journalist. It all started with a long-running affection for building his own PCs, which he did for the first time as a teenager. It evolved into a lifelong enjoyment of putting words on the internet about the subject. He's written for CNET, PCMag, Mashable and SlashGear as a freelance writer, and worked as a Senior Editor at Android Authority for 10 years. When he's not writing about tech and science, he's learning the ins and outs of DIY home repair, gaming, playing his bass guitars and posting help on PC building and gaming subreddits. He is a staunch believer that orange juice should have pulp. See full bio
English (US) ·