That could make it valuable for neuroscientists studying the brain, says Sprague. “None of us can afford 40 hours of imaging for a new subject,” he says. “It’s something like $600 to $1,000 an hour.” Tools like this one could speed up research, he says.
State of the art
The encoder and decoder aren’t perfect. “Of course we have failures,” says Irani. Over a Zoom call, she pointed out an image of a cake that her tool reconstructed as a pile of three sandwiches, and another of a dog in a bathtub that was reconstructed as a similarly colored goat in a bathtub.
But they represent the state of the art. In a comparison test, the tool was found to be much better than previously described ones. “All in all, really we outperformed the others by a significant margin,” Irani says. “Mind reading” is a “cute, jazzy name” for what they’re doing, she adds.
Irani is now planning to move beyond images to video and audio. She wants to be able to reconstruct what people are thinking about or imagining, and the contents of their dreams. “That’s something we don’t have yet,” she says. “But we’re striving to achieve it.”
Such a tool might enable people who are “locked in” and completely paralyzed to communicate using their brain activity alone, she says. It could also help scientists unpick some enduring mysteries surrounding the inner workings of our minds, such as what PTSD flashbacks look like.
Advances like this inevitably raise questions about mental privacy. What if some bad actor could reconstruct someone’s thoughts or memories?
“If you’d asked me that 10 years ago, I’d have laughed a lot,” says Sprague. Getting a willing person to lie still in a scanner and actively engage with a research question is hard enough; imagine making someone do it involuntarily. But Irani and other scientists are working on similar approaches to decode brain activity from EEG—electrical brain activity measures collected via a cap of electrodes or even through headphones.
And as models improve, it will become even easier to analyze the brain activity collected this way. “We have to be a little more serious about the ethical considerations,” says Sprague. He thinks Irani’s approach would probably “work quite well” in predicting images that a person is thinking about but not looking at.
