Brain Reading 101

Christopher Walken in dream recording
machine from the film Brainstorm (1983) 

Jack Gallant and his labmates at UC Berkeley reconstructed the "internal perceptions" of human subjects watching film clips. It's amazing work, and may mark significant progress toward a kind of machine-based brain reading.

“This is a major leap toward reconstructing internal imagery,” said Professor Gallant, in a press release about the research. “We are opening a window into the movies in our minds.”

The demonstration, published in the Sept. 2 issue of the journal Current Biology, went like this: Subjects lay in fMRI machines that recorded the brain activity in their visual cortex while they watched film clips.  Gallant took those "recordings" and used algorithm driven computer searches to "translate" those recordings into other, similar images taken randomly off of YouTube. The YouTube clips were a kind of image palate from which the original footage was reconstructed. Several reconstructions from different subjects were averaged together to get a blurry, dream-like video sequence that roughly, and eerily correspondes to the original video. You can see the result in this dyptic, which shows the video that the subjects watched, on the left, and the reconstruction from their brain scans on the right.  

The Gallant group is reading from the relatively straightforward visual cortex. Trying to interpret "thought" patterns in the cerebral cortex or distributed processes like memory and emotion would be much, much more complex. Machines that could make meaningful reproductions of thoughts,  emotions, or memories may be decades off, but they are coming. And we probably aren't ready for the ethical, legal, and emotional challenges (and opportunities) that will accompany that kind of tech. Or maybe we are: we already create representations of our internal lives all the time by doing art and having conversations. In fact, aren't the original film clips already more-or-less high-fi representations of the minds of their original directors mediated through another technology: film? 

Brains Don’t Buy Tickets, Moviegoers Do

A provocative and funny piece in February’s WIRED examines a San Diego company’s efforts to evaluate films by giving fMRIs to people watching them. The firm, MindSight Neuromarketing, sticks people into an MRI and shows them clips while watching their brains for amygdalar action. Because the little almond-shaped nuclei in the brain are associated with intense emotions, the premise is that if they light up, the movie maker is doing something right. Journalist Scott Brown has his tongue in his cheek when he says that he looks forward to films that will redline his amygdala activation nonstop and when he predicts that “movie houses will become crack dens with cup holders, and I’ll lie there mainlining pure viewing pleasure for hours.”
My favorite part though, is that the movie-loving test subjects “often… tell a human researcher one thing while the fMRI reveals they’re feeling the opposite.”
Subject: No, Dr. Cinema, I didn’t like the chess scene, it was boring.
Researcher: Actually, you loved it. You just thought it was boring you.
Who is a director going to believe, the fMRI or the human?
Letting fMRI guide your film would be like asking the idiot lights on your dashboard to tell you where to drive. Maybe they have something important to say once in a while, but they’ll suck at finding the scenic route to Mendocino.
Anyway, if you are into redlining your amygdalae, there’s plenty of material out there already that can make it glow till it drops. And THAT is boring.
Bottom line, directors may flirt with MindSight-type feedback for a couple of years, but will soon realize that people buy movie tickets, not amygdalae.