Making the invisible visible

In Mind
Written by  Jolien Linssen Friday, 17 June 2016 10:21

Before you start reading this article, take a short pause and have a look around the room. What do you see? Whether it’s a table, chairs, trees outside the window, or the traffic rushing by - your ability to perceive the outside world seems utterly self-evident. Yet have you ever wondered how our sense of sight, one of the most important senses we have, actually works? The PhD research of Thomas Emmerling brings us one step closer to unraveling the mysteries of visual perception and visual mental imagery.


Emmerling, who studied Psychology and Computer Science at the University of Trier, Germany, came to Maastricht University in 2011. He was invited by Professor of Cognitive Neuroscience Rainer Goebel to take part in his project ‘Columnar Code Cracking’, for which he had been awarded a 2,5 million euro grant from the European Research Council (ERC). Five years later, the result is a PhD thesis titled ‘Imaging imagery: an investigation of visual cognition using high-resolution fMRI’. Looking back, Emmerling states that “there hasn’t been a time in my life in which I learned as much as over the last couple of years.” What’s more, he managed to break new ground.

Visual perception
Emmerling started his research by focusing on two specific features of perception, direction of motion and binocular disparity. Binocular disparity is caused by the relative shift of images between the two eyes and is an important cue for the brain to see depth. Emmerling: “We knew from previous research that direction of motion and binocular disparity are both processed in a specific area in the monkey brain. The challenge for us was to show that the same holds true for the human brain.”

This is where the 7 Tesla MRI scanner comes in, which was used to measure the brain activity of research participants after having been presented with a specific set of stimuli. “While they were in the scanner, they looked at dots moving in different directions. By means of binocular disparity, these dots were shown as either near or far away,” Emmerling explains. “We found what we expected, as we were actually able to map these two complex features - direction of motion and binocular disparity - in the same area in the human visual cortex. And that’s really great, for it matches with models of this area very well.”

Imagined motion
As human beings, our sense of vision stretches beyond the material world. Does this sound a bit vague? Just close your eyes and imagine, let’s say, a dog. There it is; not really out there, but before your mind’s eye. What happens in the brain during mental imagery, and how to measure this?

“That’s what we addressed in the second part of the research, where we shifted from perception to imagery,” Emmerling states. “Before they went into the scanner, we trained our participants to imagine dots moving in different directions; leftward, rightward, upward and downward. During the experiment, we cued them as to which kind of motion imagery they would have to start. Afterwards, we decoded these different imagined directions of motion solely from the brain activity that we measured in the visual system. We found that certain participants, who performed very well, were able to activate their early visual cortex. In other words: area’s that would normally be used for seeing rather than imagining.”

Imagined letters
In the third and final study, Emmerling trained his participants to imagine letters instead of motion: H, T, S, and C. “We chose these letters because they’re very distinct in their features, being either edgy or curvy,” he clarifies.

“In order to understand what we did next, it’s important to know that in the early visual cortex, there’s a topographical representation of one’s retinas. By means of making a so-called retinotopy, it’s therefore possible to know how the association between the visual space and cortical space in an individual looks like. If a participant sees an H, for instance, I can now try to map back into visual space what the cortical space tells me in activation. Basically, I can recreate the picture that was seen by the participant.”

Does this also work when the participant doesn’t see anything at all, and merely imagines the letter H? The answer is affirmative. Emmerling: “When trying to decode the reconstructed imagined letters, we ended up with significant, and at times quite high accuracies of up to 60 percent. So yes, we are able to reconstruct pictures of visual mental imagery, to a rather rough extent.”

Reading minds
One might wonder, even worry, whether this will open the door to reading people’s minds. “Let me be clear,” Emmerling states. “I don’t know what my research participants think. I only know that they perform a visual mental imagery task very well, not only because they’re very cooperative but also highly trained to do so. Reading minds is out of the question.”

What his findings could be used for, on the other hand, is helping a specific group of patients with locked-in syndrome to communicate with their environment. “For patients that don’t respond well to EEG for instance, an fMRI-based Brain Computer Interface could be a solution. Yet I would also like to emphasize that we should appreciate basic research results like these for what they are. The expansion of knowledge is of general value for society, I believe.”

Thomas Emmerling (1986) studied Psychology and Computer Science at the University of Trier, Germany. In 2011, he started his PhD project at Maastricht University’s Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience. Currently, he works as a software engineer at Brain Innovation in Maastricht.

Rate this item
(5 votes)
Read 4890 times
You are here: Home Research Mind Making the invisible visible

Maastricht University Webmagazine

Marketing & Communications
Postbus 616, 6200 MD Maastricht
Minderbroedersberg 4-6, 6211 KL  Maastricht
Tel: +31 43 388 5222
This email address is being protected from spambots. You need JavaScript enabled to view it.

Download UM Magazine

Read a digital version of Maastricht University magazine,
or download the PDF.
UM Webmagazine June 2017

Connect with us