Feature: All in the mind

Of all the 'big questions', the nature of consciousness is one of the biggest. So big, in fact, that some doubt whether science can possibly have anything to say about it. For science is supposedly all about objective observation and measurement; how can one begin to study something so subjective, so intimately linked to personal experience?

So, for many years, consciousness was not judged a fit subject for rigorous academic analysis. It was best left to the philosophers or confined to secret late-night sessions in the bar at conferences. If anyone dared bring it into the open, it was only long after they had safely secured academic tenure.

"Although there is research going back at least 100 years, there was always the feeling that this wasn't a serious scientific subject," says Dr Geraint Rees, a Wellcome Senior Research Fellow at University College London.

All that changed with the intervention of Francis Crick, who, in a landmark Nature paper with Christof Koch, made consciousness a legitimate subject for scientific discourse. He dismissed the metaphysical angle and argued that the roots of consciousness must lie in the activities of the cells that make up the brain. He therefore advocated studies of the structures of the brain to track down the 'neural correlates of consciousness'.

"Consciousness is perhaps the most complicated end product of the most complicated object – the brain – in the whole universe," says Dr Rees. "I'm looking at the contents of our subjective experiences – what people call 'qualia' or 'raw feels' or sensations," he says. "They can be very hard to pin down – feelings like déjà vu or sadness can be very diffuse – so I'm keeping things simple and focusing on qualia that are associated with very specific things in the external world, such as the identity of objects, or their motion or colour."

The main tool of his trade is functional magnetic resonance imaging (fMRI), with which he aims to identify how specific subjective experiences are represented in the brain. Spookily, in doing so he can end up knowing more about what brains are doing than their owners do.

Left or right?

Key to recent work has been an important technical advance. fMRI has a spatial resolution of a few millimetres, representing the activity of many hundreds of thousands of neurons. But the brain often represents important features of the visual world, such as the orientation of an object, at a much finer spatial scale. It's like wanting to know what's going on in towns but only having data for a whole county.

Now Dr Rees and his postdoctoral fellow John-Dylan Haynes have developed a method that does not simply pool fMRI signals across the brain, but instead looks at the fine spatial pattern of the signals within an area of the brain. Each fMRI image is made up of many tiny cubes or 'voxels' – the three-dimensional equivalent of computer-screen pixels – each of which measures the activity of populations of neurons within it. It turns out that the fine pattern of activity across voxels encodes information about brain activity at a much finer spatial scale than previously realised, enabling scientists using fMRI to study types of neural process previously thought inaccessible.

Dr Rees has begun to use this new technique to measure how the visual brain responds to a classic visual stimulus: oriented lines. "We know from the Nobel Prize-winning work of Hubel and Wiesel that columns of neurons in very early areas of visual cortex at the back of the brain [V1] respond well to oriented lines," he explains. "But these columns are represented at a very fine spatial scale, below the resolution of conventional fMRI."

He has scanned the brains of subjects who were looking at one of two differently oriented stimuli.

"On average the fMRI signal from early visual cortex is pretty much the same.Individual voxels in early visual cortex have some weak preferences for one or the other stimulus, but this isn't good enough to distinguish which one the subject is seeing. But across the whole population [of voxels], if we look at the fine-grained pattern rather than just the average, a different picture emerges. There is much more information in the pattern than in either the average or individual voxels, which is not usually taken into account."

By concentrating on this fine-grained pattern of activity, Dr Rees was able to develop an algorithm that made very good predictions about the orientation of stimuli presented to subjects. "If the algorithm was just guessing you'd have a 50–50 chance of guessing the stimulus orientation. But we found that a two-second measurement of brain activity in V1 was sufficient to predict with 85 per cent accuracy the orientation of the stimulus."

This is a very different approach from conventional brain imaging, where hundreds of images are often averaged over tens of minutes to show subtle differences in brain activity. Instead, this new technique can take a single fMRI image and predict with high accuracy what the subject is seeing.

The technique allows neuroscientists to infer from fMRI signals what is happening at a very fine spatial scale, perhaps at the resolution of a neuronal column. "That's the breakthrough," says Dr Rees. "It's a fantastic technique. It gives us the most detailed insights into how the brain is functioning."

Armed with this technique, Dr Rees and his team then set out to explore the boundaries between consciousness and unconsciousness.

The mask

The algorithm provided insight into the conscious perception of subjects. But would it still work if the subjects were not actually aware of what they saw?

This called for some more clever experimental sleight of hand. Subjects were given a brief glimpse of an oriented stimulus followed by a strong 'masking' image that erased the oriented stimulus from their awareness. "We told them that an oriented stimulus was being shown and that they must really try and look for it, and then press a button indicating whether it was tilted left or right, even if they had to guess," says Dr Rees. Their responses showed that they were just guessing, so were not conscious of the orientation of the stimulus. But could the algorithm nevertheless decode its orientation from the pattern of their brain activity?

"The mathematical algorithm still worked," says

Dr Rees. "It's not quite as accurate now, it's at about 60 per cent accuracy, but it is significantly better than chance. So our algorithm can decode from a two-second sample of brain activity what was being shown to these people, even when they are unable to consciously perceive it. We could tell more about the subject's experience than the subjects themselves, with the same information. So we're reading their unconscious mind from single samples of brain activity."

The study suggests that oriented stimuli that are not consciously perceived are nevertheless processed in the visual cortex. It represents the first direct evidence that the human visual system can process information outside conscious awareness, a hypothesis first advanced by Crick and Koch a decade ago.

So what determines whether visual information becomes conscious or remains subconscious? Although cortical processing in V1 may be necessary for visual awareness, it is not sufficient by itself. Possibly information has to cross a threshold level of activity in V1 – perhaps if the targets had been shown for a split second longer, the conscious mind would have become aware from the activity in V1. Alternatively, it may depend on signals being relayed to another region of the brain.

Even more intriguingly, the findings suggest that unconscious processing can occur in the same part of the brain as conscious recognition. "Since we can get enough information from this visual area to decode what was presented to the subject, even though they were not conscious of it, there must be some neural representation of the unconscious in V1," says Dr Rees.

Stream of consciousness

More recently, Dr Rees's group has been applying this technique to explore the stream of consciousness – how conscious awareness changes spontaneously over time. To do this, they turned to a well-known illusory phenomenon known as 'binocular rivalry'. When one image is presented to one eye and a different image to the other by means of mirrors or special glasses, instead of seeing a blend of the two images superimposed, the viewer's perception flips repeatedly between the two.

This phenomenon is one example of a more general type of visual illusion known as 'bistable perception' – such as the drawing of two women, one young and one old, within the same image, or Rubin's face/vase illusion, which can appear as the profiles of two faces or a vase, but never both at the same time.

"Our stream of consciousness is very complex and constantly changing, but binocular rivalry provides a simple model system," says Dr Rees. "During rivalry we can experience just one of two states, but this fluctuates spontaneously and continuously just as our thoughts and experiences do in everyday life."

After training the algorithm for the relevant fMRI signals, he compared its predictions with what the subject reported seeing. "Again the prediction is excellent – the algorithm had an 80 per cent accuracy. This means we can accurately decode how consciousness changes over extended periods of time from brain activity alone – literally reading the subjects' minds."

The finding represents a step towards a device that could read someone's conscious mind, or determine whether someone was experiencing binocular rivalry. Such a device might have medical implications. "We could try and see if these kinds of fluctuating changes in perception occur in humans or animals that are unable or unwilling to make responses. For example, in some forms of minimally conscious or persistent vegetative state it would be very interesting to know whether individuals are having conscious perceptual experiences, even though they cannot respond."

However, Dr Rees warns that such outcomes are a long way off. "At the end of the day, what I'm studying

is one little bit of the big picture. It's a new way of analysing the brain. I'm taking basic vision science tools – very simple stimuli and low-level questions – and trying to apply them to higher-level questions about our consciousness. About how, when and why we are aware."

Further reading

• Haynes J-D, Rees G. Predicting the stream of consciousness from activity in human visual cortex. Curr Biol 2005;15(14):1301–7.
• Haynes J-D, Rees G. Predicting the orientation of invisible stimuli from activity in human visual cortex. Nat Neurosci 2005;8(5):686–91.
• Boynton GM. Imaging orientation selectivity: decoding conscious perception in V1. Nat Neurosci 2005;8(5):541–2.
• Bristow D et al. Blinking suppresses the neural response to unchanging retinal stimulation. Curr Biol2005;15(14):1296–1300.
• Burr D. Vision: in the blink of an eye. Curr Biol 2005;15(14):R554–6.