2 Why content matters: Binocular rivalry and the multiple levels of conscious experience

In 1996 Nikos Logothetis & David Leopold published a landmark study on the neural mechanisms of visual awareness. They presented their participating monkeys with a very elaborate visual stimulus display, which allowed them to show one image to one eye and another image to the other eye. For example, the left eye might be stimulated with a line pattern tilted to the left, and the right eye might be stimulated with a line pattern tilted to the right. In such cases, where conflicting input is presented to the two eyes, human participants don’t experience a fusion between the two images. Instead, conscious visual perception alternates between phases where one of the eyes’ inputs become visible and phases where the other eye’s input are seen. Perception waxes and wanes more or less randomly between two perceptual experiences—despite constant stimulation. Similarly, the monkeys that were exposed to these binocular rivalry stimuli indicated behaviorally by pressing levers that their perception alternated between the inputs to the two eyes.

In parallel, Leopold & Logothetis (1996) investigated what happened to the firing patterns of single neurons in the monkeys’ brains. Their setup allowed them to not just look at one location in the brain, but to assess neural correlates in several visual brain regions. They found only a small percentage of single neurons in early visual cortex (V1/V2) whose firing pattern was modulated by the stimulus that was currently dominant. In contrast, in a higher-level visual area—V4—they found that many more cells changed their firing rates with changes in perception. This establishes a clear dissociation between early visual areas where neural signals seem not to correlate with awareness and high visual areas where they do. In a follow-up experiment, Sheinberg & Logothetis (1997) investigated the involvement of even higher visual regions in the temporal cortex in binocular rivalry. Because cells in these regions preferentially respond to more elaborate visual features, they used complex shapes and images, such as, for example, an abstract sunburst pattern or a picture of a monkey face (Figure 1a). They found that in the superior temporal sulcus and in the inferior temporal cortex, a large percentage of cells modulated their firing rate with perceptual dominance. Taken together, these studies seem to suggest that visual awareness affects signals only at late stages of the visual system.

Figure 1: Binocular rivalry and levels of perception. (a) Two conflicting stimuli, one presented to the left and one to the right eye, lead to a perceptual alternation between phases where the input of either the left or right eye is consciously seen. In monkey single-cell electrophysiology, this perceptual alternation has a correlate in higher-level visual regions of the temporal lobe, but activity in earlier visual regions shows only small changes in activity patterns. Presumably, signals in the temporal cortex encode the complex figural properties of the stimuli, such as the left being a sunburst pattern and the right being an image of a monkey face. However, due to the invariance of brain responses in higher-level visual regions to low-level features, this cannot explain the perceptual difference between the rivalry of the left and of the right sunburst pattern shown in (b), where the central circle has changed colour but the entire shape remains similar (monkey illustration by Chris Huh, Wikimedia Commons).

But what does it mean exactly that visual awareness only affects late stages of visual processing? Does it mean that high level visual areas contain all the neural correlates of contents consciousness (NCCCs), in a way similar to a CD encoding the contents of a piece of music? If the signals in these high visual areas are really responsible for encoding all contents of visual experiences then any aspect of conscious perception that changes during binocular rivalry should be explainable by changes in signals in these higher-level brain regions. There are reasons to believe that this cannot be the case. Consider the two images as shown in Figure 1a. At one instant the monkey might consciously see the face image. This percept would be encoded in activity patterns in the higher visual areas. In the next instant the monkey might see a sunburst pattern, and this experience would also be encoded in the higher visual cortex. At first sight this seems reasonable. Higher-level visual areas are specialized for complex visual information and object features (Sáry et al. 1993). So cells that have a preference for faces might respond during dominance of the face image, and cells with a preference for sunburst patterns might respond during the dominance of that pattern. But there is one difficulty in this interpretation. The images have a high-level interpretation as complex shapes, but they are also composed of a multitude of minute visual features, edges, surfaces, colours, etc. During rivalry, our perception does not only change according to the abstract interpretation, with respect to abstract, high-level interpretation, but also in terms of the minute, fine-grained details of visual experience (see Figure 1b).

This poses a problem because responses in higher-level visual areas are invariant with respect to low-level features (Sáry et al. 1993). Cells in higher-level visual areas in the inferior temporal cortex respond selectively to specific object features in an invariant pattern (Figure 2). A cell specialized for detecting, say, a circle, will respond to this circle irrespective of the low-level features by which it is defined (here brightness, contrast, and colour contrast). This means that such a cell disregards the low-level features and does not convey information about them any more. While cells in high-level visual areas might be able to explain why we see a face one moment and a sunburst pattern the next, they cannot explain why the sunburst pattern is yellow instead of red, or why it is one specific visual pixel collection out of the many possible that would be seen as a sunburst pattern. Thus, visual experience is a multilevel phenomenon, and a theory of the neural correlates of visual awareness will have to be able to explain all the levels of our experience, not just one. This clearly shows the importance of a content-based approach to visual consciousness.

Figure 2: Invariance of single-cell responses in higher-level visual areas. Responses in low-level visual areas (top) are tuned to low-level features such as colour or luminance. In contrast, responses in higher-level object-selective regions (bottom) are largely invariant with respect to these low-level features (Sáry et al.1993).