6 Is the attentional effect unconscious — like blindsight?

The experiments I have described are “forced choice” experiments in which the subjects must choose between two alternatives. In any perception experiment the issue can be raised of whether the perception is conscious or unconscious, but the issue is often especially troublesome in forced choice experiments with brief stimuli in which subjects make a conscious choice but in which the stimuli are sufficiently evanescent that subjects do not get a really good look at them (Phillips 2011). In addition, the stimuli are presented very briefly, in the experiments described above for 80 ms or less. Many subjects will say that they are never 100% sure of anything. And this can lead to the charge that what is really going on is akin to “blindsight” in which the perception, though genuine, is unconscious (Turatto et al. 2007).

Why are the presentations so brief? Brief presentations preclude eye movements, they preclude significant perceptual adaptation (the “neural fatigue” that causes afterimages), and they preclude certain kinds of strategic responding on the part of subjects. Further, it is known that the effects of exogenous attention peak at around 120 ms after the cue, so to maximize the effects of exogenous attention, brief presentations are required.

Massimo Turatto (2007) showed, using a procedure much like Carrasco’s with judgments of perceived speed, that an unattended moving patch was treated as equal in speed to an attended moving patch that was slower by about 10%. However when they asked subjects for subjective judgments of moving stimuli in peripheral vision that really did differ in speed by 10% (without any attentional manipulation), subjects said they saw no difference. Turatto took this as showing that the “just noticeable difference” between the items being distinguished is above the size of the attentional effect so the effect of attention on speed is not conscious. A 10% difference in speed is well above the differences that people can see consciously when they are presented for longer periods, but Turatto argues that for these short presentations the just noticeable difference is larger—that is, it takes a larger difference to be consciously perceived.

There is a difficulty with his experimental procedure though. There are well known problems in asking subjects for same/different judgments. Whether the subjects say ‘same’ or ‘different’ depends not only on their percepts, but also on their decision processes, including how big an apparent difference has to be before they regard it as reflecting reality. These issues are nicely analyzed in (Anton-Erxleben et al. 2010, 2011). When Anton-Erxleben et al. corrected for these deficiencies in another same/different experiment, they found effect sizes that are in the vicinity of other paradigms from the Carrasco lab. The effect size is slightly smaller but as they note, that is probably due to inferior sensitivity of the same/different paradigm. (Similar points apply to Kerzel et al. 2010.) One of the conclusions I would draw is that the notion of a “just noticeable difference” in its usual applications is defective because noticeability is not a perceptual property but rather the result of an interaction between perception and cognition. I will not go into these issues further here. However, even if Turatto’s methodology is flawed, the issue raised is a good one. How do we know that the effects in Carrasco’s attentional experiments are in fact conscious?

Figure 7: A version of one of the stimuli used in (Carrasco et al. (2004). Fixate at the dot in the center and move your attention to the left patch without moving your eyes. If you can manage that “covert attention”, the patches should look to have about equal contrast. If you attend to where you are pointing your eyes (the center) you should be able to visually appreciate that the right patch has higher contrast. I am grateful to Marisa Carrasco for supplying this figure.

The stimulus in Figure 7 was one of the stimuli used by Carrasco and her colleagues (Carrasco et al. 2004) in the first experiment that demonstrated that attention affects perception by changing the qualities of perception, in this case increasing apparent contrast. The method used was the same as described earlier in connection with Figure 6—in fact this experiment was the model for the experiment of Figure 6. Subjects were asked to report the tilt of the patch that was higher in contrast after their attention was attracted to one side or the other by a dot as in Gobell & Carrasco (2005). The result was that when the 22% patch was attended it was treated by subjects as the same in contrast as the less attended 28% patch.[11] In order to make the judgment, subjects were shown examples of higher and lower contrast. (Contrast is a measure of the difference between light and dark portions of a stimulus.)

As I mentioned, similar experiments have shown that attention increases apparent color saturation, apparent size of a moving pattern, apparent speed, apparent flicker rate, apparent spatial frequency (more about what that is below), apparent motion coherence and apparent time of occurrence—the attended event seems to appear about 40 ms before the unattended event. As I mentioned, the subjects have to take in what parameter the experimenter is talking about—saturation, spatial frequency, gap size, contrast, etc. and then decide which stimulus is greater with respect to that parameter before they can answer the target question about orientation or side that something is on. This is more complex than any certified unconscious perception task that I know of. Further there is positive evidence, summarized in Stanislas Dehaene’s recent book on consciousness (2014) that “[m]ulti-step calculations will always require a conscious effort” (p. 95).

What further can be said about whether the effect is conscious? I would be remiss if I did not mention that when you look at a good reproduction of the Carrasco stimuli (Figure 7) you can just see the effect for yourself. (Don’t stare for more than a second or two though since adaptation will set in.) It can take a bit of practice to learn to do “covert attention”, i.e., to move your attention without moving your eyes though. (In my 2010 paper I included a figure, Figure 2 on p. 32, one of whose purposes was to give the reader practice in covert attention.) Of course you have as much time as you like to see the effect, whereas in the experiments described you have very little time. Still, what counts for the argument I am making is the effect itself, not its timing. A further difference between just seeing the effect for yourself and the experiments described is that they utilize different types of attention, endogenous for your personal demonstration and exogenous in the experiments described. Endogenous and exogenous attention have been shown to produce roughly comparable effects in Carrasco’s experiments, though in some paradigms some exogenous attention is required for endogenous attention to be efficacious (Botta et al. 2014).

I think many people are convinced of the effect because they can just experience for themselves. Not everyone can though as with almost any visual phenomenon. Of course we all know the dangers of relying too heavily on introspective judgments since they are easily manipulated. There is a line of experimentation that addresses part of the issue.

Ana Chica and her colleagues (Chica et al. 2011; Chica et al. 2010) have done a series of experiments that directly address visibility.

Figure 8: The sequence of events in (Chica et al. 2010) starting from the upper left. ITI = intertrial interval, ISI = interstimulus interval, in this case the period between the offset of the cue and the onset of the stimulus. Reprinted by permission of NeuroImage.

The Chica et al. experiment (the 2010 version) presents subjects with tilted patches that are designed to be on the threshold of conscious perception and subjects were explicitly asked whether they saw the target (after making an orientation judgment). Subjects were strongly encouraged to be conservative in saying they saw the target. They were supposed to avoid “false alarms”, i.e., saying there was a target when there was no target, and they saw periodic messages indicating how well they had been doing in avoiding false alarms. In 25% of the trials there was no target.

First subjects saw a fixation point inside the middle of 3 boxes (pictured on the upper left side of Figure 8), then there was a brief cue consisting of a square around one of the boxes. Then the target—a patch oriented either to the right or the left—could appear for 16 ms (even briefer than in Carrasco’s experiments). Next, subjects had to indicate by pressing keys—within 2 seconds—which way the patch was oriented. They had to choose one of the keys whether they saw something or not, i.e., this was a “forced choice” experiment. Then they indicated whether they saw the target or not. The experimenters adopted a procedure—tailored to each subject’s perceptual abilities—to make sure the target was at the threshold of visibility—for that subject. They started each subject with a patch of sufficiently high contrast to see the stimulus. Every 16 trials they lowered the contrast until the subject was not detecting at least 25% of the patches (by the “Did you see the stimulus” test). If the percentage of avowedly seen patches went below 60%, they increased the contrast.

The main result was that the proportion of avowedly seen patches was much higher for “validly cued targets,” i.e., when the cue was on the box that had the patch than when the cue was invalid, i.e., on the box on the opposite side or neutral (when the cue was on the middle box where no target ever appeared). In addition, the reaction time for the cued patches was much shorter than for uncued patches. Chica et al. also collected brain imaging data that suggested unsurprisingly that the valid cues attracted attention to the cued side, and more interestingly, that when the subjects saw the patch despite invalid cuing (i.e., the cue was on the opposite side), the cue had often failed to attract attention.

This experiment suggests that attention can affect whether a target is consciously visible or not. The subjects were not probed, however, on the issue of whether they actually made their judgments on the basis of the consciously visible tilt. However, when subjects reported not seeing the target, they were at chance on reporting the tilt. And when subjects reported seeing the tilt, they were substantially above chance. This is not the profile one sees in blindsight or in unconscious priming where subjects report not seeing the stimulus at all; but more significantly the tight relationship between consciously seeing the stimulus and being able to judge the tilt does suggest that they were reporting the tilt on the basis of the conscious perception.

Chica’s experiments are relevant to the consciousness of Carrasco’s stimuli in another way. Chica’s stimuli were presented very briefly: 16 ms in the experiment just described. Carrasco’s stimuli were presented for longer, up to 100 ms in some experiments. In addition, Chica’s contrasts were very low, as befits stimuli that were supposed to be at the threshold of visibility. The experiment described above does not report contrasts but in other papers with somewhat more complex experiments along the same lines (Botta et al. 2014; Chica et al. 2013), the contrasts required for 50% detection were about 3%; high detection seems to require up to 10% contrast. In Carrasco’s experiments, much higher contrasts are almost always used. I conclude that the reasons against the “blindsight” analogy in Chica’s experiments apply even more strongly to Carrasco’s methodology.

Given the high rates of conscious vision of 16 ms stimulus presentation even at lower contrasts than most of those used in Carrasco’s experiments, I will ignore the issue of brevity of stimulus presentations in the discussion to follow.

Keith Schneider (2011, 2006; Schneider & Komlos 2008) has argued that Carrasco’s results are based on salience rather than perceptual variables such as perceived contrast, gap size, flicker rate, spatial frequency, etc. (Recently, Schneider and Jake Beck have written a draft of a paper on this topic. Rather than ascribe any specific view to a paper in draft, I will discuss the issue of salience—stimulated by their remarks—but from my own point of view.) I believe that the Carrasco lab is correct in their experimental and methodological disagreement with Schneider (Anton-Erxleben et al. 2010, 2011), however it would be digressive for me to discuss the issues involved in any detail here. I believe though that it is possible to get some insight without going into those issues.

A crude version of a salience objection treats salience as a “response bias” in the sense of a behavioral disposition to respond (in the basic Carrasco paradigm illustrated in Figure 7) by choosing the attended item. The idea is that when faced with a choice between gaps, the subject is disposed to choose not the gap that looks larger but rather the attended gap. This account is ruled out by a control in many of the Carrasco experiments of asking the subject to report the properties of the smaller gap or the patch that is lower in contrast. The attended side is still boosted in apparent contrast or gap size though the effect can be slightly smaller in magnitude so there is a small effect of “response bias” together with a main effect on perception. Carrasco also showed that choosing the lower contrast patch or smaller gap did not take any extra time, ruling out a version of the behavioral disposition objection that adds on an “inversion of response”.

A more sophisticated salience objection alleges a “decision bias” in the sense of a post-perceptual feature of the cognitive process involved in making a decision of how to respond. All such accounts that I know of are ruled out by the fact, mentioned above, that the effect is substantially perceptual in nature. In addition, Carrasco showed that the effect works for some properties but not others (Fuller & Carrasco 2006). As I have mentioned a number of times, it works for saturation but not hue. Both experiments involved a procedure like that in Figure 6. In the saturation version, subjects were asked to report the tilt of the more “colorful” stimulus, where the stimuli differed in color saturation. In the hue version, subjects were asked to report the tilt of the “more bluish” stimulus, where stimuli differed along a blue/purple continuum. The result: there is an attentional effect on saturation but not hue. A cognitive decision bias should equally affect both saturation and hue. If the subjects are not aware of any difference in hue between the attended and unattended sides, it would seem that the “salience” perspective would say they would choose the attended side. But they don’t. Another possibility is that the bias is perceptual in some way, say a matter of perceptual prediction (Hohwy 2013). In either case, the conclusion is that the effect is substantially perceptual and cannot be due simply to any kind of a cognitive decision bias toward choosing attended stimuli.

Whatever understanding of salience the salience objection appeals to, salience must be or be associated with a perceptual property, i.e., a property that is genuinely represented in vision. Some say (Prinz 2012) that the perceptual properties that are involved in vision are limited to a small set whose basic low level representations are products of sensory transduction: shape, spatial relations (including position and size), geometrical motion, texture, brightness, contrast and color. In other words, according to this “lean” theory of perceptual properties, though we speak loosely of seeing something as a face or as a case of causation, in reality seeing-as is limited to a small list of properties that are the output of peripheral sense organs. Others (Block 2014b; Siegel 2010) argue for a more expansive list of genuinely perceptual properties.

How do we know which properties are perceptual? We know that contrast, size, speed, spatial frequency (roughly stripe density), etc. are perceptual properties because they participate in perceptual phenomena, for example in perceptual adaptation and perceptual popout. I mentioned the waterfall illusion in which staring at a moving stimulus makes a stationary item seem to move in the opposite direction. And I’m sure every reader is familiar with color afterimages. Adaptation is a ubiquitous perceptual phenomenon that can be used to show that size, speed, stripiness, etc. are perceptual properties. Note that I am not trying to define the notion of a perceptual property in terms of …perception. The point rather is that perception is a natural kind and the perceptual nature of a representation is revealed in participating in phenomena in that kind (Block 2014b). By these tests, for example, there is evidence that certain face and facial emotion-related properties are perceptual. Viewers seeing an array of objects including one face can pick out the face very quickly on the basis of “parallel search”, just as they can pick out a red object in a sea of green objects. Similarly there are many adaptation effects for faces and facial expressions.

Is salience a perceptual property in this sense? Attention is important to both cognition and perception, but attention can be perceptual. In order to explain the effect of attention on increasing the duration and magnitude of the tilt after-effect (and the improvement followed by impairment in discrimination) as described earlier, the visual system would have to track or register attention or where or what one is attending to in addition to being affected by attention. As I will explain in the next section, there is an open question of whether the visual system does much by way of tracking attention.

In discussions of salience there is often a conflation between salience as a perceptual property and the genuine perceptual properties that are involved in attracting attention, like high contrast or speed or sudden changes in position. We commonly speak of a saliency map in the sense of the map of locations in the visible environmental layout in terms of whether they are likely to attract attention. The perceptual properties here do not involve salience itself but rather differences with nearby locations in visible feature dimensions (Itti & Koch 2000), for example in visible motion or appearance or disappearance. People also speak of a saliency map in the brain, meaning the increased activations that correspond to attended areas. If salience is supposed to be something other than attention itself, that is If Beck and Schneider are giving an explanation that is a genuine alternative to Carrasco’s, they have to show that salience is the kind of perceptual property that is registered in the visual system and that can combine in an additive fashion with contrast to affect adaptation as in the tilt after-effect. I know of absolutely no evidence for such a thing.

Many sources of evidence contribute to our knowledge of the fact that attention increases apparent contrast. John Reynolds et al. have shown that attention boosts responses in individual neurons in monkeys (2000). They developed a model of the mechanisms of this and more complex effects involving multiple stimuli (Reynolds & Chelazzi 2004). Many brain-imaging studies have shown similar effects. See sections 4.6 and 4.7 of Carrasco (2011) where much of this work is summarized. At the behavioral level, attention increases sensitivity roughly as if contrast were increased and similarly, attention can mimic the effects of increased contrast on making a stimulus visible (as in the Chica experiment mentioned earlier). And as mentioned earlier, attention increases adaptational effects as if contrast were increased. Every result involving “salience” that I have seen is just a redescription of effects of the sort mentioned.

Here is a way of seeing the emptiness of appeals to salience: As mentioned earlier, at the neural level, there are two main types of response functions by which attention increases the firing rate of neurons, multiplicative and additive. I mentioned a recent paper that compares simulations of neural responses of these sorts to behavioral data in order to ascertain which of the types of amplication are mainly being used by the visual system. Though the multiplicative models work pretty well, one additive model works very well. Thus we have strong evidence for the functional relation between attention and the increase in contrast responses in in the visual system. For simplicity, let us focus on the multiplicative mechanisms: In multiplicative gain, the response of the neuron is multiplied by a constant factor. For example, a neuron that responds to orientation will give a large response to its preferred orientation and a smaller response to other orientations to the extent that they are distant from the preferred orientation. (For example, a neuron that likes vertical lines will give a large response to vertical lines.) Since multiplying a larger number by a constant produces a larger effect, multiplicative gain is most effective for the preferred orientation. A second multiplicative response function is response gain in which the sensitivity of the neuron is multiplied by a constant factor. The effect is one of ratcheting up the response to stimuli of every orientation. The widely accepted normalization model of attention (Reynolds & Heeger 2009) explains the balance of these two mechanisms (and of additive gain) in terms of factors such as the relative size of the target and the attentional field. The attentional function of a given neuron can show more multiplicative gain or more response gain or more additive gain depending on these factors. Here is my point. We can answer the question of what the difference between these response functions is with respect to increasing apparent contrast. For example, multiplicative gain increases the apparent contrast more for the preferred orientation and response gain increases apparent contrast more for unpreferred orientations. What is the answer to the corresponding question for salience? Does multiplicative gain increase salience more for preferred or unpreferred orientations? Is it the same as for contrast? If so, then maybe “salience” is being used as a synonym for contrast. A different answer would be: multiplicative gain and response gain are equally mechanisms of salience. In this latter case it looks as if “salience” is just being used to mean attention. To repeat the general point: Those who advocate a “salience” explanation of the phenomena have to show that there is a property that is (1) perceptual, (2) not contrast and (3) acts in the ways indicated above.

Sometimes the issue is put in terms of “phenomenal salience” (Wu 2014). I think this way of talking just muddies the waters. Perceptual properties can operate in both conscious and unconscious perception. (At least: it would be an amazing discovery that there is a perceptual property that only appears at the conscious level.) Attention—at least exogenous attention—operates in unconscious perception in a similar manner to conscious perception (Chica et al. 2011; Kentridge et al. 2008; Norman et al. 2013). Further, it has recently been discovered using optogenetic methods that top-down activation of visual area V1 is about the same in awake and anesthetized mice (Zhang et al. 2014). This top-down activation involved feedback from a brain area in the mouse that corresponds to a locus of voluntary attention in humans. If salience is a perceptual property, it should be operative in unconscious perception. So the salience issue is an issue about perception, not about just conscious perception.

The upshot is that it is not at all clear how a salience objection would work, so the burden is on those who advocate it to explain it. I raised the issue of whether we are aware of where we are attending in connection with whether we are aware of salience, so I now turn briefly to that question.