Why do we see the way we see? How is our perception different from the way a photograph is acquired on the sensor chip of a digital camera? It seems obvious that we do not see an image made of individual pixels but an integrated, smooth, colourful, and vivid image. What is the neural substrate of this marvellous capacity that makes us feel and experience the way we do? These are the central questions that Victor Lamme sets out to address in his paper The Crack of Dawn: Perceptual Functions and Neural Mechanisms that Mark the Transition from Unconscious Processing to Conscious Vision.
This is by no means an easy task, even when one stays away from the difficult problem of qualia or “what it is like to be” (Nagel 1974). The question of how awareness arises has preoccupied philosophers and scientists for centuries, and while significant progress has been made in recent decades we are still far from reaching a conclusion (Dehaene 2014; Koch 2004). One thing is clear however: success in understanding the neural machinery that instantiates consciousness rests on identifying the fundamental features that characterise a state as conscious and that distinguish it from unconscious states. A remarkable discovery of the past century is that a significant portion of all mental operations, including fairly complex ones such as decision-making and perceptual categorisation, can be carried out unconsciously. Take the case of language: while it seems effortless to understand the words that you are currently reading, you do not have conscious access to the syntactic processes that ultimately allow you to grasp the relations between the elements of this sentence and thus its meaning. These complex mental operations occur “behind the scene” of consciousness. Given that so many intricate processes can operate unconsciously, one cannot but wonder what consciousness is good for. Which mental processes require consciousness, if any? And if so, what really distinguishes conscious from unconscious cognition? Victor Lamme offers a stimulating and comprehensive review of processes in vision that can be performed outside the realm of awareness. The list is long and may be surprising (also see Kouider & Dehaene 2007), ranging from detection of simple (e.g., oriented lines) and complex features (e.g., faces; Almeida et al. 2013; de Gardelle et al. 2011; Del Zotto et al. 2013), to mathematical operations such as abstract comparisons between quantities (Greenwald et al. 2003), to triggering of motor plans (Dehaene et al. 1998), and even error-related responses to stimuli that fully escape our consciousness (Cohen et al. 2009).
What do we need consciousness for, then? Lamme proposes that consciousness is required when all sources of information need to be integrated. For instance, when we see a face, we can not only detect that is a face, a process that can be performed unconsciously, but also identify it as that of our friend Billy, whom we have not seen in ten years and that we remember warmheartedly from our childhood. Consciousness brings this unified moment in which all comes together: previous experiences are retrieved from memory (e.g., do we have reason to like Billy?) and unified with the context of the current experience (e.g., where are we now?), but also intertwined with predictions for future actions (e.g., would we like to engage in a conversation?). Thus, in one single moment, past, present and future come together and form a unified conscious experience. Many scientists nowadays agree that conscious experience provides an abstract summary of all available sources of information, from which many features are filtered out and reinterpreted in a format that is most useful for further actions, thoughts, deliberations, and chain operations that cannot be processed by non-conscious processors (Lamme this collection; Baars 2002; Dehaene 2014; Melloni & Singer 2010). Hence, what reaches our perception is a highly processed, “interpreted” version of the world. One key intuition is that the unification and “interpretation” of the experience that reaches our consciousness is achieved through the activation of myriads of neurons that signal individual features, but that it is by virtue of integrating their information through dynamic interactions (for example via synchronous coordination of their activity or via feedback processes) that a coherent experience across senses, space, and time comes about.
An important caveat is that integration of information per se is unlikely to distinguish conscious from unconscious processing as integration of many features can also proceed unconsciously (Dehaene et al. 1998; Gaillard et al. 2009; Lin & He 2009; Melloni et al. 2007; Melloni & Rodriguez 2007; Mudrik et al. 2014). In fact, integration through convergence is a key principle of the wiring of the brain, which explains the mere existence of feature-selective neurons that respond to motion, shape, or complex stimuli such as faces, and that process information in an unconscious manner. If it is not integration per se, then what kind of integration are we talking about?[1] We and others (Melloni & Singer 2010; Thompson & Varela 2001; Varela et al. 2001) have previously argued for a distinction between local and global integration, and proposed that the spatial scale of integration differentiates between unconscious and conscious states: unconscious processing is observed when local integration occurs within the divergent-convergent feedforward architecture; conscious processing however requires long-range integration through neural synchronization, which integrates information across the various levels of the cortical processing hierarchy.
Indeed, in recent years, a wealth of experimental studies (Aru et al. 2012; Gaillard et al. 2009; Hipp et al. 2011; Melloni et al. 2007; Melloni & Rodriguez 2007) have provided support to the idea that long-range integration through synchronous coupling is a mechanism for conscious perception, and that the spatial scale of synchronisation strongly correlates with the perceptual outcome. For example, we have shown that masked words are only consciously perceived when accompanied by a burst of long-distance synchronization in the gamma band, while unconscious processing, even up to a semantic level, elicits only local gamma oscillations (Melloni et al. 2007; Melloni & Rodriguez 2007). Although controversy still persists as to whether long-range integration necessitates the involvement of particular brain areas (Dehaene 2014; Edelman & Tononi 2000) or not (Lamme this collection; Melloni & Singer 2010), it is reassuring to witness some convergence on the results that have even led to clinical applications (e.g., coma classification, King et al. 2013). In his most recent work, Victor Lamme now also assigns a central role to the spatial scale of the integration for consciousness, joining an ever-increasing number of researchers proposing long-range integration as key to consciousness (Dehaene & Changeux 2011; Edelman & Tononi 2000; Melloni & Singer 2010; Thompson & Varela 2001). An interesting point of divergence from other theories is that while Lamme assigns a particular role to feedback and horizontal connections in the integration of information for consciousness, other theories, including our own, hypothesise that it is the synchronisation of neural populations that glues all experiences into one, thereby instantiating consciousness. As empirical data and theoretical considerations continue to accumulate, we expect that this and other pressing challenges such as identifying how far is “long” in the brain, or whether “long” involves the activation of specific neural cell populations, specific areas, and/or a specified number of nodes will become addressable.
However, imagine those questions have been addressed and we know that integration on a particular spatial scale is key to consciousness; would we have understood what consciousness is or how it comes about? Here I propose that we would not, as any theory that does not account for two fundamental, hitherto neglected aspects of conscious experience will fall short of explaining consciousness. In particular, our experience is never an island in isolation, but instead is shaped by previous knowledge, by priors that stem from the preceding context or from our history of learning. These priors determine our perception; and thus understanding how they become integrated is paramount to explaining consciousness. However, an even more pressing problem is that conscious experience unfolds over time, whereby the recent past moulds the current moment, which in turn creates predictions for moments to come, i.e., the future. How all those temporal processes intertwine and define our experience (the flow of consciousness) is something that most research has neglected. In the following sections I will review current research that we and others have undertaken with the purpose of raising awareness of these overlooked integrative properties of conscious experience and the challenges that they entail for the study of consciousness.