3 The neglected dimension of consciousness: Time and the flow of consciousness

But is that all? One dimension of our experience that is often neglected is time. Of course, time is an implicit component of previous experience, however, it may also be revealing to consider time by itself. In fact, living organisms seldom encounter a static image in isolation, but are instead confronted with a flow of temporally-correlated sensory inputs (Schwartz et al. 2007). Imagine for instance a tennis match, and picture the tennis ball flying over the field. If queried, you could easily estimate where the ball is, but also where it was a second ago and where it will be in a few milliseconds. Event-objects of the conscious mind[2] thus per definition unfold in time and we also act in time: we make use of current and previous input to figure out the most appropriate response predicting their consequences. There is thus a continuum of interdependencies along the time dimension whereby every past moment is integrated with the present and projected into the future, giving rise to the flow of consciousness. The same way we have been thinking about the integration of multiple source of information within a given moment of time, such as multiple features of a single object, there is thus integration across time. A case in point is strikingly vivid perceptual aftereffects, such as the waterfall illusion, where viewing motion in one direction for several seconds causes a subsequently presented static image to move in the opposite direction (Purkinje 1820). Such effects are not limited to basic perceptual features such as motion direction, colour, or orientation, but also affect high-level percepts such as the perceived gender of faces (Webster et al. 2004), numerosity (Burr & Ross 2008), or gaze direction (Jenkins et al. 2006); and they are not limited to fleeting illusions that vanish almost instantaneously, but may persist for days or even weeks (Jones & Holding 1975). This indicates that our current experience is embedded into a continuous flow of previous experience at multiple time scales, ranging from lifelong experience with our environment to short-term, moment-by-moment effects that arise from our most recent encounters, even if just milliseconds ago.

The past thus leaves traces (predictions) that determine the current contents of consciousness. This has the consequence that the contents of consciousness represent an aggregate of imprints from the past and the present moment that jointly promote a sense of stability over time. However, through which mechanism these interdependencies affect our perception is currently unclear. Experimentally, the multiple time-scales of previous experience are particularly evident when subjects are confronted with sequences of multistable stimuli such as the Necker cube.[3] Because the sensory information these stimuli provide by themselves is insufficient to determine perception, they are particularly susceptible to the effects of previous experience. Under these conditions, one can observe two different effects that temporal dependencies entail: on the one hand, an attractive effect, which increases the likelihood of continuing to perceive the same stimulus, and on the other hand a repulsive effect, which increases the likelihood of perceiving something different. The former is often referred to as hysteresis, priming, stabilisation, or perceptual memory, while the latter is commonly known as perceptual adaptation.

Recently, Chopin & Mamassian (2012) studied the temporal dynamics of these serial dependencies, addressing the question of which part of the perceptual history the system retains and how remote and recent experiences differentially determine perception. They observed a remarkable dissociation between long stretches of time that occurred in the remote past (in their case several minutes) and short stretches of time that had just recently occurred (a few seconds ago): while the former had a positive correlation with perception, and thus ensured stability over time (hysteresis), the latter had a negative correlation to perception, that is, it promoted alternative interpretations (adaptation). These two timescales indicate that previous experience can act along at least two separate timescales and hence, that there may be several mechanisms at work. Using functional magnetic resonance imaging, we set out to further elucidate how these effects are implemented in the brain, how the brain entertains these two opposing processes without mutual interference, and what determines their direction (Schwiedrzik et al. 2014). Presenting multistable visual stimuli sequentially, we found that although affecting our perception concurrently, hysteresis and adaptation map into distinct cortical networks: a widespread network of higher-order visual and fronto-parietal areas was involved in hysteresis, while adaptation was confined to early visual areas (areas V2/V3). Importantly, hysteresis and adaptation bear a differential relation with whether or not the stimuli were consciously perceived: while adaptation was present even if the adapting interpretation was not consciously perceived (in agreement with previous reports, e.g., Hock et al. 1996), hysteresis depended on what was previously consciously perceived. Hence, conscious experiences in the past affected the present experience, preserving continuity in time, while unconscious processing had the opposite effect, bringing change and novelty to perception.

This brings us back to the question of neural integration, indicating that even in the case of integration over time, the spatial scale at which neuronal processing occurs determines whether content enters awareness or not: in the case of hysteresis, a conscious moment is integrated in time with another conscious moment, which involves a widespread cortical network, while in the case of adaptation, prior information is only integrated within a local module, which happens irrespective of whether this prior information is consciously experienced or not, similar to Lamme’s “base grouping”. This interpretation fits with results that have been obtained in the auditory domain in which short temporal regularities can be detected unconsciously eliciting a locally generated event-related potential (ERP), termed mismatch negativity (MMN), while detection of long-term regularities depends on conscious perception, which elicits an electrophysiological response known as P300 from a widespread network of brain areas (Bekinschtein et al. 2009; Faugeras et al. 2011).

Together, I propose that these results mesh well with the idea that one of the functions of consciousness is to interpret the world in long timescales, bringing together the now with the past beyond the simple and automatic input-output relations rooted in unconscious processors, thus allowing for the extraction of more complex and abstract regularities. Brain areas with longer time constants such as the prefrontal cortex (Fuster 1973) would extract the world’s statistics from the remote past, creating a model of the world that keeps a stable picture. In contrast, early sensory areas with short time constants act on shorter timescales, sampling the world for alternative interpretations, thus allowing the system to stay tuned to deviations from the long-term statistics (Clifford 2012; Snyder et al. under review).[4]

While previous studies and established experimental paradigms have mostly focused on the “nowness” of conscious perception, it appears that much remains to be learned about consciousness and its fundamental phenomenological characteristics such as its flow and our sense of stability over time. In fact, considering that much of what we currently know about the NCC stems from “static” paradigms, and by those I mean paradigms that do not take the temporal context in which the stimuli unravel into account and thus only inform us about what has “changed” in consciousness, we in fact only have access to the neural processes related to the update of contents in consciousness, while the mechanisms at play in the maintenance or continuity of our experience remain obscure (but see Kleinschmidt et al. 2002). The present might be known, but the flow is still a mystery!

Thus I propose that a full account of consciousness requires a reappraisal of our object of study in which we incorporate the temporal flow of consciousness as another fundamental property that needs to be explained. This calls for a dynamic view in which a train of conscious states (the flow) would be captured as successions of neuronal meta-assemblies, each with a particular relaxation time, followed by phase transitions, which determine the time of emergence, dominance, and dissolution of a state that leads to another perceptual cycle (Melloni & Singer 2010; Varela 1999). In this framework, the rate-limiting factor for the formation of a new meta-assembly would correspond to the time needed to establish stable phase relations; while the different time constants promoting stability vs. change may be implemented by different oscillatory frequency bands, in addition to the intrinsic time window of integration of a given area (Chaudhuri et al. 2014).

In summary, much remains to be discovered about consciousness and its neural correlates, but significant progress has already been made since the seminal paper by Crick & Koch (1990) that got the field going about twenty-five years ago. Victor Lamme’s experimental work and theoretical proposals on the role of feedback connections and reentrant activity in conscious perception have been central to bringing us closer to an understanding of the neural processes that allow us to “see”. His paper in this volume contains an erudite review of the present knowledge against a background of thought provoking hypotheses, e.g., that the function of consciousness is to solve difficult perceptual problems. In Lamme’s view, consciousness is there to create, while unconscious processes are there to utilise. In close analogy to any creative process, consciousness in Lamme’s framework is slow and takes time and resources to develop. In a way, his proposal is that it is all about distance, or time. This is a powerful intuition, and an idea worth exploring, yet its contribution does not end there—more than that, it serves as a reminder of a central characteristic of consciousness that is not yet fully explored, namely that conscious experience unfolds at a characteristic spatio-temporal scale, and that it is this flow in space/time that brings the strong sense of experiential stability and continuity. The interwoven temporal scales of the flow of consciousness that bring about the “unity of experience” remain the next challenge, and maybe the one that will finally unlock the mystery of consciousness.