Memories of attention

Two of the most central aspects of human cognitive processes are attention and memory. Attention enables us to find objects that are of importance to us in complex environments, and to focus our awareness and selectively act on those objects to achieve our goals. Memory is fundamental to who we are as individuals, directly linking us with our past experiences, and via those experiences guiding our future behaviours. The fundamental nature of these processes is clearly revealed when considering the devastating effects when they break down in diseases such as Alzheimer’s.We can see that there is a relationship between attention and memory: when we focus attention on an object we are more likely to encode and later recall that object from memory. However, my colleagues and I have recently been investigating whether specific attention processes can in fact be encoded into memory, and be retrieved and influence future attention states at a later time. Such processes are important because they enable us to use past experiences to undertake future tasks.

Consider two forms of stimulus that can orient attention. In the first, a display containing three squares is presented. When one of these squares briefly flashes (the cue) attention rapidly (within 1/10th of a second) orients to the flash. This is an automatic process: we can’t stop attention from moving to the flash even if we try. This orienting system is ancient and evolved to protect us in hostile environments. For example, if the sudden movement in the periphery is a snake about to strike, it is critical that we rapidly orient towards this stimulus and produce appropriate actions.

A second kind of stimulus that triggers orienting of attention is social in nature. Humans are highly social animals, and to successfully interact with other individuals we need to be able to understand their current states and future actions. One means of predicting future behaviour is by understanding where a person is attending, as attention directed to an object often precedes and predicts whether they will act on that object. It is now clear that when we observe another person attending to a particular location, we experience a powerful urge to look in the same direction. For example, you may encounter a person on the street intently staring up at a specific location, perhaps a window. You will often find yourself automatically looking up to the same location. We can demonstrate this in an experimental situation, where a sudden gaze shift in an onscreen photo causes people to orient to the same location, even when they are trying to resist this urge.

Specific neural mechanisms based on excitatory and inhibitory states control these attention shifts. For example, an important effect discovered by Posner and Cohen (1984) is known as inhibition of return (IOR). As noted above, when attention is captured by the flashing square, people are quicker to detect targets at that cued location because attention has been attracted to that place. However, shortly after this an inhibition effect emerges, where detection of targets at the cued location is inhibited. It is assumed that this inhibition reflects an important mechanism that enables us to find targets in complex and cluttered environments. Thus, after orienting attention to examine an object, when attention is withdrawn it is critical that we do not keep returning attention to this uninteresting object. If we did keep returning to already examined objects, we might never find what we are looking for. It is the inhibition mechanism that prevents this return of attention and ensures we orient to new objects while searching for the thing of interest.

Importantly, attention mechanisms such as this inhibition have generally been assumed to be very transient. Thus the inhibition may linger for only about three seconds (e.g. Samuel & Kat, 2003), but then it decays. The transient nature of attention processes has been fundamental to most theories, and it is this fundamental assumption that we have challenged. That is, we have attempted to show that even though processes that control attention may be transient, because attention constantly moves to new states as we search the environment, it is still possible for specific states of the attention network to be encoded into memory, and to be retrieved later.
 

Retrieval of attention processes from memory

The obvious question is why such a system of encoding states of attention into memory would have evolved? Our proposal is that often the process of searching for a particular object cannot be completed during one processing episode. In such circumstances it would help to reinstate prior attention states when we resume the search. For example, consider the following scenario: You are searching your cluttered kitchen for a mislaid knife. After orienting attention to various candidate objects the doorbell rings, and you leave the room for a few minutes to greet guests. When returning to the kitchen, how is search for the knife resumed?

We suggest two mechanisms. The first is the most obvious, where you explicitly recall what you were doing (‘I was looking for the knife’). However, as we age, this explicit retrieval process can fail, and we stand there thinking: ‘I know I was looking for something, but what was it?’ However, we propose a second implicit mechanism that aids our search, of which we are not consciously aware. This second implicit mechanism is retrieval of prior attentional states such as inhibition. Hence we are less likely to look towards objects previously attended because they are associated with inhibition, and so we will orient to novel objects. Via retrieval of such inhibition processes, a momentum to search towards new objects is produced.

For inhibitory attention states to be retrieved requires that they be stored in a form accessible to memory systems. Our proposal is that inhibition is associated with object-based representations, and when those objects are re-encountered the inhibition associated with them is retrieved. However, we needed some new techniques to demonstrate this: we had to use stimuli that people are extremely efficient at encoding to memory and retrieving at a later time. To this end, Sarah Grison, Klaus Kessler and I used faces (Tipper et al., 2003).
As shown in Figure 1, a face was presented to the left and right of a fixation cross. The participant’s task was to simply locate as fast as possible a green stimulus flashed on one of the two faces with a left or right key press. On some trials one of the faces flashed red, and participants were asked to ignore this and prevent response. This red stimulus was the cue, it oriented attention to the face, which would activate IOR when attention was subsequently withdrawn.

In a first study the interval between the cue display and subsequent target display was a couple of seconds. This was similar to previous studies of IOR, and of course standard effects were observed, confirming that our new technique produced typical IOR effects. However, in subsequent studies the interval between observing the pair of faces in the cue display and then re-encountering them later was greatly increased. In one study the faces were not seen again for about three minutes, with 40 items intervening between these exposures. In another study the faces were not seen again for about 13 minutes with over 100 other events intervening.

Even though these intervals between cue and target displays were hundreds of times longer than those typically studied, we did observe IOR effects. Interestingly, unlike the short-term effects, hemisphere differences were observed in every study examining these long-term effects. That is, IOR was observed for only one of the faces, and this was generally the left face. Although further work will be required to understand these hemisphere differences, it might be that processing of faces in the right cortical hemisphere is more efficient, and hence later retrieval of this small and elusive inhibition was more efficient for these left-sided faces that project to the right hemisphere.

These long-term IOR effects therefore provide evidence that the inhibition can be associated with the identity of the face towards which attention was drawn by the sudden onset cue. Furthermore, when the face was re-encountered some minutes later, part of the recognition process also retrieved the prior history of cueing, and thus inhibition was retrieved.
Interestingly this retrieval of inhibition from memory seems to be a completely unconscious process. We showed this by asking participants to consciously recall over which face the red cue had been presented (Kessler & Tipper, 2004). We found the counterintuitive result that people are actually worse than chance: they are more likely to recall that the cue was presented on the face that was not cued. However this somewhat bizarre result can be explained in terms of retrieval of prior inhibitory processing. That is, the cued face was associated with inhibition which was not available to conscious awareness, hence when trying to recall where the cue was, response was biased away from the inhibited cued face.
 

Retrieval of gaze-evoked attention shifts

But can shifts in our attention during social interactions also be encoded into memory and retrieved later? This has remained an open issue because there are similarities and differences between gaze and sudden-onset cues.

Both types of stimulus trigger very fast shifts of attention within 1/10 of a second, and they are both automatic and obligatory in that people have to initially orient in the same direction as the cue, although they can quickly regain control and orient in other directions shortly afterwards. However, they are mediated by different neural systems. My colleague at Bangor, Bob Rafal, has shown the importance of the superior colliculus in the mid-brain for sudden-onset cues (Rafal et al., 1988); whereas Dave Perrett and his colleagues at St Andrews (Perrett et al., 1985) have shown that the superior temporal sulcus in the cortex mediates gaze shifts of attention. A second contrast is that, unlike sudden-onset cues, gaze cues did not seem to activate subsequent IOR. Rather, after observing a gaze shift attention was drawn to the same location very transiently, and there were no longer any effects after about one second.

That gaze effects were so brief and did not evoke subsequent inhibition has been puzzling for some time. However, Alex Frischen and I have recently shown that this is not in fact the case (Frischen & Tipper, 2004). In experiments where we extended the interval between gaze cue and target much further than previously studied we did indeed find longer-term effects, and these were inhibitory.

However, another contrast between these two forms of orienting attention did at first seem to be quite clear. That is, unlike sudden-onset cues, gaze cues did not seem to be associated with the identity of the person who was making the eye-movement. In a range of experiments Alex Frischen and I could find no evidence that the identity of the face had any effect, and no evidence that the gaze direction could be retrieved when a face was re-encountered at a later time. It remained puzzling to us that gaze shift could remain divorced from the person who was making the gaze shift, and that there was no encoding in to memory. This was particularly surprising when sudden-onset cues could activate attention states (inhibition) that could be retrieved at a later time. We continued to pursue this issue and discovered that in a particular set of circumstances, retrieval of prior gaze shifts could be detected (Frischen & Tipper, in press).

In these experiments we used much richer stimuli than previously tested: they were full-colour images of people ranging in age, sex and race. Half of them were famous, half had never been seen by the participants before. Figure 2 shows the typical procedure. After observing a gaze shift, this face was not encountered again for approximately 3 minutes with 40 intervening other face stimuli. We discovered that for the famous face, when gaze was oriented to the left side of space, there were consistent long-term cueing effects that could be replicated. Although this effect is only observed under a specific set of conditions it is of importance when considering that the standard view in the literature was that these gaze-cueing effects were very transient, decaying within about one second.
 

An intimate relationship

There are occasional hints of dissatisfaction with the fragmentary approach to understanding psychological processes. Attention and memory are often studied by separate groups of researchers who sometimes appear to have little to say to one another. However, our recent work has tried to emphasise the intimate relationship between these cognitive systems. Thus attention processes, such as inhibition and excitation, rather than being transient states that dissipate rapidly, may under some circumstances leave a trace in memory. When retrieval cues are strong enough, such states of attention might be reactivated and assist in our current processing, such as when we search for objects of importance to us. These processes are fundamental to our interactions with the environment and each other.

Professor Steven Tipper is at the University of Wales, Bangor. E-mail: [email protected].

References 

Frischen, A. & Tipper, S.P. (2004). Orienting attention via observed gaze shift evokes longer-term inhibitory effects: Implications for social interactions, attention and memory. Journal of Experimental Psychology: General, 133, 516–533
Frischen, A. & Tipper, S.P. (in press). Long-term gaze cueing effects: Evidence for retrieval of prior states of attention from memory. Visual Cognition.
Kessler, K. & Tipper, S.P. (2004). Retrieval of implicit inhibitory processes: The impact of visual field, object identity and memory dynamics. Visual Cognition, 11, 965-995.
Perrett, D.I., Smith, P.A.J., Potter, D.D. et al. (1985). Visual cells in the temporal cortex sensitive to face view and gaze direction. Proceedings of the Royal Society, London B, 223, 293–317.
Posner, M.I. & Cohen, Y. (1984). Components of visual orienting. In H. Bouma & D. G. Bowhuis (Eds.) Attention and performance X. Hove: Lawrence Erlbaum.
Rafal, R.D., Posner, M.I., Friedman, J.H., Inhoff, A.W. & Bernstein, E. (1988). Orienting of visual attention in progressive supranuclear palsy. Brain, 111, 267–280.
Samuel, A.G. & Kat, D. (2003). Inhibition of return: A graphical meta-analysis of its time-course and an empirical test of its temporal and spatial properties. Psychonomic Bulletin and Review, 10, 897–906.
Tipper, S.P., Grison, S. & Kessler, K. (2003). Long-term inhibition of return of attention. Psychological Science, 14, 19–25.