Other projects in our lab include social neuroscience (how to optimize community intervention programs in high-risk neighborhoods) and the use of real-time feedback neuroimaging to break drug addiction (in collaboration with Stephen LaConte at Virginia Tech).
This is a page of supplemental information for D. M. Eagleman and T. J. Sejnowski, Motion Integration and Postdiction in Visual Awareness, Science, 287(5460), 2000, and for follow-up Technical Comments.
Eagleman, D.M. & Sejnowski, T.J. (2000) Motion integration and postdiction in visual awareness. Science. 287(5460): 2036-8. [Full text]
Eagleman, D.M. & Sejnowski, T.J. (2000) The position of moving objects: Response. Science. 289(5482):1107a.[Full text]
Eagleman, D.M., Sejnowski, T.J. (2000) Flash Lag Effect: Differential latency, not postdiction: Response. Science. 290(5494): 1051a.[Full text]
Eagleman DM, Sejnowski TJ (2007). Motion signals bias position judgments: A unified explanation for the flash-lag, flash-drag, flash-jump and Frohlich effects. Journal of Vision. 7(4): 1-12. [Full text] [Demos of the Experiments]
The goal of this line of research is to close in on a connection between physical mechanisms present in neural tissue and the perceptual functions that these mechanisms embody. Please follow the links to the left for an overview of the flash lag effect, the explanations previously forwarded to explain it, our hypothesis to explain our results, and several demonstrations.
“Life must be lived forward, but can only be understood backward” – Soren Kierkegaard, The Journals of Kierkegaard
(Please note: this page is not currently maintained, and has not been updated since 2007)
The flash-lag effect is a visual illusion wherein a flash and a moving object that appear in the same location are perceived to be displaced from one another (MacKay, 1958; Nijhawan, 1994). In recent years, two explanations have been forwarded, motion extrapolation and latency difference.
The first proposed explanation for the flash-lag effect is that the visual system is predictive, accounting for neural delays by extrapolating the trajectory of a moving stimulus into the future (Nijhawan, 1994; Khurana and Nijhawan, 1995). In other words, when light from a moving object hits the retina, a certain amount of time is required before the object is perceived. In that time, the object has moved to a new location in the world. The motion extrapolation hypothesis asserts that the visual system will take care of such delays by extrapolating the position of moving objects forward in time.
A second proposed explanation is that the visual system processes moving objects more quickly than flashed objects. This latency-difference hypothesis asserts that by the time the flashed object is processed, the moving object has already moved to a new position (Baldo and Klein, 1995; Whitney & Murakami, 1998; Purushothaman et al, 1998). The latency-difference proposal tacitly rests on the assumption that awareness (what the subject reports) is an on-line phenomenon, coming about as soon as a stimulus reaches its “perceptual end-point” (Zeki & Bartels, 1998).
Motion Interpolation and Postdiction
We have proposed a third alternative (Eagleman & Sejnowski, 1999ab, 2000): visual awareness is neither predictive nor on-line, but is instead postdictive, such that the percept attributed to the time of the flash is a function of events that happen in the ~80 msec following the flash. This postdictive framework is consistent with findings in other fields, such as backward masking in visual psychophysics (Bachmann, 1994), or the color-phi phenomenon (Kolers & von Grunau, 1976). In backward masking, a stimulus followed in rapid succession by a second stimulus can block or modify the perception of the first one. In the color phi phenomenon, 2 colored dots presented sequentially within a small time and distance will appear to have changed color in the middle of their apparent trajectory. Since the viewer cannot know what the color of the second dot will be until having seen the second dot, the only explanation is that the conscious percept attributed to the ‘trajectory’ of the dots is formed after the second dot has ‘arrived’ at its destination.
We find that the perception attributed to the time of the flash depends on events in the next ~80 msec after the flash, and also that the flash seems to reset motion integration mechanisms, perhaps by temporarily diverting attention. In this way, we can draw a correspondence between the flash-lag effect and the Frohlich effect (Frohlich, 1923), wherein the first position of a moving object entering a window is misperceived.
Scroll down to watch demonstrations of the stimuli we used to elucidate several aspects of the flash-lag phenomenon.
To directly pit extrapolation into the future against interpolation of the past, we designed this two-alternative forced choice task. Subjects were instructed to indicate whether a flash of light (white disk) occurred above or below the center of a moving ring (Fig 1a in manuscript; ring speed 360 deg/sec). Beginning with the frame following the flash, the ring took one of 3 randomly interleaved trajectories: continuing, stopping*, or reversing direction. The initial trajectory of the ring (up to and including the frame with the flash) was identical in all three conditions; thus, if motion extrapolation were occurring, the predicted trajectory should be the same. Instead, the perceived position of the flash relative to the ring was independent of the initial trajectory. In the case of the continuous trajectory, subjects perceived the ring to be about 6 degrees ahead of the flash (as would be expected from previous studies of the flash-lag effect). However, in the presentations wherein the moving ring stopped, there was no illusion of displacement, indicating that the pre-flash movement was not sufficient to yield the flash-lag illusion. When the ring reversed direction immediately after the flash, participants perceived the ring about 6 degrees above the flash, which is the same size, but opposite direction, of the continuous case. In other words, what participants report to have seen at the time of the flash depends on the events after the flash.
If visual awareness were predictive, the same initial trajectory would lead to the same extrapolation. Our results replicate a recent demonstration by Whitney and Murakami, in which the perceived displacement of a flash was influenced by a motion change that occurred after the flash. In our experiment, by directly comparing stimuli with an identical pre-flash trajectory to three different post-flash trajectories, we can demonstrate that the perceived displacement of the flashed and moving stimuli is a function of the movement after the flash. (Note that in the stopped case, there is no flash-lag effect at all*). Thus, forbearing any precognitive explanations, we are left to suggest that the perception attributed to an event at time to depends on what happens in to < t < to + h, where the magnitude of h will be determined by experiment 3.
Dim lights Embed Make sure you set your player to play every frame, or you might miss the flash…
Note on the movie: for the purposes of demonstration, this movie shows the 3 conditions sequentially (continuous, reversed, stopped), and the flash appears exactly in the middle of the ring each time. In the real experiments, conditions were randomly interleaved, and the flash was put in different positions for quantification of the illusory displacement. Also, the size of the presentation is much reduced for the movie, and the frame rate will play differently on different browsers.
*Note that the flash-terminated condition had been previously demonstrated by Romi Nijhawan, both in a 1992 ARVO abstract, and also in a talk at the Salk Institute in 1999, at which I was in attendance.
In the next experiment, to further verify that the initial trajectory has no bearing on the direction of the perceived displacement, we designed the flash and ring to appear on the screen at the same time, with the movement of the ring beginning only in the next frame (Fig 1b in manuscript). Thus, there is no trajectory (no previous motion) from which to extrapolate. The results are unchanged from Fig. 1a, strengthening the conclusion that only events after the flash determine the perception. This paradigm is analogous to the ‘flash-initiated cycle’ used by Khurana and Nijhawan (1995); however, the present result makes their interpretation of motion extrapolation untenable. The juxtaposition of our Fig. 1a and 1b suggests that the flash resets the motion integration in the visual system, making motion after the flash effectively like motion that starts de novo (as in Fig. 1b). One explanation may be that the flash temporarily redirects attention
Dim lights Embed Note on the movie: for the purposes of demonstration, this movie shows the 3 conditions sequentially (up, down, and stopped), and the flash appears exactly in the middle of the ring each time. In the real experiments, conditions were randomly interleaved, and the flash was placed in different positions for quantification of the illusory displacement. Also, the size of the presentation is much reduced for the movie, and the frame rate will play differently on different browsers.
To determine how much information after the flash the brain collects for its decision, we designed stimuli analogous to those in Fig 1b, but which include a direction reversal: immediately after the flash, the ring moves in one direction before reversing direction after a variable number of frames (Fig. 2 in manuscript). If the visual system only employs information in the next 10 – 20 msec after the flash (as might be implied from Fig. 1a and 1b, and from a latency difference hypothesis), then the trajectory of the ring after that time window should not affect the percept. Contrary to that hypothesis, movement up to 80 msec after the flash influences the percept. We find that 67 – 80 msec of unidirectional movement is necessary to approach the illusory displacement measured in Fig. 1a and 1b. As the amount of time before the reversal is reduced, the illusory displacement is lessened, until with only 26 msec before reversal, the flash lag effect is effectively canceled out (as though the ring were stopped). With only one frame before reversal, the illusion turns the other direction. These data are consistent with a temporally-weighted spatial averaging that takes place over ~80 msec after the flash. The results are the same when the ring has a lifetime of only 6 frames after the appearance of the flash (as opposed to remaining on screen until the end of the trial; n=2 of the 6 subjects). Physiological mechanisms for the spatiotemporal integration may involve a form of temporal recruitment, the process by which motion signals in the neural tissue are combined over time. However, most of the available literature implicitly assumes that motion integration would occur over the time before the flash, that is, the visual system would collect information until the time of the stimulus, with perceptual processing following on-line. Our data indicate instead that visual awareness employs information after the flash. The direction reversal experiment indicates that the position of the moving object is interpolated as a point within the integrated path, and given the results of Fig. 1a and 1b, our interpretation is that the flash serves to reset the motion integration.
Dim lights Embed Note on the movie: for the purposes of demonstration, the flash appears exactly in the middle of the ring each time. In the real experiments, the flash appeared in different positions for quantification of the illusory displacement. Also, the size of the presentation is much reduced for the movie, and the frame rate will play differently on different browsers.
For more information, please see our manuscript: D. M. Eagleman and T. J. Sejnowski, “Motion Integration and Postdiction in Visual Awareness”, Science, 287(5460), 2000.
To further examine our interpretation, and to test the latency difference model, we next separated the temporal coincidence of the flashed and moving object.
Subjects were instructed to adjust the angle of a “pointer” line (flashed for 1 frame) to point at the beginning of the trajectory of the moving ring (Fig. 3 in manuscript). The pointer was flashed, follcondition the flash and ring appeared on the same frame (delta t = 0); in the remaining 4 conditions, the ring did not appear until some delay after the single frame with the flash (13 ms < delta t < 53 ms). The stimulus was repeated after a 1 sec delay, and subjects were allowed to see a condition as many times as they wished before committing to an answer. Regardless of the delay, subjects adjusted the pointer to indicate a position an average of ~6 deg ahead of the actual starting position of the ring (same magnitude as the displacements in Figs. 1 & 2). This demonstrates that subjects do not perceive the starting position of the moving object (an observation known as the Frohlich effect), but perceive instead, in our interpretation, an interpolation of its past positions. The latency difference model is not supported, for the outcome of a ‘race’ between a flash and a moving object to a perceptual endpoint should be changed by starting the flash first. Instead, the entirety of the flash-lag effect in 1b can be explained by the fact that the starting point of a moving object is interpolated (misperceived). Further, it seems the traditional flash-lag effect (Fig 1a) is well explained by our suggestion, above, that a flash resets motion integration.
Dim lights Embed About the movie: for the purposes of demonstration, this movie shows 4 presentations for each different delay time. In the actual experiment, subjects were allowed to watch as many presentations as they wanted, and were instructed to adjust the angle of the flashed ‘pointer’ to point at the middle of the ring in its starting position. When a subject was satisfied, he/she would hit the return key, and the next trial would start, with a randomly chosen delay (delta t). In this demonstration movie, the pointer is pointing at the veridical starting position of the ring (0 degree displacement). The titles on this movie are for demonstration purposes only, and were not shown to subjects. Further, the size of the presentation is reduced for the movie, and the frame rate will play differently on different browsers, which means that the delay times will not correspond exactly to the titles shown unless your browser plays this movie at 72 Hz .
DOES LIMITED EXTRACELLULAR CALCIUM DIRECT A NEW KIND OF PLASTICITY?
O.J.-M. Coenen*; D.M. Eagleman; V. Mitsner; T.M. Bartol; A.J. Bell; T.J. Sejnowski Computational Neurobiology Lab, Salk Institute, La Jolla, CA, USA
A class of synaptic learning models, in which a sum of postsynaptic activity from many neurons drives plasticity, has generally been considered biologically infeasible. After all, postsynaptic cell bodies may be far apart, and there are no backward signals known to sum activity in a terminal-specific manner. However, some specialized synapses, known as glomeruli, become ensheathed by glial cells, and we suggest that these structures may allow for just such a postsynaptic summation. The ensheathment may force enclosed, neighboring dendrites to share a limited resource: extracellular calcium (ECa). We propose the theory that the ECa concentration in glomeruli may encode the level of spike activity in postsynaptic cells. We investigate here cerebellar glomeruli, where dendrites from granule cells swirl around a mossy fiber terminal, and the ensemble is tightly ensheathed in an astrocyte. Computer analyses of 3D simulated glomeruli, with realistic channel kinetics and Monte Carlo modeling of calcium diffusion using MCell, indicate the range of conditions under which ECa will be proportional to the sum of granule cell activity. We also show how these ECa changes can be interpreted from an information-processing point of view, generating a novel learning rule for control of plasticity at the mossy fiber/granule cell synapse. This learning rule approaches a sparsely distributed and statistically independent coding in the parallel fibers. Although traditional neural models emphasize only neurotransmitters and connectivity, these results highlight the need to quantitatively address the 3D context in which axons and dendrites are found.
Click here for an MPEG movie of David Eagleman’s Neural Growth simulator. (This movie may not work in Windows; try playing it on a Unix machine).
For more information, please contact us about our upcoming manuscript: D. M. Eagleman, O. J-M. D. Coenen, T. Bartol, V. Mitsner, A. J. Bell, P. R. Montague, and T. J. Sejnowski, “Cerebellar glomeruli: Does limited extracellular calcium implement a sparse encoding strategy?”, in preparation, 2001.
Computational Properties of Extracellular Calcium Dynamics in Mammalian Neural Tissue
One fifth of the mammalian brain is extracellular space (ECS). The extracellular space is not empty, but instead comprises a complex network of proteins and a variety of molecular species. One extracellular species, calcium, holds a prominent position as one of the most important messengers known in the brain. However, calcium exists in low concentrations in the ECS, and its diffusion is slowed by the restricted volumes of the extracellular space – therefore, normal neural activity may cause calcium to flow out of the ECS faster than diffusion can fill it in. As opposed to the traditional view that extracellular calcium exists at a stable concentration, we explore the hypothesis that calcium concentrations may change. Such changes would be expected to carry significant functional impact, due to the many roles calcium plays. The hypothesis is explored both at the biophysical and theoretical levels. At the biophysical level, we explore the dynamics of external calcium changes under a wide range of reasonable parameter settings. At the theoretical level, we attempt to interpret how such fluctuations may serve as information-bearing signals in the nervous system.
Specifically, we have employed Monte Carlo simulations, finite differencing schemes, and numerical analyses to develop a computational method capable of describing populations of neural elements, and the fluids that communicate through the spaces between them. Such a method has allowed me to address several issues, the broadest interpretations of which can be cast as a set of questions: Is neural tissue engineered to allow – or prevent – fluctuations in extracellular ionic concentrations? If fluctuations can occur, do they carry information? What are the computations carried out by that flow of information?
Beginning with a model of pre-synaptic terminals, we show that reasonable assumptions about the kinetics of consumption and diffusion will lead to rapid, local changes in external calcium. The exact size of the calcium signal depends on several parameters. For example, the cleft width might be used by the tissue as a control parameter: changing the gap between elements can amplify or squelch the calcium signal. The distribution of calcium channels will also critically influence the calcium fluctuations: by clustering the channels, the local signal amplitude is greatly increased. In some circumstances, the density of the calcium channels can become high enough that the total calcium influx becomes limited by the speed of extracellular diffusion. We calculate that the calcium signal will not travel far through the tissue – the signal will remain approximately as local as neurotransmitter signals such as glutamate. Finally, we show that changes in the average background rates will significantly change the average calcium concentration available to any given terminal.
Moving the focus to dendrites, we demonstrate that action potentials propagating along a dendrite can induce large calcium fluctuations, lowering the external calcium available to overlying pre-synaptic terminals. Since neurotransmission depends on the availability of external calcium, it may be that a post-synaptic neuron can employ back-propagating action potentials to modulate the transmission probabilities of overlying afferent terminals. The geometrical distribution of calcium sinks again influences the time and spatial extent of fluctuations in external calcium. In particular, clusters of co-active dendrites can prolong and amplify an external calcium fluctuation. This latter effect provides a natural substrate for a computational mechanism that indexes (locates) specific volumes of neural tissue on rapid time scales.
We suggest that the detailed structure of the extracellular space, in combination with the three-dimensional distribution of calcium sinks, will play a role in neural information processing. We discuss many roles extracellular calcium is known to serve – however, throughout this work, we highlight two roles that are directly interpretable from a computational perspective: neurotransmission and tissue re-organization. The steep dependence of neurotransmitter release on calcium allows for interpretation of calcium dynamics at short time scales. Over longer epochs, we appeal to the calcium-dependence of certain call-adhesion molecules to propose that calcium dynamics may drive learning in neural tissue. Both explorations will underscore the dual role that calcium plays: when an ion moves from outside to inside, its presence in the cytoplasm is as important as its absence from the ECS.
In conclusion, the geometrical arrangement of neural tissue forces neighboring neural elements to share a resource that is necessary for neural transmission, but is in limited supply on short temporal and spatial scales. As we demonstrate, the resting levels of external calcium are not sufficiently high to protect against large decrements in this important resource. Instead, it seems as though the tissue is engineered so that external calcium levels are meant to fluctuate dramatically; given the functional importance of external calcium, we are led to the strong suspicion that external calcium fluctuations are an important class of information-bearing signal in the nervous system.
Wiest, M.C., Eagleman, D.M., King, R.D., Montague, P.R. (2000) Dendritic spikes and their influence on extracellular calcium signaling. Journal of Neurophys iology. 83(3):1329-1337.
King, R.D., Wiest, M.C., Montague, P.R., Eagleman, D.M. (2000) Do extracellular calcium signals carry information through neural tissue? Trends in Neuroscie nces 23(1):12-13.
Eagleman, D.M., Montague, P.R. (1999) Calcium dynamics in the extracellular space of mammalian neural tissue. Biophysical Journal 76(4):1856-1867.
Eagleman, D.M., Montague, P.R. (1998). Computational properties of peri-dendritic calcium fluctuations. Journal of Neuroscience. 18(21): 8580-8589.
Eagleman, D.M., King, R.D., Montague, P.R. (1998). Interaction of nitric oxide and external calcium fluctuations: a possible substrate for rapid information retrieval. Prog Brain Res 1998;118:199-211.
(Also pubished in Nitric Oxide and other diffusible messengers in development, plasticity, and disease. Mize,R.R., Friedlander,M.J., Dawson,T.J., Dawson,V.M., Eds. Amsterdam: Elsevier Press.)
Eagleman, D.M., Wiest, M.C., Montague, P.R. (1998). Extracellular calcium dynamics as a rapid information-bearing signal in neural tissue. Soc. Neurosci. Abstr. 24:526.
Montague, P.R. (1996) The Resource Consumption Principle: attention and memory in volumes of neural tissue. Proc. Nat. Acad. Sci. USA 93(8):3619-3623.
Montague, P.R. (1996). General properties of the resource consumption principle of neural function. J Physiol Paris, 90(3-4):239-42.
Person, C., Eagleman, D.M., King R.D., Montague PR. (1996) Three-dimensional synaptic distributions influence neural processing through the resource consumption principle. Journal of Physiology, Paris, 90(5-6): 323-5.
Human Decision Making: The computational role of dopamine delivery
David Eagleman, Chris Person, P. Read Montague
(Work performed at Baylor College of Medicine)
Fluctuations in dopamine delivery at target structures seems to represent an evaluation of future events that can be used to direct learning and decision making. To examine the behavioral consequences of this interpretation, we gave simple decision making tasks to 66 human subjects and to a network based on a predictive model of mesencephalic dopamine systems. The human subjects displayed behavior similar to the network behavior in terms of choice allocation and the character of deliberation times. The agreement between human and model performances suggests a direct relationship between biases in human decision strategies and fluctuating dopamine delivery. We also show that the model offers a new interpretation of deficits that result when dopamine levels are increased or decreased through disease or pharmacological interventions. The bottom-up approach presented here also suggests that a variety of behavioral strategies may result from the expression of relatively simple neural mechanisms in different behavioral contexts
Eagleman, D.M., Person, C., Montague, P.R. (1998). The computational role of dopamine delivery. Journal of Cognitive Neuroscience. 10(5): 623-630.