Eye Fixations and Diagrams

Name and Affiliations of the Instructor: Mary Hegarty, Department of Psychology, University of California, Santa Barbara, CA 93106.
Email: hegarty@psych.ucsb.edu

Benefits: In this tutorial, the attendees will gain an understanding of the main methods of measuring eye fixations on diagrams and how these data are coded and analyzed to make inferences about internal cognitive processes. This will enable attendees to better interpret and critically evaluate the results of studies that use this measure. It will also give them an introduction to the advantages of using eye fixation data and the effort involved in setting up an eye-tracking laboratory and analyzing eye fixation data. The tutorial will also identify problems in the analysis and interpretation of eye fixations that might lead to the development of new software tools for analyzing and interpreting this type of data.

Content: Although there is a long history of eye-tracking research on such topics as reading, scene perception and visual search, this method has only recently been applied to the study of problem solving and reasoning with diagrammatic representations (Rayner, 1998). There has been considerable interest in the use of eye fixations to study diagrammatic and spatial reasoning. With the increase in availability of eye tracking technology and the interest in this methodology, it seems timely to devote a tutorial to the use of eye tracking research on diagrams. The overall goal of the tutorial will be to give attendees an understanding of what is involved in measuring eye fixations during cognitive processing of diagrams and what types of inferences can and cannot be made from these data. The tutorial will begin with a brief introduction to the history of eye-movement studies and a description of the main methods of collecting eye fixation data that are currently in use. This will include a videotape of a typical eye tracking experiment. Then we will discuss basic research on how the eyes move during comprehension tasks (fixations, saccades etc.) and the relationship between eye fixations and attention. This will also include a discussion of the assumptions underlying interpretation of eye fixations, such as the eye-mind hypothesis (Just & Carpenter, 1976) Then we will review some of the main ways in which eye fixations are coded and analyzed. It will cover two main stages of eye-fixation analysis (1) visualization of the data to gain a general understanding of the patterns of fixations and (2) derivation of quantitative methods from the data, which can be entered into statistical analyses. We will discuss some of the main measures that have been derived from eye-fixation studies with diagrams in the past. For example, we will discuss the identification of regions of interest (ROIs) in a diagram that represent different meaningful entities (e.g., the legend in a graph, a component of a machine, a particular region of space in a map). Then we will examine how these are used to derive quantitative measures, for example:

• Gaze duration on regions of interest in the of the diagram at different stages of task performance
• Number of fixations on different regions of interest at different stages of task performance
• Number of transitions between different regions of interest
• Sequence of fixating different regions of interest

This review will focus on measures used in studies of comprehension, reasoning and problem solving with diagrammatic displays, broadly defined (including diagrams, maps and graphs). For example, representative tasks to be discussed might include chess problem solving, (e.g., Charness, Reingold, Pomplun & Stampe, 2001) graph comprehension (Shah & Carpenter, 1998; Peebles & Cheng, 2003), mechanical reasoning (Hegarty, 1992), mental rotation (Just & Carpenter 1985) and insight problem solving with diagrams (Grant & Spivey, 2003; Knoblich, Ohlsson, & Raney, 2001). In reviewing and evaluating these studies, the focus will be on identifying the types of inferences that we can make from eye fixation data and also the types of inferences that one cannot make from eye fixations. For example in a given study, it might be possible to identify which region of a map a person is thinking about at a given time, but not exactly what they are thinking about that region. The presentation will end with a discussion of how eye fixations are jointly influenced by top-down factors (such as knowledge, expectations and task demands) and bottom up factors such as relative salience of elements in a display. The goal would be to use an interactive format in the tutorial, in which I do not just lecture, but bring examples of eye-fixation data and invite participants to interpret these data either alone in small groups, and then share their interpretations. I would also like to invite a discussion on the state of the art in eye-tracking analysis and perhaps identify ways in which our analysis tools could be improved. If time permits, it might be interesting to break people into discussion groups for a short time and then report back to the whole group, to facilitate this type of evaluation.

Audience: The main goal of the tutorial is to enable researchers to interpret and critically evaluate the results of eye tracking studies. As such, the tutorial is of \ to any attendees who read the empirical literature on cognitive processing of diagrammatic representations. The tutorial should be of particular interest to cognitive psychologists, education researchers, and researchers in human-computer interaction, because eye tracking is used in all of these disciplines. I would also hope that it would be attended by researchers interested in information visualization, software development, and analysis of spatially distributed data, because I believe that there is great potential for improving our methods of visualizing and analyzing eye tracking data, and it would be outstanding if the tutorial could inspire some work on these topics. Finally, although I have been conducting eye fixation studies on diagrammatic representations for some time, I certainly do not consider myself to be an expert on all of the potential uses of these data or all of the techniques used to analyze these data, so I would welcome participation by other individuals who have used this technique in their research.

References:
Carpenter, P. A. & Shah, P. (1998). A model of the perceptual and conceptual processes in graph comprehension. Journal of Experimental Psychology: Applied, 4, 75-100.
Charness, N. Reingold, E. M. Pomplun, M. & Stampe, D (2001). The perceptual aspect of skilled performance in chess: Evidence from eye movements. Memory & Cognition, 29, 1146-1152.
Grant, E. R. & Spivey, M. J. (2003) Eye movements and problem solving: Guiding attention guides thought. Psychological Science, 14, 462-466.
Hegarty, M. (1992). Mental animation: Inferring motion from static diagrams of mechanical systems. Journal of Experimental Psychology: Learning, Memory and Cognition, 18(5) 1084-1102.
Just, M. A. & Carpenter, P. A. (1976). Eye fixations and cognitive processes. Cognitive Psychology, 8, 441-480.
Just, M. A. & Carpenter, P. A. (1986). Cognitive coordinate systems: Accounts of mental rotation and individual differences in spatial ability. Psychological Review, 92. 137-172.
Knoblich, G., Ohlsson, S. & Raney, G. E. (2001). An eye movement study of insight problem solving. Memory & Cognition, 29, 1000-1009.
Peebles, D. & Cheng, P. C-H. (2003). Modeling the effect of task and graphical representation on response latency in a graph-reading task. Human Factors, 45, 28-46.
Rayner, K. (1998). Eye movements in reading and information processing: Twenty years of research. Psychological Bulletin, 124, 372-422

Instructor background: Mary Hegarty is a professor in the Department of Psychology at the University of California, Santa Barbara. She received her BA and MA from University College Dublin, Ireland. She worked as a research assistant for three years at the Irish national educational research centre before attending Carnegie Mellon, where she received her Ph.D. in Psychology in 1988. Dr Hegarty’s research interests are in spatial thinking and she is the author of numerous articles and chapters on comprehension of diagrams, mechanical reasoning and spatial abilities. She is a fellow of Association for Psychological Science (APS), is a former Spencer Postdoctoral Fellow and is a consulting editor of Journal of Experimental Psychology: Learning, Memory & Cognition. Her current research is funded by the Office of Naval Research and the National Science Foundation. In her Ph.D. dissertation, Dr Hegarty conducted one of the first studies of eye fixations on diagrams. In this study she examined how people integrate text and diagrams to understand how simple machines work. This work revealed different patterns of eye movements on diagram of mechanical systems when people were reading about the static configuration of these systems and when they were reading about how the motion of the machines (Hegarty & Just, 1993). This lead to a series of studies on “mental animation” of static diagrams, in which she used eye fixation data to develop a model of how people infer motion from static diagrams (Hegarty, 1992; Hegarty & Sims, 1994). Dr Hegarty has also conducted eye-fixation studies in the area of mathematical problem solving (Hegarty, Mayer & Green, 1992 Hegarty, Mayer & Monk, 1995) and inferences in text comprehension (Revlin, & Hegarty, 1999) In current research she is using eye fixations to examine interpretation of weather maps and to evaluate the effectiveness of different visual displays (Canham & Hegarty, 2004).

Relevant Publications:
Canham, M. & Hegarty, M (2004, July). Influences of knowledge on eye fixations while interpreting weather maps. Paper presented at the annual meeting of the Cognitive Science Society, Chicago, IL.
Hegarty, M. (1992). Mental animation: Inferring motion from static diagrams of mechanical systems. Journal of Experimental Psychology: Learning, Memory and Cognition, 18(5) 1084-1102.
Hegarty, M. (2004). Mechanical reasoning as mental simulation. Trends in Cognitive Sciences, 8, 280-285.
Hegarty, M. Haarslev, V. & Narayanan, N. H. (2002). Diagrammatic reasoning. Kuntscliche Intelligenz, 4, 38-39.
Hegarty, M. & Just, M.A. (1993). Constructing mental models of machines from text and diagrams. Journal of Memory and Language, 32, 717-742.
Hegarty, M., Mayer, R. E & Green, C. E. (1992). Comprehension of arithmetic word problems: Evidence from students' eye fixations. Journal of Educational Psychology, 84, 76-84.
Hegarty, M., Mayer, R. E. & Monk, C. (1995). Comprehension of arithmetic word problems: A comparison of successful and unsuccessful problem solvers. Journal of Educational Psychology, 87, 18-32.
Hegarty, M. & Sims, V. K. (1994). Individual differences in mental animation during mechanical reasoning. Memory & Cognition, 22, 411-430.
Revlin, R. & Hegarty, M. (1999). Resolving signals to cohesion: Two models of bridging inferences. Discourse Processes, 27, 77-102.

Requirements List: I should not need anything other than a standard overhead projector and a computer projector for a PC.

back to workshops / tutorials