Metrics in Eye Tracking Research - METR2026

Motivation

Quantitative metrics are essential for describing visual behavior in eye tracking research. However, the current metric landscape is fragmented and characterized by inconsistent nomenclature and ambiguous definitions. Identical metrics carry different names; conversely, metrics with different names share similar definitions (Sharafi et al.). Furthermore, non-transparent or software-specific implementations introduce additional inconsistencies. Given the exact same data, different software packages can produce different values for identically labeled metrics (Dörzapf et al.). This lack of consistency and clarity hinders understanding, reproducibility, and interpretability, and it particularly hinders newcomers from comprehending the field.

Metrics can be described by formal mathematical notation or algorithmic procedures on the one hand, and by labels or textual descriptions on the other, with an often ambiguous mapping between the two. The core problem is that operationalization choices in studies are frequently implicit because they are encapsulated in software, leading readers to infer meaning from labels that do not uniquely determine the underlying computation.

METR2026 aims to address these challenges by fostering a community dialog that brings together different perspectives, approaches, and current best practices. The workshop will produce a synthesis of this dialog, which will serve as the foundational blueprint for a living document: A Landscape of Eye Tracking Metrics.

Goals

• Collect best practices for applying, evaluating, and reporting metrics
• Provide an overview of existing implementations, definitions, or theoretical links
• Collect opinions on future directions in the field
• Highlight different perspectives and approaches
• Provide didactical value for newcomers
• Emphasize inconsistencies and problems
• Offer a living, commentable resource, with non-prescriptive commentary
• Help make implicit choices explicit in studies

Limitations and other perspectives

One might argue that the described problems depend on perspective:

Perspective 1: Studying a cognitive process involves selecting gaze measures that are plausibly informative about the cognitive process in the study design, treating them as proxies whose validity and interpretation are context-dependent and non-unique. Varying metric names, definitions, and implementations across literature and tools lead to aforementioned problems.

Perspective 2: Operationalizing a cognitive process involves explicitly stating a construct → mechanism → observable mapping (how cognition links to the eye tracking metric) and treating this mapping and its assumptions as contestable parts of the empirical claim. The implementation of the eye tracking metric is part of the operationalization and must therefore be examined and reported by the researcher. Thus, inconsistent metrics arise logically from context-specific mechanisms, assumptions, and operationalizations.

Further caution must be applied when considering the scope of the resulting document.

• Metrics are context-specific and hardware-dependent. It is impossible to make general statements about the interpretation and validity of metrics. Still, it can be useful to compile well-supported theoretical links from prior work—mapping constructs to eye tracking observables—so metric choices are grounded in concepts that have been shown to work in comparable contexts.
• Metrics may be too abstract or ambiguous, or it may simply be impossible to tie those concepts to a clear, formal description. Arguably, tying down these concepts to a single formal notation would not do them justice, would be unsuitable due to different hardware constraints, and would restrict the flexibility needed at the forefront of science.
• Due to the above reasons, providing a complete, coherent, internally consistent document is not possible and should be avoided. Showcasing different approaches prevents the collection document from being read in a normative way. The document highlights diversity and thus encourages discussion and engagement.