Metrics in Eye Tracking Research
METR is a community initiative to make eye tracking metrics transparent, comparable and reproducible. It started as a workshop at the 2026 ACM Symposium on Eye Tracking Research & Applications (ETRA) in Marrakech, Morocco — and is now a growing working group turning the workshop's results into a joint journal paper and a living reference of metric definitions.
The problem we set out to address
Quantitative metrics are essential for describing visual behavior — but the current metric landscape is fragmented. Meaning often cannot be read from a metric's label, because the actual computation is hidden in software.
Same metric, different names
Heat map, fixation map, attention map — one concept, many labels (Sharafi et al.).
Same name, different definitions
“Dwell time” = ∑ fixations? + saccades? + revisits? Or raw samples in the AOI?
Same data, different values
Different software packages produce different values for identically labeled metrics on identical inputs (Dörzapf et al.).
No systematic guidance
Which metric fits which study context? Newcomers in particular are left guessing.
The consequence: reproducibility, interpretability and accessibility suffer — especially for newcomers to the field.
The workshop: METR at ETRA 2026
On June 2026 in Marrakech, METR brought the community together for half a day: 7 peer-reviewed paper presentations followed by a 100-minute interactive working session with live polls, a folding spectrogram and dot-voted group boards.
Live poll: “What is dwell time?”
17 votes, 4 definitions from the literature
No consensus: even the majority definition leaves a third of the room computing a different number from the same recording.
Four group boards, dot-voted
Statement debated: “Inconsistent metric definitions are primarily an author and review problem — restrictive standardization is not the answer.”
Categories with minimums, physiological norms, reliability/validity testing, reporting standards; guidelines per modality (screen / VR / mobile).
The lower the feature level, the more standardisation; open-spec computation on all levels, incl. vendors; probabilistic methods for latent constructs.
Rigid taxonomies hinder discovery; standardise reporting and evaluation protocols instead.
General baseline plus application-specific definitions per domain; involve manufacturers.
Consistent across all boards: transparent reporting and evaluation protocols, and “rails, not a rigid system” — best-practice repositories, recipes and community review, revisited regularly.
The 7 workshop papers
Computational Operationalization Choices for Spatial Data Quality in Screen-Based Eye Tracking
The Need for Real-Time Metrics and Related Research Protocols
With a Grain of Salt: When Pupillary Metrics Do—and Do Not—Predict Effort States and Why Reporting Standards Matter
VIVA Stimuli: A Web-Based Platform for Eye Tracking Stimuli
Is this it? Benchmarking Scanpath Metrics for Information Display
Constructing Machine Learning Features from Eye Movement Metrics: Feature Engineering Techniques for HCI Research
As Far as Eye See: Vergence-Pupil Coupling in Near-Far Depth Switching
Published in the ACM Digital Library as part of the ETRA 2026 Workshop Proceedings.
What's next — decided with the community
After the workshop we surveyed the community (22 responses, June–July 2026). The direction below combines the two most-requested working models.
Workshop at ETRA 2026 Done — June 2026
7 papers, interactive working session, community formed.
Community survey Done — July 2026
Top requests: a joint journal paper (19 would contribute) and an open, continuously maintained collection of metric definitions.
Journal paper + living reference In progress
A journal paper about the approach — the argument — and a living online reference where concrete metric definitions live and evolve. Monthly virtual meetings for decisions and synthesis; writing happens asynchronously in work packages.
Yearly METR workshop at ETRA Planned
The in-person anchor of the community (18 of 21 in favour).
Organizers
Jürgen Mottok
OTH Regensburg, Germany
Roman Bednarik
University of Eastern Finland, Joensuu
Join the initiative
Want to contribute to the paper, the living reference or the next workshop — or just stay in the loop? Let us know.