Metrics in Eye Tracking Research - METR2026

"Yet, several of these metrics are identical or similar but carry different names. Conversely, some metrics have similar names but different definitions."

"The lack of standardization in analyzing eye-tracking data poses a significant threat to the comparability and reproducibility of eye-tracking studies in software engineering... Our case studies revealed that the choice of analysis tool can have a significant influence on the experimental results of a study"

"When talking about eye tracking study measurements, it is important to keep in mind that the handling of eye movement metrics is far from consistent... current naming conflicts, e.g., the term scan path as explained in “Gaze Visualizations”... This helps to avoid misunderstandings due to inconsistent naming, e.g., what is called heat map in this article is called fixation map in [44] or attention map in [62]"

"These eye-movement researchers held a variety of definitions of fixations and saccades, of which the breadth seems even wider than what is reported in the literature. Moreover, these definitions did not seem to be related to researcher background or experience. We urge researchers to make their definitions more explicit by specifying all the relevant components of the eye movement under investigation"

"Note that a survey of the works included in the table shows that the nature of the distance—whether it is actual or normalized—is sometimes left undisclosed by the original research, creating potential inconsistency between the formulas used for error in different works."

"Additionally, the lack of standardized protocols or metrics for eye-tracking research complicates the comparison of study results and the development of methods for clinicians to diagnose or monitor AD and MCI."

"Regarding the analysis of the information gathered, it is also necessary to comment that there is still no consensus on the terminology, the name of the metrics, or their interpretation"

"Given that the measurement method and blink detection approach used can affect the determination of blink onsets and offsets, it is difficult to compare study results even when the same metrics are reported."

"The utilization of non-standard terminology, self-defined terms for eye tracking parameters, and occasional confusion between distinct gaze-related attributes contributed to the challenges in comprehending papers... The decision to diverge from established terminology not only complicates the understanding of these studies but also poses a more profound threat: it undermines the broader community’s ability to retrace and replicate the findings presented."

"eye tracking itself lacks consistent definitions and measures, such as fixation and saccade criteria"

"Researchers commonly interpret the cognitive meaning of the same eye tracking metric differently... few studies disentangled the complex relationship between eye tracking metrics and cognitive activities these metrics represent"

"a lack “of a methodology in conducting such experiments, which resulted in terminological inconsistency in the names and formulas of metrics used,"

"Exploring eye-tracking metrics, this study finds out that there is a lack of a procedure in performing such experiments, resulting in a terminological lack of consistency in the names and formulas of metrics used."

"The study’s primary contribution to the body of knowledge is that it identifies the need for standardization of eye tracking analyses to better ensure that future eye tracking studies be more consistent and reproducible. ... This work illustrates how different eye tracking glance/visit analysis parameters chosen by researchers can lead to differing results and research conclusions."

"Spatial accuracy is one of the major characteristics of eye-tracking. There are two ways to define it, where the first is the distance between the ground truth point and the centroid of gaze samples... and the second is the mean distance between the ground truth point and the gaze samples... Furthermore, there is an ambiguity when calculating RMSE. ... This has led to a variety of inconsistent benchmark results, even for identical eye-tracking systems."

"Spatial accuracy is referred to by various terms and calculated through different methods in prior eye-tracking research"

"a key challenge in applying eye-tracking to software engineering is the lack of standardization. Currently, there are no consistent terms, definitions, metrics, or methods in this field. ... Even a comparison of experimental software from providers like Tobii, SMI, and SR Research shows inconsistencies ... it still lacks consistent terminology, metrics, and methodologies in the context of eye tracking research"

"There is not a particular study where the authors have provided a universally accepted definition of fixation in the context of eye tracking research because such definition differs between on-screen system of eye tracking and mobile eye tracking system"

"General eye metrics and tracking face several challenges, including variability in how fixations, saccades, and blinks are measured, as well as reliance on small, contextspecific datasets that limit generalisability to real-world applications. This review highlights the need for policies to standardise eye-tracking metrics like fixations, saccades, and pupil dynamics, ensuring consistency across research and commercial applications."

"inconsistent interpretations of eye-tracking metrics limit generalizability. studies ... often fail to align theoretical constructs ... with the metrics used, leading to interpretative ambiguity"

"Although one might assume that these measures are somewhat self-explanatory, definitions are far from standardized (Godfroid, 2020). Researchers have used one and the same definition to refer to different measures and have used the same measure in different ways across studies"