Gaze-Based Interaction on Commodity Hardware: A Survey of Appearance-Based Estimation, Selection Techniques, And Multimodal Error Correction

Abstract

Eye gaze is the fastest natural pointing signal available to interactive systems, but for three decades accurate gaze interaction has remained tied to dedicated infrared eyetracking hardware. The convergence of deep appearance-based gaze estimation, ubiquitous front-facing cameras, and on-device neural inference has opened a credible path to gaze interaction on unmodified consumer devices, and with it a distinct research agenda whose problems differ from those of laboratory eye tracking. This survey reviews that agenda across three layers. First, appearance-based gaze estimation: the CNN architectures, large-scale datasets, calibration strategies, and head-pose handling techniques that determine achievable accuracy on RGB cameras, and the persistent gap between benchmark and in-use performance. Second, gaze as an input modality: dwellbased selection, the Midas-touch problem, and the design space of confirmation mechanisms, smoothing filters, and target-aware interfaces that convert noisy fixation estimates into reliable selection. Third, the emerging layer of multimodal and implicit error correction, including physiological and behavioural signals as post-hoc feedback for detecting and repairing erroneous selections — an approach that relocates robustness from prevention to recovery. The survey synthesises evaluation practices across the three layers, identifies the accessibility deployment context as both the most demanding testbed and the strongest motivation for the field, and sets out open problems in personalisation, drift, privacy-preserving on-device processing, and the extension of gaze interaction to spatial computing platforms.