Combining eye tracking and physiology for detection of emotion and workload

Abstract

Peripheral physiological measures such as electrodermal activity (EDA), heart rate and pupil dilation, as well as neurophysiological measures such as electroencephalography (EEG), can inform us about individuals’ cognitive and emotional state. We are interested in exploiting such measures in real life situations. A challenge of interpreting physiological measures as markers of mental state in real life is the lack of context information. We here approach this challenge by relating physiological measures to eye tracking. Participants scanned stimuli that induced different levels of workload (small sets of numbers that needed to be added or not) and different types of emotion (neutral, pleasant and unpleasant pictures). EDA, heart rate, pupil size and EEG were related to the first eye fixation on the stimulus. Our first results indicate that this may be especially helpful in situations related to cognitive workload, e.g. determining whether operators are not only looking at, but are also cognitively processing information that is presented on a screen.

M.Sc. Katharina Lingelbach

PhD student, Neuroscientist, and Psychologist (she/her/hers)

My research focuses on interacting neuronal dynamics of emotional and cognitive processes, decoding approaches of neuronal dynamics using machine learning using electrophysiology and neuroimaging methods