Low-level Action Schemas Support Gaze Guidance Behavior for Action Planning and Execution in Novel Tasks
Authors: Keshava, A.; Nezami, F. N.; Neumann, H.; Izdebski, K.; Schueler, T.; Koenig, P.
Category: neuroscience
Date: 2022-02-09

AO_SCPLOWBSTRACTC_SCPLOWEye movements in the natural environment have primarily been studied for over-learned activities such as tea-making, sandwich-making, and hand-washing, which have a fixed sequence of actions associated with them. These studies indicate an interplay of low-level cognitive schemas that facilitate task completion. However, it is unclear if this strategy is also in play when the task is novel and a sequence of actions must be planned. Here, to study attention mechanisms, we recorded gaze and body movements in a naturalistic sorting task. In a virtual environment, subjects moved objects on a life-size shelf to achieve a given order. To study the action planning and execution related gaze guidance behavior we also controlled the complexity of the tasks, i.e. the objects were to be sorted based on a single or two different object features. We show that subjects are close to optimal while performing the easy tasks and are more sub-optimal while performing the more complex tasks. Fixations aligned with action onset showed gaze is tightly coupled with the action sequence and less so with task complexity. Task complexity elicited differences at specific time points where oculomotor behavior could be categorized into searching, guiding, checking fixations. Task complexity also affected the scan-paths on the task-relevant regions of interest during action planning and execution where subjects exhibit a greater search and action monitoring behavior in more complex tasks. Furthermore, task complexity also affected the timing of first fixations on the task-relevant regions of interest for action planning but not for action execution. Finally, the scan paths and the temporal sequence of first fixations suggest a just-in-time strategy of gaze allocation. Based on the spatio-temporal features of gaze control, we can conclude that eye movements during novel tasks correspond closely to low-level cognitive schemas that support the production and execution of actions.

Author SummaryEye movements in the natural environment have primarily been studied for over-learned and habitual everyday activities (tea-making, sandwich-making, hand-washing) which have a fixed sequence of actions associated with them. These studies typically categorize eye movements related to low level action plans of locating the object of interest, directing the body or hand to the object, and monitoring the action execution. In this study, we were interested in generalizing these task-oriented eye movements for novel tasks that are not associated with an inherent action sequence. To that end, we asked subjects to sort objects based on object features on a life-size shelf in a virtual environment as we recorded their eye and body movements. We then analyzed gaze features relative to the action onset i.e. when the hand made contact with an object. By only studying the sequence of fixations based on past, current and next actions, we aimed to elucidate the general characteristics of gaze behavior relative to action planning and execution and independent of object identity. With our analysis, we could categorize fixations into searching, guiding, monitoring fixations in a data-driven manner. Based on the spatio-temporal features of gaze control, we can conclude that novel tasks predominantly elicit eye movements corresponding to low-level action plans that support the production and execution of actions.