title: Interacting with an uncertain physical world: probabilistic models of human perception and action
Creators: name: Neupärtl, Nils
Publisher: name: UNSPECIFIED
Year: 2022

description: Humans interact with their environment and its physical laws with ease and thereby demonstrate the ability to predict how dynamical situations unfold. Having an appropriate internal model is indispensable to do so, however, it is unclear how our brain can encompass this wealth of information and complexity of environmental states and dynamics. For instance, dropping trash into a bin while passing by is an effortless, almost unconscious process and yet a significant share of people show tremendous misconceptions when being asked about the exact same dynamics in physical reasoning tasks. This is also true for similar tasks when people are asked to make judgments about other dynamical scenes like swinging pendula or moving objects after curved trajectories. But how can this discrepancy between routine acting and deficient reasoning be explained? An early attempt to explain this discrepancy, especially the non-rational human deviations from optimal behavior, is the reliance on rules of thumbs, often called heuristics. Based on the idea that people’s internal models are likely not able to reflect the environmental complexity and thus need to rely on helpful, yet error-prone approximations of processes and dynamics, heuristics try to reveal the underlying mechanism for specific biases. However, these heuristics usually need to be individually adapted to the problem at hand and do not yield a general explanation beyond the specific task. In contrast, probabilistic models of bounded rationality have been able to quantify and explain these deviations as a consequence of human uncertainties, a priori assumptions about their environment, and internal costs such as effort. With this thesis we want to contribute to the understanding of this seeming discrepancy and reconcile these two phenomena of humans being well tuned to daily interactions and deficient in their reasoning about it using diverse tasks in controlled environments as well as computational models and algorithms describing deviations based on individual constraints. First, we take a look at distance estimations in a judgment and a continuous action control task and the resulting deviations from optimal responses. With respect to physiological constraints, as perceptual uncertainty and action variability, and biased a priori beliefs about the size of familiar objects we describe individual deviations using probabilistic models and yet show the individual’s consistency across tasks and beliefs. Since in both tasks people were constrained on viewing two-dimensional projections of distant objects and thus could only access the visual angle or apparent size they had to rely on assumptions about object sizes to infer a potential distance. The fact that the observed objects being of constant and familiar size and people likely having inaccurate and noisy beliefs can partially explain deviations in distance judgments and estimations. Size beliefs were inferred using different estimation techniques and the identified biases agreed across both techniques and were largely consistent with behavior in both distance tasks. Overall, we are showing that deviations in tasks about distance perception can be explained to a certain extent with consistent biases in human prior beliefs. Thus, we are providing evidence for human near-optimal behavior given constraints and the adequacy of probabilistic models with individual size prior for distance perception in two dimensions. In a second experiment we extended the experimental paradigm to test human prior beliefs and internal models under conditions of varying feedback with a continuous action control task. We investigated people’s belief about the non-linear dynamics of sliding objects on a surface under the effect of friction with and without visual feedback as well as their ability to transfer relevant information about mass, gained by watching collisions, to this continuous action control task. Comparison of models based on either a linear approximation or on the actual relationship described by Newtonian physics revealed that people’s behavior could indeed be best described by the model prescribed by Newtonian physics, especially while feedback was available. However, even without ever having seen the object’s trajectory in the feedback deprived phase people were able to accurately transfer their gained knowledge and perform extraordinary well. Not only the high Bayes factors favoring the noisy Newton model and the fact that it describes behavior well, but also the fact, that only the sheer existence of an appropriate internal model for both, sliding with and collisions without friction, can explain people correctly transferring the information to the action control task, thus strongly support the near-optimal probabilistic view on people’s behavior. In summary, the results of the second experiment further highlight the superiority of probabilistic models with resource and physiological constraints over heuristics as fixed rules in explaining human behavior and apparent deviations from optimal responses. Subsequently, we present an algorithm for the evaluation of individual cost functions to unravel an additional cause for human deviations from optimal behavior. So far only the puck sliding model considered subjective cost functions. There, the three common cost functions 0-1, hinge and squared loss were tested for by implicitly implementing different shifts of the action distribution. But here, we allowed individual parameterization of cost functions and the inclusion of effort specific costs, scaling with the magnitude of the action itself. Since action selection is finally shaped by cost functions considering these on an individual basis can be crucial to explain behavior. Using generated data we demonstrate the algorithm’s capability to recover these parameters and to predict the varying influence of perceptual uncertainty and action variability on responses in production and reproduction tasks. When used on data of human behavior in diverse continuous action control tasks we were able to explain pervasively observed undershoots as interaction of asymmetric cost functions and action variability as well as identifying similarities between specific tasks. Thereby, we provide further evidence in favor of explanations for human behavior in terms of probabilistic model of decision making. Finally, we transferred the puck sliding experiment to a VR setup enabling a naturalistic interaction with the task. Here, the assumption was that holding an actual physical puck and being able to accelerate it with a natural arm movement should facilitate the recruitment of an appropriate internal model, which is in accordance with the literature on embodied cognition. We compared data from this naturalistic task design with the previously conducted experiment on a keyboard and found that indeed individuals’ behavior was significantly better described by a noisy Newton model than the next best linear approximation. This was particularly interesting since participants did not receive any feedback about the objects’ trajectories and final positions. Thus the internal models governing the responses had to be a priori learned and accurately reflect the non-linearity of the environmental dynamics. These results eventually demonstrate the relevance of naturalistic interactions to investigate human behavior and again the capability of probabilistic models to describe it. In summary, we present several experimental designs, probabilistic models and algorithms in order to investigate people’s internal beliefs about functional relationships and dynamics of their environment. By running these experiments in controlled setups on screens and in VR we were able to constrain available information and to identify relevant features supporting people in the recruitment of appropriate internal models. Our results emphasize: first, that naturalistic interaction facilitates the recruitment of realistic models in accordance with both the idea of near-optimal resource constrained models and embodied cognition. Second, people’s behavior can be biased but lawfully consistent and thus pointing out the importance and generality of prior beliefs in modeling. And third, that individual cost functions incorporating an effort related term can help to quantify and explain suboptimal behavior. These results help to disentangle the mechanism behind the transition between deficient reasoning and accurate routine behavior in humans. Future research will uncover how the brain can achieve this level of performance, represent the enormous abundance of information and interlink domains of knowledge.