DOI: 10.5061/dryad.v9s4mw6sp
title: Unobtrusive tracking of interpersonal orienting and distance predicts the subjective quality of social interactions

description: Interpersonal coordination of behavior is essential for smooth social
 interactions. Measures of interpersonal behavior, however, often rely on
 subjective evaluations, invasive measurement techniques or gross measures
 of motion. Here, we constructed an unobtrusive motion tracking system that
 enables detailed analysis of behavior at the individual and interpersonal
 levels, which we validated using wearable sensors. We evaluate dyadic
 measures of joint orienting and distancing, synchrony and gaze behaviors
 to summarize data collected during natural conversation and joint action
 tasks. Our results demonstrate that patterns of proxemic behaviors, rather
 than more widely used measures of interpersonal synchrony, best predicted
 the subjective quality of the interactions. Increased distance between
 participants predicted lower enjoyment, while increased joint orienting
 toward each other during cooperation correlated with increased effort
 reported by the participants. Importantly, the interpersonal distance was
 most informative of the quality of interaction when task demands and
 experimental control were minimal. These results suggest that
 interpersonal measures of behavior gathered during minimally constrained
 social interactions are particularly sensitive for the subjective quality
 of social interactions and may be useful for interaction-based phenotyping
 for further studies.
description: This data was collected using Kinect V2 sensors and
 Tinkerforge IMU Brick 2.0 inertial measurement units, and questionnaires.
 Gaze behaviors are manually annotated from video images saved by the
 Kinect sensors. Anonymous data derived from the Kinect and IMU tracking
 results is shared in preprocessed format due to limitations in the ethics
 approval for the study. Preprocessing steps applied to these intermediate
 data are included in the provided analysis script. Video images are not
 included in this dataset to preserve the privacy of our participants and
 to follow the requirements of the ethics approval.
description: # InteractionTracking __Code and motion
 tracking results for producing results in:__
 __Lahnakoski JM, Forbes PAG, McCall C, Schilbach L -__<br>
 _Unobtrusive tracking of interpersonal orienting and distance predicts the
 subjective quality of social interactions_ Note:
 Because of randomization, the exact statistics produced by the analyses
 may change slightly on repeated runs of the script. (C)
 2019, 2020 Juha Lahnakoski, juha.lahnakoski@gmail.com
 For license information, see LICENSE file.
 Directory structure:<br>
 ------------------------------------------------------------------- _Code_\<br> Matlab code for loading data and performing analyses and visual studio project for the Kinect motion tracking application. _PreprocessedData_\<br> Data containing manual gaze ratings and Kinect data from the corresponding timepoints and other preprocessed data either created by the analysis script or presaved when raw data cannot be shared. _matDataKinectAndIMU_\<br> Basic tracking results converted to Matlab format for reproducing the final results. _QuestionnaireData_\<br> Overall scores of questionnaires (responses to Intrinsic Motivation Inventory are included directly in the analysis script). _SupportingFunctions_\<br> Functions used for reading data, estimating thresholds, plotting, etc. _Include_\<br> Any additional files needed for creating the output. Currently contains only the colormap legend image. <br> Code description<br> ------------------------------------------------------------------- __Motion tracking software__ _Kinect_ADOS_tracker_with_skeleton_Release.zip_\<br> Visual studio project including a compiled binary for the motion tracking system (/Release folder). Requires Windows 8/10, Kinect 2.0 with Kinect for Windows adapter, associated software and compatible hardware. _SensorReader.zip_\<br> Visual studio project for recording data from the inertial measurement units. Compiled binary is not included because the sensors have unique IDs that are hard coded and need to be changed for each new unit individually. __Analysis scripts__ _Lahnakoski_et_al_paper_analyses.m_\<br> Main code for loading and analysing data. In the beginning of the script, you must set the base directory of these files so that Matlab can find the code and the tracking results. __Video annotation scripts__ These are provided for reference only. Due to privacy concerns, the video frames used by these scripts are not available outside of the research site; contact authors in case reanalysis of the video frames is required. _zitternix_fix_trial_time_windows.m_\<br> Used for verifying that the timepoints marked by the experimenter<br> during the measurement correspond to actual times when trials<br> start (due to duplicate/missing button presses). _zitternix_annotate_gaze_from_video_frames.m_\<br> Script for manual annotation of gaze directions used by the first<br> rater. Randomly select timepoints from all conditions in all dyads,<br> and save Kinect data as well as subjective annotation of whether<br> each individual is looking at the partner, looking at a target or<br> neither. _zitternix_annotate_gaze_reliability.m_\<br> Script for manual annotation of gaze directions used by the second<br> rater. Loads the original rater's data to extract the same (randomly<br> sampled) timepoints _zitternix_annotate_gaze_reliability_third.m_\<br> Script for third "tiebreaker" participant for annotating gaze during<br> timepoints when initial two raters disagreed. __Supporting functions__ _aaltoColors.m_\<br> Color palette (can be replaced by other colormaps). _correlationToProbability.m_\<br> Simple function for transforming correlation values to p-values assuming<br> t-distribution (similar to the corr-funtion in Matlab). _gaussKernel.m_\<br> Function for producing a Gaussian smoothing kernel. _nanconv.m_\<br> Convolution for data with NaNs, from Matlab Central:<br> Copyright (c) 2013, Benjamin Kraus, All rights reserved. _patchline.m_\<br> Function to visualize lines as patches.<br> Brett Shoelson (2020). Patchline<br> (https://www.mathworks.com/matlabcentral/fileexchange/36953-patchline),<br> MATLAB Central File Exchange. Retrieved January 30, 2020. _probabilityToCorrelation.m_\<br> Simple function for transforming p-values to correlation values assuming<br> t-distribution (similar to the p-values returned by corr-funtion in Matlab). _readADOSheader.m_\<br> Reads headers of Kinect result files saved by the<br> tracking system. _readADOSlog.m_\<br> Reads the Kinect tracking result files saved by the<br> tracking system. _read_zitternix_sensorlog.m_\<br> Reads in IMU values in the format saved by the tracking system. _rigidBodyRegister.m_\<br> Function for registering two point clouds.<br> (Points are assumed to be in the same order in the two clouds). _rm_anova2multi.m_\<br> Massively univariate 2-way repeated measures ANOVA<br> Based on rm_anova.m by Aaron Schurger (2005.02.04)<br> -Derived from Keppel & Wickens (2004) "Design and Analysis" ch. 18<br> Edited by Juha Lahnakoski (Jan 10th 2020)<br> -Updated to handle multiple variables at once and save results to a<br> struct rather than a cell table. _windowedCrossCorrelation.m_\<br> Function for producing windowed cross correlation matrices (and optionally,<br> peak timecourses. _zitternix_convert_kinect_IMU_to_Matlab.m_\<br> Creates the motion tracking results from the raw data files.<br> This is included for reference only; raw data logs cannot be<br> shared due to privacy concerns. __Required toolbox licenses for Matlab__ curve_fitting_toolbox\<br> robotics_system_toolbox\<br> signal_toolbox\<br> statistics_toolbox ------------------------------------------------------------------- July 18th, 2019 \<br> Last revision, July 10th, 2020 Juha Lahnakoski, Max Planck Institute of Psychiatry, FZ Juelich \<br> juha_lahnakoski@psych.mpg.de, juha.lahnakoski@gmail.com \<br> https://orcid.org/0000-0002-5223-7822 -------------------------------------------------------------------