# Reliability of Perceptual-Cognitive Skills in a Complex, Laboratory-Based Team-Sport Setting
## 4 Discussion
### Intra- and Inter-Session Reliability  
Reliable measurements constitute a basic prerequisite for reproducible correlational studies, cross-sectional group comparisons and longitudinal studies (within or between groups) [47; 48]. Surprisingly though, to the best of our knowledge, no study hitherto systematically investigated the reliability of multi-categorical performance metrics in TO research. Cross-sectional studies revealed some evidence for reliability in the TO paradigm in team sports. Internal consistency analysis of a computer-based TO test in volleyball, where participants had to distinguish between a smash and lob, found acceptable reliability (\(r\) = 0.72) for video pair responses on the interval scale level [14]. When novices (no experience in competitive volleyball) and skilled volleyball players were separately analyzed, coefficients decreased to 0.66 and 0.55, respectively. With respect to intra-session reliability, we generally found moderate-to-substantial response consistency (right and left-handed) in all occlusion time points evaluated. Therefore, besides one exception (pass), the reliability estimates reported here were comparable to those of [PERSON] et al. [14]. A closer look at our data revealed attack-specific differences in terms of reliability, which emphasizes the specificity of varying open-game situations. For example, in breakthrough, the late occluded videos (t\({}_{3}\); t\({}_{2}\); t\({}_{1}\)) revealed relatively high levels of agreement. In these occlusions, the full action intentions of the attacker seemed to be terminated due to a highly dynamic run-up (t\({}_{0}\) till t\({}_{1}\)) and a deceptive movement at t\({}_{3}\), which is why most participants just reacted subconsciously in the following occlusions (t\({}_{4}\) till t\({}_{6}\)) with most likely identical decisions. That may explain the visible rise in the level of agreement in breakthrough. A further example for the necessity of varying game-situations is given by the different distinctions of reliability between right and left-handed jump throw attacks (inter-session). Higher left-hander reliability in later occlusion points (t\({}_{4}\) - t\({}_{0}\)) could be traced back to different defensive behavior based on greater uncertainty in how to defend against left-handed players. In fact, left-handed players in team handball are less frequently represented in team handball [33], which leads to divergent levels of agreement. Similarly to within-session reliability, there are only a few longitudinal studies that report between-session reliability. Without a TO approach, a related study of [PERSON] and [PERSON] [49] assessed long-term reliability in the context of option-generating research in team handball. Over a 2-year measurement period, their experimental setup contained full video clip presentations of competition-like attack situations, from the perspective of an attacker coming onto the defense line. After the end of each video clip (frozen video frame), participants were instructed to verbally report options for the player in possession of the ball. Reliability estimations for decision quality within four measurement points were calculated using the split-half test. Spearman-Brown coefficients for quality of the first option ranged from 0.49 in test wave one to 0.89 in test wave two. The variability of response reliability in their analyses can be compared to the distinctions among our inter-session kappa values, ranging between fair and substantial. Recommendations by the authors about further longitudinal studies in heuristic settings in sports were given as well. Other investigations with the TO paradigm, executed in cricket, tennis and softball, found overall high reliability (\(r\) = 0.74-0.99) for decision and response accuracy [20; 21; 22].