title: NEPS Technical Report for Reading: Scaling Results of Starting Cohort 4 for Grade 9 in Special Schools
Creators: name: Gnambs, Timo
Publisher: name: LIfBi Leibniz Institute for Educational Trajectories
Year: 2020
Subjects: subject: item response theory
subject: special educational needs
subject: reading competence
subject: scientific use file

description: The National Educational Panel Study (NEPS) investigates the development of competencies <br>across the life span and develops tests for the assessment of different competence domains. <br>In order to evaluate the quality of the competence tests, various analyses based on item <br>response theory (IRT) are performed. This paper describes the data and scaling procedures <br>for the reading competence test in Grade 9 of Starting Cohort 4 (ninth grade) that was <br>administered to students in special schools. The feasibility of including students with special <br>educational needs in the NEPS was investigated with five different test versions. Version 1 <br>was identical to the test administered in the main sample to students from general schools, <br>version 2 represented a shorter version of the test with fewer items, whereas in version 3 <br>the most difficult items were replaced with items designed for a younger age cohort, thus, <br>resulting in a considerably easier test as compared to version 1. Moreover, to examine <br>potential item order effects, versions 4 and 5 presented the test versions 2 and 3 in reversed <br>order. These five test versions were randomly distributed among a sample of N = 976 <br>students (44% girls) from special schools. As a control group, a matched sample of N = 500 <br>students (43% girls) attending lower secondary schools (“Hauptschule”) was drawn from the <br>main study. The responses of the two samples were scaled using the partial credit model. <br>Item fit statistics, differential item functioning, and Rasch-homogeneity were evaluated to <br>examine the quality of the tests. In particular, differential item functioning analyses between <br>the five test versions were conducted to evaluate whether a common reading score can be <br>estimated. These analyses showed that standard competence tests are too long for students <br>in special schools; items at the end of the administered tests were finished by rather few <br>students, resulting in large missing rates. Moreover, the tests exhibited somewhat limited <br>variances and reliabilities, thus, allowing only rather crude analyses of interindividual <br>differences between students with special educational needs. Nevertheless, a common <br>reading score was estimated for test versions 1, 2, and 3, which allow cross-sectional <br>analyses of students reading abilities. Importantly, there was substantial differential item <br>functioning between special schools and lower secondary schools. Therefore, comparative <br>analyses between the two school types using the administered reading competence test are <br>not recommended. Overall, these results highlight substantial difficulties in assessing reading <br>competence among students with special educational needs at special schools in educational <br>large-scale assessments. Besides the scaling results, this paper also describes the data <br>available in the scientific use file and presents the R syntax for scaling the data.
description: NEPS Survey Papers, No. 63 (2020)