Investigation of Coefficient of Individual Agreement in Terms of Sample Size, Random and Monotone Missing Ratio, and Number of Repeated Measures.

(ProQuest: . denotes formulae omitted.)Science and technology are rapidly advancing, and it is a major priority of most countries to keep up with these developments. Advanced technologies and microelectronics have developed in the last quarter of the 20th century and have found applications in all areas-this is believed to constitute the third stage of the Industrial Revolution. This stage has allowed immense progress in technological development and has made it possible to move forward from an industrial society to a knowledge society. In a knowledge society, information technologies are at the center of production and the economy. Advanced information technologies allowing easy access to all parts of the world have facilitated knowledge attainment, trading knowledge and passing it from one to another. This stage, in which stupendous progress has been possible, has been termed the Knowledge-Information Age. Rapid spread of inventions such as computers and the Internet in this era has expedited scientific and technological advances and facilitated significant industrialization (Erdogan, 2011). Countries' development can now be measured with the level of generated and transferred knowledge, and societies that can train individuals with high self-esteem and who can research and question are at the top of this league table. Training individuals with these qualifications is only possible with well-planned, high quality and sustainable educational initiatives. Such an education requires continuous and long-term assessment of individuals and includes the identification and removal of deficiencies in education. This is feasible through studies based on longitudinal data.Studies based on longitudinal data in general focus on the change and development in the situation that is being investigated and allow the examination of issues such as education, individual development, cultural change and socioeconomic development in time (Rajulton, 2001). In other words, data related to longitudinal studies are obtained from the same variable at different time frames and are intended to measure the development of the variable in time.Assessment instruments used for this purpose may include paper and pencil and psychokinetic tests for basic competences as well as instruments that will assess affective competencies. Taking repeated measures or obtaining scores from multiple raters may be necessary to ensure reliability of the measurements obtained through these instruments.However, reliability is observed as one of the weakest links in essays, verbal and kinetic examinations that involve multiple raters for gathering longitudinal data. Therefore, it is crucial to ensure inter-rater or intra-rater harmony in these studies for the reliability of scoring (Cohen, 1990; Guler & Gelbal, 2010; Lin, Hedayet, & Wu, 2012). In other words, when inter-rater reliability, measurement tools, and longitudinal studies are considered together, error-free measurement of the change and development observed in time will be seen as the function of assessment tools and inter-rater reliability.In this context, Haber, Gao, and Barnhart (2007) proposed that disagreement between data obtained from the same individuals towards the same variable with different methods is similar to disagreement between repeated measures data obtained from same individuals towards the same variable with different methods, and thus developed the coefficient of individual agreement (CIA), a function of disagreement in which one of the raters acts as a reference to measure agreement or disagreement between methods.The agreement calculations between raters differ according to the measurement level of measuring device (nominal, ordinal, interval etc.) and the number of raters (Carletta, 1996; Cohen, 1960). The most basic agreement coefficient is Cohen's kappa coefficient- an agreement coefficient in a measurement tool measured at the classification level of two raters. …