Determination of Type I Error Rates and Power of Answer Copying Indices under Various Conditions

Cheating, a factor which threatens tests' psychometrics properties, is a serious problem within the processes of measurement and assessment. Examinees' test scores do not reflect their true scores in the presence of cheating. So, this situation results in giving wrong decisions about the cheaters in terms of the features to be tested. From another point of view, if cheating systematically increases the number of correct answers, this gets evaluated as a systematic error, and systematic errors jeopardize test validity. If cheating does not systematically increase the number of correct answers, it can be evaluated as a random error, and this type of error threatens reliability. As a result, cheating damages the psychometric properties of tests. Nowadays, research on cheating and copy detection methods have increased in order to deal with the problem of cheating. Cheating methods were categorized by Cizek (1999) into three main headings: giving/taking/receiving information, using forbidden materials, and circumventing or taking advantage of the testing process. Researches show that more than 50% of students have admitted to cheating on their exams, and the rate of cheating has increased rapidly (Cizek, 1999, p. 35). For this reason, researchers have developed various copy detection methods to detect examinees' cheating levels. One of these methods depends on statistical evidence.Saupe (1960) placed methods that depend on statistical evidence into two categories: empirical methods and chance methods. In empirical methods, the answer distribution of examinees suspected of cheating is compared with that of a group known not to have cheated. In methods that depend on chance probability, the answer distribution of examinees suspected of cheating is compared with a theoretical distribution, such as binomial, Poisson, or standard normal distribution. Over time, methods of detection have moved away from empirical approaches and toward chance methods (Cizek, 1999, p. 139). Numerous indices since 1927 have been developed for detecting cheating. Most of those indices are based upon classical test theory, while some others on item response theory. Most of these methods focus on the probable similarity of answers (chance probability) that independent examinees give without cheating or collaborating. Bird (1927; 1929), Crawford (1930), Anikeef (1954), and Saupe (1960) are pioneers of these indices.However, because calculating these indices is complicated, satisfactory results of the indices' sensitivity for detecting copying cannot be precisely obtained (Argenal, Co, Cruz, & Patungan, 2004). Angoff(1974) developed eight indices (A through H) that use the same procedure as the index developed by Saupe (1960) by using different parameters for identifying cheaters. Angoff(1974) showed parameters that conclude that the number of correct and wrong answers (B and H indices) were more successful at detecting cheating. A number of indices were developed after Angoff's, and numerous statistical methods have been used for detecting cheating. When reviewing the related literature, the following indices and methods for detecting cheating have been discovered: g1 and g2 indices (Frary, Tideman, & Watts, 1977, 1993), PAIR1 and PAIR2 indices (Hanson, Harris, & Brennan, 1987), ESA index (Bellezza & Bellezza, 1989), Bm index (Bay, 1994), Harpp-Hogan index (Harpp, Hogan, & Jennings, 1996), K index (Holland, 1996), ω index (Wollack, 1997), K? 1 and K? 2 indices (Sotaridona & Meijer, 2002), S1 and S2 indices (Sotaridona & Meijer, 2003), Differential Item Functioning (DIF; Giardano, 2005), kappa statistic (Sotaridona, van der Linden, & Meijer, 2006), response time (RT) models (van der Linden, 2006), generalized binomial test (GBT; van der Linden & Sotaridona, 2006), τ statistics (van der Ark, Emons, & Sijtsma, 2008), CTAD (Deng, 2008), cumulative sum statistic (CUSUMLR; Armstrong & Shi, 2009), factor analysis (Clark, 2010), Kullback-Leibler (K-L) divergence and K-index (Belov & Armstrong, 2010), Final Log Odds Ratio (FLOR; Hui, 2010), Variable Match Index (VM-index; Belov, 2011), and Deterministic, Gated Item Response Theory Model (DGIRTM; Shu, 2011). …