Table 6: Relation between expression of characteristics and scale levels of data for the assessment of distinctness and uniformity Example Name of characteristic Distinctness Uniformity Unit of assessment Description (states of expression) Type of Unit of assessment Description (states of expression) Type of 1 Sex of plant 1 2 3 4 dioecious female dioecious male monoecious unisexual monoecious hermaphrodite nominal discrete True-type Off-type Number of plants belonging to the variety Number of nominal discrete 2 Leaf blade: variegation 1 9 absent present nominal discrete True-type Off-type Number of plants belonging to the variety Number of off-types nominal discrete 3 Length of plant cm assessment in cm without digits after decimal point ratio conti- nuous cm assessment in cm without digits decimal point ratio continuous True-type Off-type Number of plants belonging to the variety Number of off-types nominal discrete 4 Number of stamens counts 1, 2, 3, ... , 40, 41, ... ratio discrete counts 1, 2, 3, ... , 40, 41, ... ratio discrete 5 Time of beginning of flowering Date e.g. May 21, 51st day from April 1 interval discrete Date e.g. May 21, 51st day from 1 interval discrete True-type Off-type Number of plants belonging to the variety Number of off-types nominal discrete 6 Intensity of anthocyanin 1 2 3 4 5 6 7 8 9 very low very low to low low low to medium medium medium to high high high to very high very high ordinal discrete (with an underlying variable) True-type Off-type Number of plants belonging to the variety Number of off-types nominal discrete trial trial
7 Shape 1 2 3 4 5 6 7 deltate ovate elliptic obovate obdeltate circular oblate nominal discrete True-type Off-type Number of plants belonging to the variety Number of off-types nominal discrete 8 Flower color 1 2 3 4 5 6 7 8 9 10 dark red medium red red light blue medium blue dark blue red violet violet blue violet nominal discrete True-type Off-type Number of plants belonging to the variety Number of off-types nominal discrete
κ(O1,O2) = (P(agreement between O1 and O2) – P(e)) / (1 – P(e)) where: P(agreement) = (3+5+0+1+0+2)/30 = 11/30 ≈ 0.3667 (diagonal elements) P(e) = (3/30).(15/30) + (17/30).(6/30) + (3/30).(1/30) + (1/30).(3/30) + (3/30).(0/30) + (3/30).(5/30) ≈ 0.1867. (pair-wise margins) So κ(O1,O2) ≈ (0.3667-0.1867) / (1-0.1867) ≈ 0.22 This is a low value, indicating very poor agreement between these two observers. There is reason for concern and action should be taken to improve the agreement. Similarly the values for the other pairs can be calculated: κ(O1,O3) ≈ 0.72, κ(O2,O3) ≈ 0.22.Observer 1 and 3 are in good agreement. Observer 2 is clearly different from 1 and 3 and the reasons for the deviation requires further investigation (e.g. consider need for additional training). 3.8 References Cohen, J. (1960) A coefficient of agreement for nominal scales. Educational and Psychological Measurement 20: 37-46. Cohen, J. (1968) Weighted kappa: Nominal scale agreement provision for scaled disagreement or partial credit. Psychological Bulletin, 70(4): 213-220. Bland, J. M. Altman D. G. (1986) Statistical methods for assessing agreement between two methods of clinical measurement, Lancet: 307–310. (ISO 5725-2 based software)