Talk:Fleiss' kappa

message: 1	2	3	4	5	p_i
1				6		1.000
2		3			3	0.400
3		1	4		1	0.400
4					6	1.000
5		3		3		0.400
6	2		4			0.467
7			4		2	0.467
8	2		3	1		0.267
9	2			4		0.467
10					6	1.000
11	1			5		0.667
12	1	1		4		0.400
13		3	3			0.400
14	1			5		0.667
15		2		3	1	0.267
16			5		1	0.667
17	3			1	2	0.267
18	5	1				0.667
19		2		4		0.467
20	1		2		3	0.267
21					6	1.000
22		1		5		0.667
23		2		1	3	0.267
24	2			4		0.467
25	1			4	1	0.400
26		5		1		0.667
27	4				2	0.467
28		2		4		0.467
29	1		5			0.667
30					6	1.000
total	26	26	30	55	43	
p_j	0.144	0.144	0.167	0.306	0.239	
- FrancisTyers · 20:18, 17 July 2006 (UTC)
thread_title: Original example from Fleiss (1971)

message: *B34 = 0.144, C34 = 0.144, D34 = 0.167, E34 = 0.306, F34 = 0.239
*K3 = 6, K4 = 30, G37 = 16.6730
*G40 = 0.558, G41 = 0.2201
* Pbar =sum(1/(K4*K3*(K3-1))*(G37*K3*(K3-1)))
* Pbar_e =sum(B34^2+C34^2+D34^2+E34^2+F34^2)
* k =sum((G40-G41)/(1-G41))
* k ~ .430
- FrancisTyers · 21:15, 25 July 2006 (UTC)
thread_title: Spreadsheet

message: Correction:  There is an error in the semifinal equation.  I get the sum for the Pbar_e to come out to .213.  kappa then comes out to .2099.
Without rounding (using php) the value is 0.20993070442196  — Preceding unsigned comment added by 71.162.5.63 (talk) 18:01, 22 October 2013 (UTC)
replies: text: Thanks! I'll take care of these. - FrancisTyers · 23:05, 17 July 2006 (UTC)
replies: text: Following up last point I found the first sentance perplexing as I did not instantly know what inter-rater reliability meant. I had to go further down to the example section to get a feal for whats its about. A line such as if two or more people assign numercical ratings or scores to a number of items then the Fleiss' kappa will give a measure for how consistant the ratings are might help explain the subject better. --Salix alba (talk) 00:14, 18 July 2006 (UTC)
replies: text: Ok, I'll add something to that effect, thanks :) - FrancisTyers · 10:24, 18 July 2006 (UTC)

text: Thanks for the changes.  It is somewhat more clear now.  I substantially rewrote the Equations sections to make it flow.  There are still a few things I haven't had time to do in the worked example.
* I still haven't checked it for accuracy.
* The numbers for \bar{P} seem wrong.  Use the simpler formula from the equations section above and simply plug in the numbers.  Also, write out the sum of the P_i since (1) it is easy to miss this number below the table and (2) the worked example should follow the formula given in the section above.
* Make sure that rounding errors aren't introduced.  It looks as if there is some potential for this.  Also, it is technically more appropriate to use \approx instead of = if you are using rounded figures.
That's all for now.  Thanks.  VectorPosse 06:06, 18 July 2006 (UTC)
replies: text: You're right, the numbers for \bar{P} do seem wrong. I'm going to check my calculations again. Thanks for spotting that :) - FrancisTyers · 10:35, 18 July 2006 (UTC)
replies: text: Ok, I've got to go out, I'm going to add the  tag until I get back and work this out. - FrancisTyers · 13:11, 18 July 2006 (UTC)

text: Ok, looks like I fixed it. - FrancisTyers · 21:29, 25 July 2006 (UTC)

text: Thanks, you're right. I must have changed the values after making the totals without changing the totals. Thanks for spotting that! - FrancisTyers · 21:36, 28 July 2006 (UTC)
thread_title:  Potential Corrections 

message: I'm nominating this again, as most if not all the previous complaints were dealt with, and the article has been static for a while. - Francis Tyers · 20:51, 7 April 2008 (UTC)
thread_title: Second nomination

message: What about unifying this with http://en.wikipedia.org/wiki/Cohen%27s_kappa ? It seems to be the same quantity...  —Preceding unsigned comment added by 72.70.76.11 (talk) 23:13, 18 January 2010 (UTC)
thread_title: Cohen's kappa

message: What about correcting the English? Fleiss is the last name of the man. Therefore, the possessive is "Fleiss's" not "Fleiss'" (Fleiss is not a plural noun.) — Preceding unsigned comment added by Parhelia (talk • contribs) 18:01, 18 December 2012 (UTC)
replies: text: Per Wikipedia:Article titles#Common names, we generally use "the most common name for a subject based on as determined by its prevalence in reliable English-language sources". Google Books has 389 hits for "Fleiss' kappa" and 33 for "Fleiss's kappa"; on Google Scholar the results are 1,490 and 75. — Malik Shabazz Talk/Stalk 05:20, 19 December 2012 (UTC)
thread_title: What about correcting the English?

message: The article mentions that Fleiss' kappa is likely to be lower when more categories are introduced. In this case, is there any evidence for normalising the kappa based on number of categories, almost as if calculating an arithmetic mean? Surely this would reduce disparity between values computed for different numbers of categories. — Sasuke Sarutobi (talk) 10:52, 24 March 2014 (UTC)
thread_title:  Normalised versions?