# On Cronbach’s Alpha as the Mean of All Possiblek-Split Alphas

Authors: Matthijs J. Warrens
Journal: Advances in Statistics
 (2014)

---

## Abstract

Coefficient alpha is the most commonly used internal consistency reliability coefficient. Alpha is the mean of all possiblek-split alphas if the items are divided into k parts of equal size. This result gives proper interpretations of alpha: interpretations that also hold if (some of) its assumptions are not valid. Here we consider the cases where the items cannot be split into parts of equal size. It is shown that if a k-split is made such that the items are divided as evenly as possible, the difference between alpha and the mean of all possible k-split alphas can be made arbitrarily small by increasing the number of items.


---

## Body

## 1. Introduction

In test theory, test scores are used to summarize the performance of participants on a test. An important concept of a test score is its reliability, which concerns the precision of the administration of a participant’s score. Reliability can be conceptualized in different ways. In layman’s terms, a test score is said to be reliable if it produces similar outcomes for participants under similar administration conditions. A more formal definition of reliability comes from classical test theory; namely, reliability is defined as the ratio of the true score variance and the total score variance [1–3]. The true score variance cannot be observed directly. Therefore, it has to be estimated from the data. If there is only one test administration reliability can be estimated using the so-called internal consistency reliability coefficients [4]. The internal consistency coefficient that is most commonly used in psychology and other behavioral sciences is coefficient alpha [5–10].Coefficient alpha was already discussed in Guttman [11]. The coefficient was later popularized by Cronbach [6]. Since then it has been applied in thousands of research studies [5, 9]. It has been pointed out that alpha is not a measure of the one-dimensionality of a test score [6, 9, 12]. Furthermore, a problem is that the value of alpha is affected by the length of the test. Alpha is often high if the test consists of 15 items or more [5, 13, 14]. Moreover, the derivation of alpha is based on several assumptions from classical test theory. For example, alpha is an estimate of the reliability if the items of the test are essentially tau-equivalent [1, 2]. These assumptions may not hold in practice. Various authors have therefore studied alpha’s robustness to violations of its assumptions [12, 15–17].Other authors have presented results that give a meaning to alpha and to its estimate from a sample in case its assumptions do not hold [6, 18]. The items of a test can be split into two parts. This will be called a 2-split. For each part, the sum score of the items can be determined. If we apply Cronbach’s alpha to the two sum scores, we obtain the Flanagan-Rulon split-half estimate of the reliability [19, 20]. The split-half reliability is an alternative approach of estimating the reliability of a test when there is only one administration [3, 7, 8]. The problem with a 2-split of a test is that there are multiple ways to divide the items into two parts. The value of the 2-split alpha therefore depends on the way the 2-splits are made [7]. Cronbach [6] showed that if the 2-splits are made such that the parts have equal size, then the overall alpha is the mean of all possible 2-split alphas. Cronbach’s result was generalized by Raju [18] to other types of splits. Let k≥2 be an integer. Raju [18] showed that if the k-split is such that the k parts have the same number of items, then alpha is the mean of all possible k-split alphas.The algebraic results by Cronbach [6] and Raju [18] are important since they provide a proper interpretation of alpha and of its estimate from a sample in the case that its assumptions do not hold [2, 5]. Thus, if the number of items is a composite number n, then there exists an interpretation of alpha for any proper divisor of n. Raju [18] also showed that if the k parts do not have equal size, then the mean of all k-split alphas is strictly smaller than the overall alpha. Moreover, this difference can be substantial [18]. Thus, at present, it is unknown if there is an (approximate) interpretation of alpha as a mean of k-split alphas in case the number of items is a prime number. This is the topic of investigation of this paper.The paper is organized as follows. Notation and Definitions are presented in Section2. We also present a formula of the mean of all possible k-split alphas in Section 2. In Section 3, the nonnegative difference between alpha and the mean of all k-split alphas is studied. It is shown that if we divide the items as evenly as possible, the difference is strictly decreasing in the number of items. Furthermore, the difference goes to zero if the number of items increases. Hence, the difference can be made arbitrarily small by increasing the number of items. In Section 4, we determine for three criteria and for small values of k how many items are needed. Finally, Section 5 contains a conclusion.


## 2. Notation and Definitions

Suppose we have a test that consists ofn≥2 items. Let σii′ for 1≤i, i′≤n denote the covariance between items i and i′, and let σX2 denote the variance of the total score. Guttman’s lambda3 or Cronbach’s alpha is defined as
(1)α≔nn-1·∑i≠i′σii′σX2.
In the summation ∑i≠i′‍σii′ both counters i and i′ run from 1 to n, but they are never equal.Next, suppose we split the test intok parts of sizes n1,n2,…,nk with 1≤nj<n for 1≤j≤k and
(2)∑j=1k‍nj=n.
Let pj=nj/n for 1≤j≤k denote the proportion of items in part j. Furthermore, let σjj′ denote the covariance between the sum scores of parts j and j′. For a k-split coefficient alpha is defined as [18]
(3)αk≔kk-1·∑j≠j′σjj′σX2.
We have αn=α. Furthermore, for fixed k and n1,n2,…,nk, the number of different k-splits is given by [21, page 823]
(4)S=n!n1!n2!⋯nk!.
For each k-split, there is an associated αk. The mean of all k-split alphas is denoted by E(αk). An expression for E(αk) is presented in Lemma 1. The result is not derived in Raju [18] and is believed to be new.Lemma 1.
It holds that(5)E(αk)=nn-1·kk-1·(∑j≠j′pjpj′)(∑i≠i′σii′)σX2.Proof.
The variance of the total scoreσX2 is not affected by the k-split. Furthermore, k/(k-1) is fixed since k is fixed. Hence,
(6)E(αk)=kk-1·E(∑j≠j′σjj′)σX2,
where, using (4),
(7)E(∑j≠j′‍σjj′)=1S∑l=1S‍(∑j≠j′‍σjj′).
We want to determine how often each covariance σii′ between items i and i′ occurs in the summation on the right-hand side of (7). Raju [18, page 559] showed that this amount is given by
(8)2S·nn-1·∑j≠j′‍pjpj′.
Since each covariance σii′ occurs twice in the summation ∑i≠i′‍σii′, (7) is given by
(9)E(∑j≠j′‍σjj′)=nn-1(∑j≠j′‍pjpj′)(∑i≠i′‍σii′).
Using (9) in (6), we obtain (5).


## 3. Difference betweenα and E(αk)

In this section, we study the difference between alpha and the mean of allk-split alphas. Using (1), we can write (5) as
(10)E(αk)=α·kk-1∑j≠j′‍pjpj′.
Cronbach [6] showed that α=E(α2) if p1=p2. A proof can also be found in Lord and Novick [1]. Raju [18, page 560] showed that α=E(αk) if p1=p2=⋯=pk and that α>E(αk) if p1,p2,…,pk are not all equal. Hence, the difference
(11)α-E(αk)=α(1-kk-1∑j≠j′‍pjpj′)
is nonnegative. Difference (11) can be used to study how close the values of alpha and the mean of all k-split alphas are. Since 0≤α≤1, we have the inequality
(12)α-E(αk)≤1-kk-1∑j≠j′‍pjpj′.
The right-hand side of (12) is an upper bound of α-E(αk) for all values of α (0≤α≤1). It only depends on k and the proportions of items in the parts.It turns out that difference (11) can be quite large [18]. This is the case when the variance between p1,p2,…,pk is large, that is, if p1,p2,…,pk are very different. To illustrate this property of difference (11), suppose we have m+k items, where m≥3 is an integer. Furthermore, suppose we distribute the items such that we have k-1 parts with 1 item and 1 part with m+1 items. The proportions of items per part are given by
(13)p1=m+1m+k,p2=⋯=pk=1m+k.
Using the proportions in (13) we have
(14)∑i≠j‍pipj=(k-1)(k-2)(m+k)2+2(k-1)(m+1)(m+k)2.
Due to the m2-term in the denominator of the right-hand side of (14) we have, for fixed k<m, that ∑i≠j‍pipj→0 as m→∞.In this paper, we are not interested in an arbitraryk-split of the items but in a k-split in which the items are distributed as evenly as possible over the parts. Suppose we have km+c items where m, c≥1 are integers with 0<c<k. If we want to distribute the items as evenly as possible over the k parts, an optimal k-split is to have k-c parts with m items and c parts with m+1 items. In this case the proportions of items per part are given by
(15)p1=⋯=pc=m+1km+c,pc+1=⋯=pk=mkm+c.
Lemma 2 shows that for a split of the type in (15) the quantity ∑j≠j′‍pjpj′ approaches the ratio (k-1)/k as the counter m goes to infinity.Lemma 2.
It holds that(16)∑i≠j‍pipj⟶k-1kasm→∞.Proof.
Using the proportions in (15), we have the identity
(17)∑i≠j‍pipj=c(c-1)(m+1)2(km+c)2+2c(k-c)m(m+1)(km+c)2+(k-c-1)(k-c)m2(km+c)2.
Using the identity
(18)c(c-1)+2c(k-c)+(k-c-1)(k-c)=k(k-1),
we can write (17) as
(19)∑i≠j‍pipj=c(c-1)(2m+1)+2c(k-c)m+k(k-1)m2(km+c)2=c(c-1)+2c(k-1)m+k(k-1)m2(km+c)2=c(c-1)+(k-1)(2cm+km2)(km+c)2.
For m→∞, we have
(20)∑i≠j‍pipj⟶k(k-1)k2=k-1k.It follows from Lemma2 that the right-hand-side of (12), and thus the difference α-E(αk), goes to zero as m increases. In other words, if the k-split is of the type in (15), the difference between alpha and the mean of the best k-splits vanishes as the number of items becomes large. It is not clear from Lemma 2 that ∑j≠j′‍pjpj′ is strictly increasing in m. This is shown in Lemma 3.Lemma 3.
For split (15) ∑j≠j′‍pjpj′ is strictly increasing in m.Proof.
Using (19), the first order partial derivative is
(21)∂∑i≠jpipj∂m=2(k-1)(km+c)2-[c(c-1)+(k-1)(2cm+km2)]2k(km+c)3=2c(k-c)(km+c)3.
Since k-c>0, (21) is strictly positive, and it follows that ∑j≠j′‍pjpj′ is strictly increasing in m.


## 4. Numerical Results

Lemmas2 and 3 from the previous section show that the nonnegative difference α-E(αk) decreases strictly to zero as m increases, if the k-split is of the type in (15). Hence, for sufficiently large m, the difference α-E(αk) can be made smaller than the specified criteria. In this section, we determine the numbers of items that are needed so that the difference α-E(αk) is less than 0.010, 0.005, and 0.001. Since a difference of 0.010 is negligible for most practical purposes, we may say that α is approximately equal to E(αk) if the difference is less than 0.010, that is, if
(22)α-E(αk)≤0.010.
The values 0.005 and 0.001 can be used if one wants a stronger definition of approximately equal.If we are interested in ak-split but the number of items is not a multiple of k, there are k-1 different scenarios to consider. For example, if k=3 but the number of items is not a multiple of 3, we either have 3m+1 items or have 3m+2 items. Inequality (22) may hold for different m depending on whether we have 3m+1 items or 3m+2 items. We therefore consider the number of items of all k-1 variants of a k-split separately. For all possibilities of k and c with 2≤k≤7 and 1≤c<k, we applied the following numerical strategy. For fixed k and c, we increased m in steps of 1, starting at 1, until inequality (22) was satisfied. The results for k=2,3,…,7 are presented in Table 1. The table presents for each variant of a k-split the minimum number of items for which the difference α-E(αk) is ≤0.010 (column 3), ≤0.005 (column 4), and ≤0.001 (column 5).Table 1
The minimum number of items per split variant for three criteria.
Split type
Split variant
Minimum number of items
≤0.010
≤0.005
≤0.001
2-split
2
m
+
1
11
15
33
3-split
3
m
+
1
10
16
34
3
m
+
2
11
17
32
4-split
4
m
+
1
13
17
33
4
m
+
2
14
18
38
4
m
+
3
11
15
35
5-split
5
m
+
1
11
16
36
5
m
+
2
17
22
42
5
m
+
3
13
18
43
5
m
+
4
14
19
34
6-split
6
m
+
1
13
19
37
6
m
+
2
14
20
44
6
m
+
3
15
21
45
6
m
+
4
16
22
46
6
m
+
5
11
17
35
7-split
7
m
+
1
15
15
36
7
m
+
2
16
23
44
7
m
+
3
17
24
45
7
m
+
4
18
25
46
7
m
+
5
19
19
47
7
m
+
6
13
20
34Table2 summarizes for each k-split the minimum number of items required per criteria. For example, consider the case k=2. If the number of items is even we have a perfect 2-split and α=E(α2). If the number of items is odd and we distribute the items as evenly as possible, Table 1 shows that the difference α-E(α2) is no more than 0.010 if we have at least 11 items, no more than 0.005 if we have at least 15 items, and no more than 0.001 if we have at least 33 items. Hence, we may conclude that in the case that we distribute the items as evenly as possible the difference α-E(α2) is no more than 0.010 if we have at least 10 items, no more than 0.005 if we have at least 14 items, and no more than 0.001 if we have at least 32 items. These numbers are presented in Table 2.Table 2
The minimum number of items per split type for three criteria.
Split type
Minimum number of items
≤0.010
≤0.005
≤0.001
2-split
10
14
32
3-split
9
15
32
4-split
11
15
35
5-split
13
18
39
6-split
11
17
41
7-split
13
19
41As another example, consider the casek=3. If we have 3m items, we have a perfect 3-split and α=E(α3). If we have 3m+1 items, Table 1 shows that the difference α-E(α3) is no more than 0.010 if we have at least 10 items, no more than 0.005 if we have at least 16 items, and no more than 0.001 if we have at least 34 items. If we have 3m+2 items, Table 1 shows that the difference α-E(α3) is no more than 0.010 if we have at least 11 items, no more than 0.005 if we have at least 17 items, and no more than 0.001 if we have at least 32 items. Hence, we may conclude that in the case that we distribute the items as evenly as possible, the difference α-E(α3) is no more than 0.010 if we have at least 9 items, no more than 0.005 if we have at least 15 items, and no more than 0.001 if we have at least 32 items. These numbers are presented in Table 2.


## 5. Conclusion

Alpha is the most commonly used coefficient for estimating reliability of a test score if there is only one test administration [5–9]. Cronbach [6] and Raju [18] have presented results that give a meaning to alpha and to its estimate from a sample, even if its assumptions do not hold. Cronbach [6] showed that if the 2-splits are made such that the parts have equal size, then alpha is the mean of all possible 2-split alphas. Raju [18] showed that if a k-split for k≥2 is made such that the k parts have the same number of items, then alpha is the mean of all possible k-split alphas. The importance of these results lies in the fact that they provide a proper interpretation of alpha if (some of) its assumptions do not hold [2, 5].In this paper, we consider the cases where the items cannot be split into parts of equal size. In these cases, the value of alpha exceeds the mean of all possiblek-split alphas [18]. However, it was shown that if a k-split is made such that the items are divided as evenly as possible, the difference between alpha and the mean of all possible k-split alphas is negligible for sufficiently many items (Lemmas 2 and 3). Using numerical simulations, it was shown, in the case of a 2-split, that the difference is no more than 0.010 if we have at least 10 items, no more than 0.005 if we have at least 14 items, and no more than 0.001 if we have at least 32 items and, in the case of a 3-split, that the difference is no more than 0.010 if we have at least 9 items, no more than 0.005 if we have at least 15 items, and no more than 0.001 if we have at least 32 items.The results in Cronbach [6] and Raju [18] provide interpretations of alpha if the number of items is a composite number n, for any proper divisor of n. The results and simulations in this paper provide interpretations of alpha if the number of items is prime.



---

*Source: 742863-2014-09-30.xml*