# Understanding and Enhancement of Internal Clustering Validation Indexes for Categorical Data
## 3 Understanding of Internal Clustering Validation Indexes
### Analysis of Indexes \(CU_{1/k}\), Cloper, and \(R\)
: _If the conditions are the same as in Theorem 1, and we have the compactness measure as followed:_  
\[G(P)=n\cdot m\cdot\sum_{l=1}^{k}p(C_{l})\cdot\left[\sum_{j=1}^{m}\left|D(A_{j} \left|C_{l}\right.)\right.\right]^{-r}, \tag{34}\]  
_where \(r\) is a parameter greater than zero, then \(G(P_{1})\leq G(P_{2})\), and the equality holds if and only if \(D(A_{j}\left|\right.C_{k})=D(A_{j}\left|\right.C_{s1})=D(A_{j}\left|\right.C_{s2 })=\ldots=D(A_{j}\left|\right.C_{st})\), (\(j=1,\ldots,m\))._  
Proof of Theorem 4.: Similar to the proof of Theorem 1, we can show that:  
\[\frac{G(P_{1})-G(P_{2})}{n\cdot m}=\sum_{j=1}^{m}\Bigg{[}p(C_{k})\cdot\left|D( A_{j}\left|C_{k}\right.)\right.\left|^{-r}-\sum_{l=1}^{t}p(C_{st})\cdot\left|D(A_{j} \left|C_{sl}\right.)\right.\right|^{-r}\Bigg{]}, \tag{35}\]  
Since \(C_{s1}\cup C_{s2}\cup\ldots\cup C_{st}=C_{k}\), \(C_{sl}\
eq\Phi\), and \(C_{sl}\cap C_{sl^{\prime}}=\Phi\) (\(l\
eq l^{\prime}\); \(l\), \(l^{\prime}=1,\ldots,t\)), to any attribute \(A_{j}\):  
\[\left\{ \begin{array}{l}p(C_{k})=\sum_{l=1}^{t}p(C_{sl}),\\ \left|D(A_{j}\left|C_{k}\right.\right)\right.\left|^{-r}\leq\left.\left|D(A_{j} \left|C_{sl}\right.)\right.\right|^{-r},r>0,j=1,...m,l=1,...t.\end{array} \right. \tag{36}\]  
Therefore, the value of Equation (35) is not greater than zero, the equality holds if and only if \(\left|\right.D(A_{j}\left|\right.C_{k})\left|\right.=\left|\right.D(A_{j} \left|\right.C_{s1})\left|\right.=\left|\right.D(A_{j}\left|\right.C_{s2}) \left|\right.=\ldots=\left|\right.D(A_{j}\left|\right.C_{st})\left|\right.\), (\(j=1,\ldots,m\)), which is equal to \(D(A_{j}\left|\right.C_{k})=D(A_{j}\left|\right.C_{s1})=D(A_{j}\left|\right.C_{ s2})=\ldots=D(A_{j}\left|\right.C_{st})\), (\(j=1,\ldots,m\)). \(\square\)  
To look more deeply, the essential effect of parameter \(r\) in \(Clope_{r}(P)\) is the subjectively adjusted trade-off between compactness and separation (assumption). The compactness would be less important in the evaluation as the value of \(r\) decreases, and it would be entirely ignored when \(r=0\) (although it is avoided). As a result, in Table 3, \(Clope_{2}(P)\) and \(Clope_{3}(P)\) choose the partition with more clusters, due to the effect of compactness, and \(Clope_{1}(P)\) chooses the partition of least clusters, due to the effect of the separation assumption. Therefore, setting \(r\) is actually setting the preference for compactness or separation, and could be unfounded in the unsupervised scenario. The compactness measure of index \(R\) is also monotonic to the partition size, since maximizing the compactness of \(R(P)\) can be easily proven to be equivalent to minimizing the function \(F(P)\). The separation measure of \(R(P)\) evaluates the inter-cluster similarity \(Sim(C_{t},\)\(C_{s})\) pairwise. The compactness and the separation of \(R(P)\) only considers the most and least common values, respectively, which might lower the sensitivity to the value distribution. We will test and discuss the effectiveness of such methods in the experimental section.