# Clustering of a Health Dataset Using Diagnosis Co-Occurrences
## 2 Materials and Methods
### Statistical Analysis: A New Hierarchical Agglomerative Clustering Algorithm
#### 2.2.1 Similarity Measure  
Clustering categorical data like ICD10 diagnostic codes is challenging, as the lack of direct continuous and Euclidian distance-based similarity measures requires innovative approaches [29]. In this study, we propose to take advantage of multimorbidity, relying on statistical co-occurrences of pairs of diagnoses on patterns, to measure their similarity. Hence, we introduce a new measure of similarity based on the risk of co-occurring diagnoses in an episode of ED visits, namely the relative risk described by Equation (1):  
\[\mathrm{RR_{ij}}=\frac{\mathrm{p_{ji}}}{\mathrm{p_{i}P_{j}}} \tag{1}\]  
where \(\mathrm{p_{ji}}\) designates the probability of diagnoses i and j co-occurring in episodes of ED visits and \(\mathrm{p_{i}}\) (respectively \(\mathrm{p_{j}}\)) is the marginal probability that block i (respectively j) occurs in an episode and is used to weight the relation [30]. These probabilities could be computed using raw empirical observations from the contingency tables, which contain co-occurrences of pairs of diagnoses. Let \(\mathrm{a_{ij}},\ \mathrm{b_{ij}},\ \mathrm{c_{ij}}\) and \(\mathrm{d_{ij}}\) be the numbers of visits verifying, respectively, the positive-positive, positive-negative, negative-positive and negative-negative matches in the contingency table related to diagnostics i and j. Then, \(\text{p}_{\text{ij}}=\frac{\text{a}_{\text{ij}}}{\text{N}_{\text{e}}}\), \(\text{p}_{\text{i}}=\frac{\text{a}_{\text{ij}}+\text{b}_{\text{ij}}}{\text{N}_{ \text{e}}}\), \(\text{p}_{\text{j}}=\frac{\text{a}_{\text{ij}}+\text{c}_{\text{ij}}}{\text{N}_{ \text{e}}}\) and thus, Equation (1) is equivalent to Equation (2):  
\[\text{RR}_{\text{ij}}=\frac{\text{a}_{\text{ij}}\text{N}_{\text{e}}}{\big{(} \text{a}_{\text{ij}}+\text{b}_{\text{ij}}\big{)}\big{(}\text{a}_{\text{ij}}+ \text{c}_{\text{ij}}\big{)}} \tag{2}\]  
with \(\text{N}_{\text{e}}=\text{a}_{\text{ij}}+\text{b}_{\text{ij}}+\text{c}_{\text {ij}}+\text{d}_{\text{ij}}\) the total number of episodes, independent of the considered diagnoses. Even though this comorbidity measure is not new [31], this study is the first to use it for multimorbidity clustering purposes. This measure was chosen because it focusses on the relevant positive matches \(\text{a}_{\text{ij}}\) and provides a comprehensive interpretation with a comparison between the observed co-occurrence probability \(\text{p}_{\text{ij}}\) and the probability \(\text{p}_{\text{i}}\text{p}_{\text{j}}\), which corresponds to the probability of co-occurrence in the case where diagnoses would have been probabilistically independent. The relative risk measure \(\text{RR}_{\text{ij}}\) will be used for similarity computations, instead of the Jaccard similarity measure \(\text{J}_{\text{ij}}\) usually used with HAC approaches in the literature [16; 17; 20]. In our study, \(\text{J}_{\text{ij}}\) would be equivalent to \(\frac{\text{p}_{\text{ij}}}{\text{p}_{\text{i}}+\text{p}_{\text{j}}-\text{p} _{\text{i}}}=\frac{\text{a}_{\text{ij}}}{\text{a}_{\text{ij}}+\text{b}_{ \text{ij}}+\text{c}_{\text{ij}}}\). Compared to our relative risk measure, the Jaccard similarity measure is less sensitive to variation in marginal probabilities and does not consider well the external scarcity context of marginal occurrences that allow a better appreciation of the co-occurrence's strength. Furthermore, it cannot be associated with the notion of probability independence. [PERSON]'s Q coefficient of association, equivalent to \(\frac{\text{a}_{\text{ij}}\text{d}_{\text{ij}}-\text{b}_{\text{ij}}\text{c}_ {\text{ij}}}{\text{a}_{\text{ij}}\text{d}_{\text{ij}}+\text{b}_{\text{ij}} \text{c}_{\text{ij}}}\), is also frequently used as a similarity measure, but is not focused on positive matches \(\text{a}_{\text{ij}}\)[32].