SlowGuess/ACL_2020_OCR huggingface dataset:
$$
\operatorname {p r o x} (\text {n e g}, \text {n e g}) = - \log \frac {1 0 / 2}{1 0 0} = 4. 3 2 \quad \operatorname {p r o x} (\text {n e g}, \text {n e u}) = - \log \frac {1 0 / 2 + 6 0}{1 0 0} = 0. 6 2 \operatorname {p r o x} (\text {n e g}, \text {p o s}) = - \log \frac {1 0 / 2 + 9 0}{1 0 0} = 0. 0 7
$$
$$
\operatorname {p r o x} (\mathrm {n e u}, \mathrm {n e g}) = - \log \frac {6 0 / 2 + 1 0}{1 0 0} = 1. 3 2 \quad \operatorname {p r o x} (\mathrm {n e u}, \mathrm {n e u}) = - \log \frac {6 0 / 2}{1 0 0} = 1. 7 4 \quad \operatorname {p r o x} (\mathrm {n e u}, \mathrm {p o s}) = - \log \frac {6 0 / 2 + 3 0}{1 0 0} = 0. 7 4
$$
$$
\operatorname {p r o x} (\mathrm {p o s}, \mathrm {n e g}) = - \log \frac {3 0 / 2 + 6 0 + 1 0}{1 0 0} = 0. 2 3 \operatorname {p r o x} (\mathrm {p o s}, \mathrm {n e u}) = - \log \frac {3 0 / 2 + 6 0}{1 0 0} = 0. 4 2 \operatorname {p r o x} (\mathrm {p o s}, \mathrm {p o s}) = - \log \frac {3 0 / 2}{1 0 0} = 2. 7 4
$$
$$
\mathrm {C E M} ^ {\text {O R D}} (A, g) = \frac {5 * 4 . 3 2 + 5 * 0 . 6 2 + 7 * 0 . 0 7 + 1 * 1 . 3 2 + 5 0 * 1 . 7 4 + 8 * 0 . 7 4 + 4 * 0 . 2 3 + 5 * 0 . 4 2 + 1 5 * 2 . 7 4}{1 0 * 4 . 3 2 + 6 0 * 1 . 7 4 + 3 0 * 2 . 7 4} = 0. 7 1
$$
$$
\mathrm {C E M} ^ {\text {O R D}} (B, g) = \frac {7 * 4 . 3 2 + 1 2 * 0 . 6 2 + 4 * 0 . 0 7 + 1 * 1 . 3 2 + 4 5 * 1 . 7 4 + 8 * 0 . 7 4 + 2 * 0 . 2 3 + 3 * 0 . 4 2 + 1 8 * 2 . 7 4}{1 0 * 4 . 3 2 + 6 0 * 1 . 7 4 + 3 0 * 2 . 7 4} = 0. 7 6
$$
Figure 3: Example computation of  $\mathrm{CEM}^{\mathrm{ORD}}$  values for two hypothetical systems A and B with respect to the same dataset. The first two tables represent the confusion matrices for both systems. The third table shows  $prox(c_i, c_j)$  for the ground truth, according to the distribution of items in the negative, positive and neutral classes (10, 60 and 30, respectively). The rest of the equations illustrate how proximity values between classes are computed, and the resulting  $\mathrm{CEM}^{\mathrm{ORD}}$  values for both systems.
$$
\begin{array}{l} \operatorname {E f f} \left(g _ {d _ {1} \rightarrow c _ {2}}, g\right) - \operatorname {E f f} \left(g _ {d _ {3} \rightarrow c _ {2}}, g\right) \\ \propto \operatorname {E f f} (g, g) - \log \left(\frac {n _ {1} + \frac {n _ {2}}{2}}{N}\right) - \left(- \log \left(\frac {\frac {n _ {1}}{2}}{N}\right)\right) \\ \left. - \left(\operatorname {E f f} (g, g) - \log \left(\frac {n _ {3} + \frac {n _ {2}}{2}}{N}\right) - \left(- \log \left(\frac {\frac {n _ {3}}{2}}{N}\right)\right)\right) \right. \\ \propto \log \left(\frac {\frac {n _ {1}}{2} \left(n _ {3} + \frac {n _ {2}}{2}\right)}{\left(n _ {1} + \frac {n _ {2}}{2}\right) \frac {n _ {3}}{2}}\right), \\ \end{array}
$$
which is larger than 0 whenever  $n_1 > n_3$ .