SlowGuess/NAACL_2022_OCR huggingface dataset:
In the encoder phase, we set the number  $P$  of randomly selected words in the [Span] to 2, the reason for which is illustrated in parameter analysis. Therefore the output entity-level features  $\mathbf{h}_i$  and  $\mathbf{h}_i'$  possess the dimension of  $4 \cdot b_R$ , where  $b_R = 768$ . We use the pretrained BERT-Base-Cased model to initialize both the Momentum Encoder and Propulsion Encoder respectively, and use AdamW (Loshchilov and Hutter, 2017) to optimize the loss. The encoder is trained for 20 epochs with  $1e - 5$  learning rate. In the HPC phase, we set the numbers of layers  $L$  to 3 after parameter analysis and the maximum iterations at Line 8 to 400 to make sure the algorithm terminates in time and make the converge condition as  $E^l$  not change for 10 iterations. We set temperature parameter  $\tau = 0.02$  and momentum parameter  $m = 0.999$  following (He et al., 2020) and adjust the number of negative samples  $J$  to 512 to accommodate smaller batches.
B Evaluation metrics
We follow previous works and use  $\mathrm{B}^3$  (Bagga and Baldwin, 1998), V-measures (Rosenberg and Hirschberg, 2007) and Adjusted Rand Index (ARI) (Hubert and Arabie, 1985) as our end metrics.  $\mathbf{B}^3$  uses precision and recall to measure the correct rate of assigning data points to its cluster or clustering all points into a single class. We use V-measures (Rosenberg and Hirschberg, 2007) to calculate homogeneity and completeness, which is analogous to  $\mathbf{B}^3$  precision and recall. These two metrics penalize small impurities in a relatively "pure" cluster more harshly than in less pure ones. We also report the F1 value, which is the harmonic mean of Hom. and Comp. Adjusted Rand Index (ARI) (Hubert and Arabie, 1985) measures the similarity of predicted and golden data distributions. The range of ARI is [-1,1]. The larger the value, the more consistent the clustering result is with the real situation.