SlowGuess/NAACL_2021_OCR huggingface dataset:
Table 10: The ICR for labels of each topic set using Krippendorff's  $\alpha$ , Fleiss'  $\kappa$ , and Percentage Agreement  $Q_{agr}$
LDA	MetaLDA	LDA	MetaLDA
Qalin	Qalin	Qdif	Qdif
All	ρ	p	ρ	p	ρ	p	ρ	p
CNPMI-ABC	-0.0423	0.7486	0.141	0.283	-0.1454	0.2676	0.1688	0.197
CNPMI-AP	0.3394	0.008	0.39	0.002	0.2224	0.0877	0.3694	0.004
CNPMI	-0.2156	0.0981	0.148	0.259	-0.1291	0.3257	0.2342	0.072
CA	0.123	0.3491	0.297	0.021	-0.0672	0.61	0.2276	0.080
CP	-0.1016	0.4397	0.226	0.082	-0.0712	0.5887	0.1334	0.310
CV	0.2376	0.0676	0.147	0.264	-0.079	0.5485	0.2321	0.074
CUnmass	-0.397	0.0017	0.029	0.827	-0.1671	0.2018	0.0131	0.921
Proportion	0.0717	0.5861	0.09	0.493	0.0037	0.9774	0.0536	0.684
Effwords	-0.2657	0.0402	-0.239	0.066	-0.1528	0.2438	-0.285	0.027
LDA	MetaLDA	LDA	MetaLDA
Qaln	Qaln	Qdif	Qdif
Top25pcnt	ρ	p	ρ	p	ρ	p	ρ	p
CNPMI-ABC	0.8245	0.0002	0.08	0.778	0.1643	0.5586	-0.1956	0.485
CNPMI-AP	0.4838	0.0677	0.11	0.697	0.2918	0.2914	-0.2999	0.278
CNPMI	0.5904	0.0205	0.347	0.206	0.0545	0.8469	0.2962	0.284
CA	-0.3135	0.2552	-0.014	0.961	-0.3557	0.1932	-0.3181	0.248
CP	0.568	0.0272	0.236	0.397	0.1609	0.5667	0.24	0.389
CV	-0.3442	0.209	0.119	0.674	-0.2046	0.4645	0.4085	0.131
CUmass	-0.2858	0.3017	-0.326	0.235	-0.2283	0.4132	-0.3965	0.143
Dtp	-0.0897	0.7505	-0.592	0.02	0.2163	0.4388	-0.5307	0.042
Dew	-0.1391	0.621	-0.776	0.001	-0.1171	0.6778	-0.7638	0.001
Qalin	Qalin	Qdif	Qdif
Top50pcnt	ρ	p	ρ	p	ρ	p	ρ	p
CNPMI-ABC	0.079	0.6781	0.213	0.259	-0.0166	0.9308	0.0862	0.651
CNPMI-AP	0.2038	0.2801	0.045	0.815	0.2194	0.2441	-0.1221	0.521
CNPMI	0.3603	0.0505	0.227	0.228	0.0184	0.9233	0.1811	0.338
CA	-0.0986	0.6041	0.12	0.528	-0.1881	0.3194	0.0143	0.940
CP	-0.0628	0.7415	0.342	0.065	-0.1011	0.5949	0.221	0.241
CV	0.0694	0.7157	-0.217	0.248	-0.1216	0.5223	0.019	0.921
CUmass	-0.4059	0.026	-0.023	0.904	-0.3119	0.0933	0.1051	0.581
Dtp	0.2278	0.2261	-0.338	0.068	0.0306	0.8726	-0.256	0.172
Dew	-0.2545	0.1748	-0.65	0	-0.2132	0.2581	-0.6298	0.000
LDA	MetaLDA	LDA	MetaLDA
Qalin	Qalin	Qdif	Qdif
Bot15pcnt	ρ	p	ρ	p	ρ	p	ρ	p
CNPMI-ABC	0.5317	0.1407	0.037	0.924	0.8165	0.0072	-0.2282	0.555
CNPMI-AP	0.1581	0.6845	-0.091	0.815	0.1862	0.6315	-0.0797	0.839
CNPMI	-0.0851	0.8276	0	1	0.2294	0.5527	-0.0913	0.815
CA	0.0769	0.844	-0.159	0.682	-0.1491	0.7019	-0.01	0.980
CP	-0.4473	0.2274	-0.169	0.663	-0.1101	0.778	0	1.000
CV	0.2946	0.4416	0.356	0.347	-0.2092	0.5891	0.2926	0.445
CUmass	0.4873	0.1833	-0.186	0.631	-0.3119	0.0933	0	1.000
Dtp	-0.523	0.1486	0.523	0.149	-0.3486	0.3579	0.2739	0.476
Dew	0.2305	0.5507	0.693	0.039	0.3578	0.3444	0.2635	0.493
LDA	MetaLDA	LDA	MetaLDA
Qalin	Qalin	Qdif	Qdif
Bot10pcnt	ρ	p	ρ	p	ρ	p	ρ	p
CNPMI-ABC	0.6377	0.1731	0.463	0.355	0.6768	0.1398	-0.0926	0.862
CNPMI-AP	-0.0304	0.9545	0	1	-0.206	0.6954	0.1014	0.848
CNPMI	-0.3189	0.5379	0.44	0.383	-0.0304	0.9545	0.8783	0.021
CA	0.239	0.6483	0.101	0.848	-0.2	0.6059	0	1.000
CP	0.7537	0.0835	-0.44	0.383	0.8024	0.0547	0.0976	0.854
CV	0.1543	0.7704	0.741	0.092	-0.3719	0.4679	0.7407	0.092
CUmass	0.7537	0.0835	-0.216	0.681	0.8024	0.0547	0.414	0.414
Dtp	-0.8452	0.0341	0.828	0.042	-0.8452	0.0341	0.6831	0.135
Dew	-0.0883	0.8679	0.82	0.046	-0.4414	0.3809	0.4938	0.320
F Examples
Examples of poorly aligned topics are shown in Table 11.
Topic Label	Collection Label	Topic	NPMI
Gun Control	Foreign interference act	law, bill, power, gun, democracy, control, freedom, rule, protect, legislation	0.0734
Cost of Living	Politician's rental prop-erty	house, free, property, home, rent, pay, live, buy, move, money	0.0814
Addiction help	Legalization of drugs	health, drug, care, test, medical, doctor, access, alco-hol, live, death	0.0702
Table 11: Topics which did not align well with the document-collection despite having a high coherence.