SlowGuess/ACL_2020_OCR huggingface dataset:
Given the result  $P^{*}(y|x) = Q(y|x)$ , the consistent learners should be asymptotically immune to different assumptions regarding  $Q(Z)$ , where a
learner is defined as consistent if the learning algorithm can find a model  $\theta$  that is equivalent to the true model at producing class conditional probabilities given an exhaustive training data set (Fan et al., 2005). In practical, however, as the requirements are often hard to met, we note that models may still be affected due to the deviation between  $P^{*}(x)$  and  $Q(x)$ , which is widely studied in the covariate shift problem (Shimodaira, 2000; Ben-David et al., 2007; Jiang and Zhai, 2007). In our paper, as we don't assume the availability of extra resources and prior knowledge, we simply set  $P(Z) = Q(Z)$ . We leave more explorations about this assumption for future work.
C Details about the IPTTS Generation
Templates	Label
Verb inoffensive (identity-term).	0
Verb offensive (identity-term).	1
I am (identity-term).	0
I hate (identity-term).	1
Being (identity-term) is (adj. inoffensive).	0
Being (identity-term) is (adj. offensive).	1
You are a (adj. inoffensive) (identity-term).	0
You are a (adj. offensive) (identity-term).	1
Table 6: Templates used to generate IPTTS.
Type	Example Words
adj. inoffensive	great, fun, nice, neat
adj. offensive	disgusting, filthy, nasty, rotten
verb inoffensive	hug, love, like, respect
verb offensive	kill, murder, hate, destroy
male identity	actor, airman, boy, man
female identity	actress, airwoman, girl, woman
Table 7: Examples of slotted words to generate IPTTS.
For experiments with the Sexist Tweets dataset, we generate IPTTS following Park et al. (2018). The templates used are the same as Park et al. (2018), as shown in Table 6. We use the released codes by Dixon et al. (2018) and use the gender word pairs released by Zhao et al. (2018) as "identity-term." Some of the slotted words are presented in Table 7. To make sentences longer, we also add some semantic-neutral sentences provided by Dixon et al. (2018) as a suffix to each template. Finally, we get 75238 samples, 37538 of which are abusive, and the mean of sentence lengths is 17.5.
For experiments with Toxicity Comments datasets and Jigsaw Toxicity, we use the IPTTS
Term	Origin	Weight
Toxic	Overall	Δ	Toxic	Overall	Δ
white	5.98	2.13	3.85	2.89	2.14	0.75
black	3.10	1.07	2.03	1.22	1.07	0.15
muslim	1.57	0.58	0.99	0.58	0.59	-0.01
gay	1.29	0.35	0.94	0.39	0.34	0.05
american	2.70	2.11	0.59	2.76	2.13	0.63
canadian	1.38	1.82	-0.44	1.48	1.82	-0.34
old	2.62	2.18	0.44	2.63	2.18	0.45
christian	0.89	0.54	0.35	0.73	0.55	0.18
male	0.73	0.44	0.29	0.41	0.45	-0.04
blind	0.51	0.28	0.23	0.55	0.28	0.27
catholic	0.63	0.82	-0.19	0.65	0.83	-0.18
homosexual	0.26	0.08	0.18	0.09	0.08	0.01
straight	0.51	0.37	0.14	0.46	0.38	0.08
female	0.50	0.37	0.13	0.28	0.37	-0.09
transgender	0.21	0.09	0.12	0.10	0.09	0.01
african	0.30	0.20	0.10	0.20	0.20	0.00
jewish	0.28	0.19	0.09	0.17	0.19	-0.02
older	0.16	0.25	-0.09	0.15	0.25	-0.10
lesbian	0.11	0.03	0.08	0.03	0.03	0.00
african american	0.16	0.09	0.07	0.09	0.10	-0.01
mexican	0.20	0.13	0.07	0.17	0.13	0.04
heterosexual	0.09	0.03	0.06	0.03	0.03	0.00
Table 8: Frequency of a selection of identity-terms in toxic samples and overall in Jigsaw Toxicity dataset. % is omitted.
released by Dixon et al. (2018). The testing set is created by several templates slotted by a broad range of identity-terms, which consists of 77000 examples,  $50\%$  of which are toxic.
D Frequency of Identity-terms in Toxic Samples and Overall
To give a better understanding of how the weights change the distribution of datasets, we compare the original Jigsaw Toxicity dataset and the one with calculated weights for the frequency of a selection of identity-terms in toxic samples and overall, as shown in Table 8.
We can find that after adding weights, the gap between frequency in toxic samples and overall significantly decrease for almost all identity-terms, which demonstrate that the unintended bias in datasets is effectively mitigated.
4145