SlowGuess/ACL_2020_OCR huggingface dataset:
Table 7: Full experimental results (\%) of various models on the classification tasks (gdy: argmax, smp: sampling; constr: constrained decoding). Some models did not predict the positive class for "in-group language," their performance is denoted by "-". We bold the  $F_{1}$  scores of the best performing model(s) on the development set. For easier interpretation, we also report the percentage of instances in the positive class in the development set.
${post}
- crawled from ${source}
1.a) Could this post be considered offensive, disrespectful, or toxic to anyone/someone?
Yes, this could be offensive.
Maybe. I'm not sure.
No, this is harmless.
I don't understand the post
1.b) Was the intent of this post to be offensive/disrespectful to anyone?
E.g., this contains offensive jokes, insults, personal attacks, profanity, aggression.
Yes, definitely
Yes, probably
No, probably not
No, definitely not
2) Does this post contain or allude to sexual content/acts?
This includes euphemism, double entendre, etc.
Yes
$\bigcirc$  Maybe
No
3) Who is referred to/targeted by this post? - Select all identity-based groups that apply.
3.a) Does the post imply offense/disrespect to an identity-related group of people?
E.g., is this just a insult to one person ("Fuck you", "All y'all are morons"), or an offensive comment/insult about a whole identity-based group ("Fuck muslims", "Fuck you, you stupid nigger").
Yes, this could be offensive to a group (minority, identity-based, etc.)
No, this is just an insult to an individual or a non-identity-related group of people.
race/ethnicity
3.b)
Which identity group is referred to in this post?
black folks
□ asian folks
□ latino/latina folks
□ native american/first nation folks
other
3.c) What aspect/stereotype/characteristic of this group (often unfairly assumed) is referenced or implied by this post? — Use simple phrases and do not copy paste from the post.
I.e., actions/characteristics that US society (usually wrongly) associates with the group
GROUP does
GROUP does
#
#
[optional]
gender/gender identity/sexuality
culture/origin/religion
age/body
mental or physical disabilities/disorders
socio-economic/political/lifestyle
crime/violence/tragedy victims
3.d) Does the author of the post sound like they belong to the same minority group that is referred to by this post?
Try your best to guess. For example, posts with "nigga" in them usually come from black authors. Additionally, members sometimes make fun of their own
M
Yes
Maybe
No
Figure 4: Snippets of the annotation task used to collect SBIC. The collection of structured annotations for our framework is broken down into questions pertaining to offensiveness, intent of the post, targeted group and minority speaker.