Before revision:  We additionally release code, a new intrinsic metric, and an annotated test set for gender bias for hatespeech .
After revision:  We additionally release code, a new intrinsic metric, and an annotated test set focused on gender bias in hate speech .