Before revision:  We advise that efforts to debias embedding spaces be always also paired with measurement of downstream model bias, and suggest that that community increase effort into making downstream measurement more feasible via creation of additional challenge sets and annotated test data.
After revision:  We urge researchers working on debiasing to focus on extrinsic measures of bias, and suggest that that community increase effort into making downstream measurement more feasible via creation of additional challenge sets and annotated test data.