ethical-spectacle/social-bias-ner

Token Classification • Updated Oct 28, 2024 • 7.88k • 1

Note SoTA Model for Social Bias Multi-Label Token Classification

Note Dataset containing GUS entities (main dataset used). Synthetic annotations by gpt-4o agents with DSPy pipeline.

Note Underlying corpus used in GUS dataset, synthetically generated by Mistral 7B to contain a balanced distribution of demographics. This dataset also contains text sequence classes for types of bias (could be used in classifier).

Note GUS dataset formatted for LLMs in chatml format (role/content). Beware: LLMs underperformed encoder-only token classifiers (poor token/label alignment, and resource intensive inference).

ethical-spectacle/phi-3.5-4B-social-bias-ner-lora

Updated 30 days ago

Note Literally not a single response could be parsed/aligned lol.

Note Another reputable dataset, annotated with our GUS entities pipeline.