SallySims/equibert-generator

EquiBERT — DEI Text Generator

Model ID: SallySims/equibert-generator

T5-base fine-tuned for conditional DEI text generation. Generates inclusive, equitable organisational text across seven task types given a task prefix and input.

Task Prefixes

Prefix	Task	Input → Output
rewrite inclusive:	Inclusive rewriting	Biased text → Inclusive version
generate policy:	Policy generation	Topic → Policy section
generate jd:	Job description	Role description → Inclusive JD
rewrite framing:	Framing correction	Victim-blaming text → Structural framing
generate commitment:	DEI commitment	Goal → Measurable commitment
rewrite review:	Review debiasing	Biased review → Unbiased version
generate awareness:	Awareness content	Topic → Awareness statement

Usage

from transformers import T5ForConditionalGeneration, T5Tokenizer

model     = T5ForConditionalGeneration.from_pretrained("SallySims/equibert-generator")
tokenizer = T5Tokenizer.from_pretrained("SallySims/equibert-generator")

prompt = "rewrite inclusive: We need a rock star developer who can dominate the roadmap."
inputs = tokenizer(prompt, return_tensors="pt", max_length=256, truncation=True)
output = model.generate(**inputs, max_new_tokens=200, num_beams=4)
result = tokenizer.decode(output[0], skip_special_tokens=True)
print(result)
# "We are looking for a skilled developer with strong technical expertise
#  who can contribute meaningfully to our product roadmap."

Applications

• Automated inclusive job description generation
• DEI report framing improvement
• Performance review debiasing assistance
• Policy language generation
• Leadership communication coaching

Model Description

EquiBERT is a multi-task DEI (Diversity, Equity and Inclusion) transformer built on a dual-encoder backbone that fuses RoBERTa-base and DeBERTa-v3-base via a learned weighted sum (α parameter). The fused representation is fed into task-specific heads covering 17 distinct DEI analysis tasks.

Organisation: SallySims Framework: PyTorch + HuggingFace Transformers Backbone: RoBERTa-base + DeBERTa-v3-base (dual encoder, fused) Language: English Domain: Organisational DEI text — HR communications, policies, job descriptions, performance reviews, leadership statements, reports

Architecture

Input Text
    │
    ├──▶ RoBERTa-base encoder ──▶ Linear projection
    │                                     │
    └──▶ DeBERTa-v3-base encoder ──▶ Linear projection
                                          │
                              Weighted fusion (learned α)
                                          │
                                   Layer Norm + Dropout
                                          │
                              Task-specific head (see below)

Training Data

Trained on synthetic DEI organisational text generated by the EquiBERT synthetic data pipeline, covering 20 DEI categories across HR, policy, leadership, and workforce analytics domains. For production use, fine-tune on real labelled DEI data.

Limitations

• Trained on synthetic data — predictions should be validated before use in real HR or policy decisions.
• English-only.
• Not a substitute for qualified DEI practitioners or legal advice.
• May reflect biases present in the training corpus.

Citation

If you use EquiBERT in your research, please cite:

@misc{equibert2024,
  author    = {SallySims},
  title     = {EquiBERT: A Multi-Task DEI Transformer},
  year      = {2024},
  publisher = {HuggingFace},
  url       = {https://huggingface.co/SallySims}
}

All EquiBERT Models

Model	Task	Primary Metric
equibert-bias-classifier	Bias Detection	Macro F1
equibert-microaggression	Microaggression Detection	Macro F1
equibert-category-tagger	DEI Category Tagging	Macro F1
equibert-event-exclusion	Event Exclusion Classification	Macro F1
equibert-inclusive-language	Inclusive Language Scoring	Span F1
equibert-review-auditor	Performance Review Auditing	Span F1
equibert-washing-detector	DEI Washing Detection	MAE
equibert-framing-scorer	Report Framing Scoring	MAE
equibert-awareness-scorer	DEI Awareness Scoring	MAE
equibert-similarity	Semantic Similarity	Accuracy
equibert-ner	DEI Entity Recognition	Span F1
equibert-relation-extraction	Relation Extraction	Macro F1
equibert-qa	Extractive QA	Span EM
equibert-search	Semantic Search	MRR@10
equibert-nli	NLI / Textual Entailment	Macro F1
equibert-generator	DEI Text Generation	ROUGE-L