alisa-yingjia-wan/gapa-predictor-olmo2-7b

GAPA Predictor (OLMo-2 7B + LoRA)

Model released with How Humans and LLMs Read Gender into "Gender-Neutral" Physical Descriptions (COLM 2026).

This model predicts how strongly a physical description reads as gendered to a human reader. Given a physical attribute such as "facial stubble" and a described gender (woman, man, or nonbinary person), it returns the mean rating a US-based annotator would give on a 1–7 scale, where 1 means not at all associated and 7 means very strongly associated.

Collecting those ratings from people is slow and expensive, which is what makes them difficult to use at scale. The GAPA predictor, trained on the GAPA dataset, lets you score descriptive language across a whole corpus instead of a hand-annotated sample. In the paper we use it to analyze character descriptions across the 100 novels of LitBank.

What it does

Each call pairs one attribute with one described gender, fills them into a fixed prompt, and regresses the sentence to a single number on the human rating scale:

Scoring one attribute against all three genders therefore means three forward passes. The predict() helper does this for you and returns the results as a table:

Attribute	woman	man	nonbinary person
facial stubble	2.37	5.57	3.55
a graceful figure	5.76	3.16	4.09
a wide jawline	3.25	5.52	3.79
a curvy build	5.85	3.40	4.16
glowing eyes	3.62	3.58	3.54

Quick start

The repository ships only the 33 MB LoRA adapter and the regression head, so the base model is downloaded from allenai/OLMo-2-1124-7B the first time you load it (~14 GB). You will want a GPU with roughly 16 GB of memory for comfortable bf16 inference.

pip install torch transformers peft huggingface_hub pandas
hf download alisa-yingjia-wan/gapa-predictor-olmo2-7b --local-dir gapa-predictor
cd gapa-predictor

from modeling import GAPARegressor

model = GAPARegressor.from_pretrained(".")
df = model.predict(["facial stubble", "a graceful figure", "glowing eyes"])
print(df.pivot(index="attribute", columns="person_term", values="rating").round(2))

predict() scores every attribute against all three described genders by default, batching the forward passes (batch_size=32), and returns a tidy DataFrame of attribute, person_term, and rating. Pass person_terms=[...] to restrict the comparison, or as_dataframe=False if you would rather have plain Python objects.

Model details

The architecture is an OLMo-2 7B backbone adapted with LoRA, plus a small regression head that turns the sentence representation into a single score. Only the LoRA adapter and the head were trained; the base weights stay frozen.

Component	Detail
Base model	allenai/OLMo2-7B-1124 (also addressable as allenai/OLMo-2-1124-7B)
Adapter	LoRA, 33 MB, in lora_adapters/
Regression head	Linear(4096, 1) → Sigmoid over the mean-pooled final-layer hidden state, in regression_head.pt
Objective	Scalar regression, MSE loss against ratings normalized to [0, 1]
Prompt template	A {person_term} has {attribute}.
Tokenization	truncation=True, padding="max_length", max_length=128
Output range	[0, 1] internally, rescaled by x * 6 + 1 to the 1–7 scale

Training configuration

All models in the accompanying hyperparameter search shared a standardized budget.

LoRA		Optimization		Schedule
r	16	learning rate	1e-4	max epochs	15
lora_alpha	16	scheduler	cosine	early stopping patience	3
lora_dropout	0.1	warmup ratio	0.1	max sequence length	128
target modules	q_proj, v_proj	weight decay	0.01	seed	123
		batch size	8

The full configuration file and the training script live in predictor/.

Training data

The GAPA predictor is trained on the mean rating per attribute–gender pair in the GAPA dataset, which gives 945 training targets in total (315 attributes × 3 genders).

Evaluation

Results are on the held-out test split of 282 attribute–gender pairs, reported on the 1–7 scale.

Person term	Pearson r	RMSE	MAE	n
woman	0.807	0.653	0.528	94
man	0.717	0.698	0.560	94
nonbinary person	0.590	0.536	0.424	94
Overall	0.764	0.633	0.504	282

How to interpret these numbers. The rating task is subjective, so perfect correlation is not attainable: a typical individual annotator matches the consensus of the others at only r = 0.506 (leave-one-rater-out), while √ICC(1,k) = 0.961 marks the ceiling an ideal model could reach given measurement noise in the averaged ratings. At r = 0.764 the model predicts the group consensus considerably better than any single human rater. See §4.3 and Appendix B.3 of the paper for how both statistics are computed and interpreted.

_{Paper Figure 7 — training results of the predictor, evaluated on GAPA (test). Dashed lines mark the two reliability bounds above.}

Intended use and limitations

The model is intended for research on gender associations in descriptive language: auditing generated image or person descriptions, corpus-scale sociolinguistic analysis, and studying how closely language models track human interpretation. The associations are culturally and temporally specific.

This model predicts how US-based annotators associate a description with a gender category. It says nothing about the person being described. A high man rating for "facial stubble" reports a pattern in reader interpretation, not a fact about anyone's gender. It must not be used to infer, assign, or verify any individual's gender identity.

Citation

@inproceedings{wan2026gapa,
  title         = {How Humans and LLMs Read Gender into ``Gender-Neutral'' Physical Descriptions},
  author        = {Wan, Yingjia and Lin, Lin L. and Kreiss, Elisa},
  booktitle     = {Conference on Language Modeling (COLM)},
  year          = {2026},
  eprint        = {2609.16366},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CL},
  url           = {https://arxiv.org/abs/2609.16366}
}

Licence

Apache 2.0, inherited from the base model: these are LoRA adapter weights over allenai/OLMo-2-1124-7B, and an adaptation cannot be released under looser terms than what it adapts. The dataset and the code are MIT.