just-bert-aug-strat

Model accompanying the project:

(link will be updated once released)

---

This repository provides the stratified augmented BERT classifier used in experiments evaluating whether grammatical perspective influences moral acceptability predictions.

The model is fine-tuned on the Justice dataset and predicts binary moral acceptability judgments for short scenario descriptions.

This model corresponds to the model trained on a stratified subset of perspective-augmented training data, balancing exposure across perspectives.

---

Related models

This model is part of a set of three models used in the experiments:

Model	Description
just-bert-base	baseline model trained on original sentences
just-bert-aug-full	trained on fully perspective-augmented training data
just-bert-aug-strat	trained on a stratified subset of perspective-augmented training data

All three models are available on Hugging Face.

---

Model description

just-bert-aug-strat is a BERT-base-uncased model fine-tuned for binary classification.

The model predicts whether a scenario is morally acceptable or unacceptable based on natural language descriptions.

It serves as the stratified augmented model, trained on a controlled subset of perspective-rewritten data.

---

Experimental context

The model is part of a study evaluating whether grammatical perspective influences moral acceptability predictions in language models.

The experiment compares:

• Baseline model trained on original sentences
• Augmented models trained on perspective-rewritten sentences

The analysis measures:

• prediction flip rate
• probability bias across perspectives
• confidence instability
• statistical significance of differences

---

Training data

The model was trained using the Justice dataset, with perspective-rewritten variants.

Instead of using all generated variants, the training data is stratified across perspectives, which tries to balance representation while limiting dataset expansion.

Example input:

You used to walk around the park with your coworker on lunch but you don't anymore because she reads on her lunch break.

Example prediction:

acceptable

Training uses a subset of perspective variants.

---

Model details

Property	Value
Architecture	BERT
Base model	bert-base-uncased
Task	Binary classification
Labels	moral acceptability
Framework	Hugging Face Transformers

---

Evaluation

Models were evaluated on four dataset splits.

Model	test	test_hard	test_pers	test_hard_pers
just-bert-base	Acc 0.7456 / F1 0.7714	Acc 0.5838 / F1 0.6433	Acc 0.7457 / F1 0.7731	Acc 0.5832 / F1 0.6426
just-bert-aug-strat	Acc 0.7496 / 0.7707	Acc 0.5848 / F1 0.6381	Acc 0.7515 / F1 0.7759	Acc 0.5877 / F1 0.6458
just-bert-aug-full	Acc 0.7726 / F1 0.7899	Acc 0.6145 / F1 0.6641	Acc 0.7711 / F1 0.7907	Acc 0.6152 / F1 0.6674

Evaluation splits:

• test - original scenarios
• test_hard - challenging scenarios
• test_pers - perspective-rewritten scenarios
• test_hard_pers - challenging scenarios with perspective rewrites

Full experimental results and statistical analyses are available in the accompanying repository.

---

Intended use

This model is intended for research purposes, including:

• studying moral judgment prediction in language models
• evaluating robustness to linguistic framing
• analyzing perspective sensitivity in NLP systems

The model should not be used in real-world moral decision-making systems.

---

Limitations

The model inherits limitations from both:

• the Justice dataset
• the BERT architecture

Known limitations include:

• potential sensitivity to linguistic framing
• potential biases present in the training data
• limited reasoning about complex moral scenarios

The purpose of the accompanying research is precisely to investigate these sensitivities.

---

Ethical considerations

Predictions produced by the model represent statistical patterns learned from the dataset, not objective moral judgments.

Outputs should therefore be interpreted cautiously and used only for research purposes.

---

Repository

Full experimental code and additional models are available at:

(links will be added upon release)

---

License

This model is released under the MIT License.