Woochachino/cbsent

Sentence-level hawkish/dovish stance for Federal Reserve and Bank of Canada communications. Trained and evaluated with cbsent; every number below is reproducible from that repository with one command and is recorded in its RESULTS.md.

Use

from transformers import AutoModelForSequenceClassification, AutoTokenizer
import torch

tok = AutoTokenizer.from_pretrained("Woochachino/cbsent")
model = AutoModelForSequenceClassification.from_pretrained("Woochachino/cbsent")

text = "Inflation remains elevated and the labour market is tight."
with torch.no_grad():
    probs = model(**tok(text, return_tensors="pt")).logits.softmax(-1)[0]
print(model.config.id2label[int(probs.argmax())])  # hawkish

Or through the package, which adds central-bank sentence segmentation and document-level aggregation:

pip install "cbsent[inference]"
cbsent score "Inflation remains elevated."

How it was built

1. Domain-adaptive pretraining. Continued masked-language-model training of answerdotai/ModernBERT-base on 269,210 unlabelled Fed and Bank of Canada sentences (8.6M tokens), for 2 epochs. Held-out MLM loss 1.6064 after adaptation.
2. Fine-tuning. On the benchmark's own train split, extended with 1,200 Bank of Canada sentences. 2,867 training sentences, seed 20250811, best epoch 5 chosen on a held-out validation slice.

Both of those steps were tested against controls at three seeds each, and neither helped on this benchmark: domain-adaptive pretraining measured at -0.002 weighted F1 against an off-the-shelf backbone, and the Bank of Canada extension at +0.004, both inside a between-seed spread of 0.013 to 0.026. The adaptation did cut held-out masked-language-model loss by 24%, which did not transfer to classification accuracy. The steps are described because they are what produced these weights, not because they are recommended.

Evaluation

system	weighted F1	macro F1	accuracy
dictionary (Apel & Blix Grimaldi)	0.5478	0.5156	0.5605
cbsent fine-tune, 3-seed mean	0.658
zero-shot gpt-5	0.7133	0.7019	0.7137

Scored on the held-out test split of the FOMC hawkish-dovish benchmark (Shah, Paturi & Chava, ACL 2023), 496 sentences annotated by its authors. Inference for reported numbers runs on CPU, which is deterministic here; MPS is not (measured in the repository's RESULTS.md).

Intended use

Research on central bank communication: scoring sentences or documents for policy stance, and building point-in-time tone indices where the publication timestamp of every input is known.

Limitations

• Not state of the art. Zero-shot GPT-5 scores higher on this benchmark. What this model offers is roughly 120 sentences/second locally at zero marginal cost, deterministic and reproducible output, and no data leaving the machine.
• Negation is handled poorly. On a 24-item minimal-pair probe it gets 4 of 10 negated sentences right, against 10 of 10 for a frontier LLM. Do not use it where negated policy constructions carry the signal.
• English only, and specific to Fed and BoC prose. Other central banks and other financial text are out of distribution.
• Labels are sentence-level and context-free by construction: a sentence whose stance depends on the surrounding paragraph is labelled neutral.
• Part of the training data carries LLM-generated labels; see the repository for the provenance breakdown.
• The stance score is a model output, not a forecast, and nothing here is investment advice.

Training data

• benchmark train split: 2,867 sentences total after extension
• Bank of Canada extension: 1,200 sentences, of which 0 human-verified (the remainder carry LLM bootstrap labels)
• validation: 317 sentences held out of the training pool
• test: 496 sentences, the benchmark's official split, untouched during training and model selection

License

CC BY-NC 4.0, inherited from the benchmark dataset this model was fine-tuned on. The cbsent source code is MIT.

Citation

The benchmark this model is trained and evaluated on:

@inproceedings{shah-etal-2023-trillion,
    title = "Trillion Dollar Words: A New Financial Dataset, Task & Market Analysis",
    author = "Shah, Agam and Paturi, Suvan and Chava, Sudheer",
    booktitle = "Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics",
    year = "2023",
    pages = "6664--6679",
}