FinBERT-financial-news

A continued fine-tune of ProsusAI/finbert on 5,900 sentences from real English-language financial news articles (2010s-2020s), labeled by an LLM (DeepSeek deepseek-chat) from an investor/price-impact perspective — not a from-scratch retrain: the label space (positive/negative/neutral) and task framing are unchanged from the base model, only the training sentences are new.

Why this exists

ProsusAI/finbert was fine-tuned on Financial PhraseBank — ~4,840 sentences from 2014 English-language news about OMX Helsinki (Nordic) listed companies, via LexisNexis. That's a real, measurable domain/vocabulary gap for a pipeline scoring 2010s-2020s English-language financial news on globally-listed companies: real disagreement cases found during evaluation included the base model missing "crushed" as a positive idiom ("Amazon and Alphabet crushed earnings"), and terms/instruments (e.g. cryptocurrency) that didn't meaningfully exist in the base model's training window. This model targets that specific gap — a vocabulary/domain refresh, not an architecture or label-space change.

Training data

• Base draw (5,000 sentences): split from the body_text of real financial-news articles already sampled across this project's own LLM-as-judge sentiment evaluation runs (a pool naturally covering positive/negative/neutral content via that harness's stratified sampling design, not a fresh uniform draw). Class distribution: 938 positive (18.8%), 893 negative (17.9%), 3,169 neutral (63.4%) — real financial-news sentences skew neutral/factual, consistent with prior findings on this same corpus.
• Idiom-family augmentation round (900 sentences, added after publishing v1 of this model): a manual spot-check of the first published version found it still mislabeled sentences like "Amazon and Alphabet crushed earnings" as negative — the base draw happened to contain almost none of this idiom family (only 75 hits found across the 11,322-article eval pool). A second, targeted mining pass scanned the full ~480k-article source corpus for the same idiom family (crushed/smashed/trounced/clobbered/routed/walloped/demolished/hammered + earnings/estimate/ guidance/consensus/stock context) and labeled 900 more sentences the same way. This idiom family is genuinely ambiguous, not one-directional — "Nvidia stock got crushed" is negative (the company/stock is the object) while "Meta crushed its earnings estimates" is positive (an estimate/target is the object) — so labeling used an explicit instruction covering both directions rather than a blanket rule. Class counts: 487 negative, 316 positive, 97 neutral. 100 of these were held out entirely from training as an "idiom probe" set, purely to measure the fix directly (see Evaluation results below); the other 800 were merged into training.
• Labels: generated by deepseek-chat (temperature 0), one sentence at a time, in isolation, with the same investor/price-impact framing Financial PhraseBank's own human annotators used.
• Total / split: 5,900 sentences, 100% labeling success rate across both rounds. 80/10/10 train/validation/test, stratified per label (4,642 / 579 / 579) — plus the separate 100-row idiom probe, held out of all three splits.

Important limitation: these labels are LLM-generated (silver-standard), not human-annotated ground truth. They were not independently verified against a human-labeled reference set (beyond a manual spot-check of a 20-sentence sample of the idiom probe, which checked out). Treat the test-set metrics below as "agreement with this specific LLM's sentence-level judgment," not an absolute accuracy figure — the same caveat this project applies to every LLM-as-judge number in its own evaluation docs.

Training procedure

Continued fine-tuning from the ProsusAI/finbert checkpoint (not vanilla BERT), retrained from scratch on the full merged dataset (not a second round of fine-tuning on top of v1, to avoid double-fine-tuning drift) via Hugging Face Trainer:

• learning rate: 2e-5
• batch size: 16 (train) / 32 (eval)
• epochs: 4
• weight decay: 0.01
• mixed precision (fp16)
• load_best_model_at_end=True, selected by validation macro F1 (best was epoch 3)
• seed: 42

Evaluation results

Held-out sentence-level test set (n=579, LLM-labeled — see limitation above)

metric	v1 (base draw only, n=498)	v2 (+ idiom augmentation, n=579)
Accuracy	0.813	0.798
Macro F1	0.774	0.779
F1 — positive	0.798	0.775
F1 — negative	0.667	0.726
F1 — neutral	0.858	0.838

v2's test set is a harder mix (it includes the idiom-augmented sentences), so accuracy dips slightly while macro F1 and, notably, negative F1 both improve — consistent with the targeted fix below rather than a regression.

Idiom probe (n=100, held out of training entirely — direct before/after on the original bug)

The exact failure class that motivated this update, measured directly: 100 idiom-family sentences (crushed/smashed/trounced/clobbered/routed/walloped/demolished/hammered + earnings/estimate/guidance context) never seen during training, scored by the currently- published model before this update (v1) vs. after (v2, this version):

metric	v1 (published, pre-fix)	v2 (this version)
Accuracy	0.750	0.870
Macro F1	0.664	0.759
Recall — positive (the original bug: "crushed earnings" mislabeled negative)	0.710	0.903
Recall — negative	0.797	0.932
F1 — positive	0.759	0.889
F1 — negative	0.847	0.917

The exact original failure case now scores correctly: "Amazon and Alphabet crushed earnings." → positive (0.935 confidence), while "The stock got crushed after the disappointing guidance." → negative (0.994 confidence) — the model has learned to condition on what is being crushed/smashed/hammered, not just pattern-match the verb. neutral on this idiom-specific probe is still weak (f1 0.47, up from 0.39) — expected, since almost none of the 900 mined sentences were neutral (97/900) and this probe isn't representative of the pipeline's overall neutral-heavy traffic; see the downstream pipeline table below for neutral performance on real traffic.

Downstream pipeline evaluation (real financial news, LLM-judge, article-level)

This model was also evaluated in the actual production pipeline it was built for — portfolio-nlp — substituted into an entity-scoped, chunk-level aggregation scheme (score each ~510-token chunk, weight chunks naming the article's own subject company/ticker over everything else), on the same 2,000-article real-news sample and LLM-judge harness used to validate every other candidate design for this pipeline's sentiment stage:

metric	base FinBERT (same aggregation)	fine-tuned v1	fine-tuned v2, this version
recall_negative (this pipeline's priority metric)	0.856	0.812	0.808
precision_negative	0.376	0.505	0.513
f1_negative	0.523	0.623	0.628
macro_f1_vs_judge	0.625	0.737	0.731
agreement_rate	0.585	0.697	0.701
agreement_rate_representative	0.700	0.852	0.866
recall_positive	0.520	0.790	0.801
precision_neutral	0.871	0.935	0.936
mean_severity (lower is better)	0.481	0.350	0.341

The idiom-augmentation fix holds steady-to-slightly-better on every aggregate downstream metric (v1→v2 deltas are all within ±0.01, i.e. noise) while directly fixing the specific, previously-disclosed idiom failure above — not a trade-off, a real fix with no measured cost on the broader real-traffic distribution.

See docs/evaluation.md's 2026-09-13 fine-tuning follow-up in the repo above for the full comparison, methodology, and caveats (the LLM-judge is itself a model, not ground truth; the judge and this model's training labels both ultimately trace to the same LLM provider, a methodological limitation disclosed there in full, not glossed over here).

Usage

from transformers import AutoTokenizer, AutoModelForSequenceClassification, pipeline

tokenizer = AutoTokenizer.from_pretrained("gamug/FinBERT-financial-news")
model = AutoModelForSequenceClassification.from_pretrained("gamug/FinBERT-financial-news")

classifier = pipeline("text-classification", model=model, tokenizer=tokenizer)
classifier("Acme Corp reported record profit and raised its full-year guidance.")

Recommended pre/post-processing: same as the base model — chunk long documents rather than truncating (this checkpoint keeps BERT-base's 512-token limit), and if aggregating multiple chunks/sentences per document yourself, consider scoping/weighting toward the actual subject of the document rather than a plain average (see the portfolio-nlp repo above for one worked design and its measured trade-offs).

License

Derivative of two upstream works with different licenses:

• Base model ProsusAI/finbert: itself derivative of nlpaueb/sec-bert-base-adjacent work and Financial PhraseBank (CC-BY-NC-4.0).
• Training data here: LLM-generated from real news article text; the underlying articles are not redistributed, only derived per-sentence sentiment labels used for training.

Released under CC-BY-NC-4.0, matching the base model's own license: attribution required, non-commercial use only.