andreadm/reddit-pulse-gemma2_2b-xqdora

Reddit-pulse Gemma 2 2B xQDoRA+

Retrained checkpoint. This model was trained again after the paper, on the same gold dataset and with the same seed protocol. The metrics on this card come from that retraining and may differ from the ones reported in the paper.

Reddit-pulse Gemma 2 2B xQDoRA+ is a three-way directional inflation-expectation classifier for short English texts about the economy, fine-tuned on Reddit submission titles: a xQDoRA+ adapter on google/gemma-2-2b (4-bit NF4 base, DoRA adapters on the attention projections, LoRA+ learning rates). Given a title (or any sentence-length text), it predicts whether the text conveys that inflation / prices are going up, going down, or carries no directional signal (neutral).

It is not a sentiment model: "inflation falls sharply" is good news but labelled down; "rents are out of control" is bad news but labelled up. The direction is about the price level, not the mood.

The checkpoint is one of the small-model classifiers behind the Reddit inflation signal in

Del Monaco, A., Longo, L., Marcucci, J. & Tafani, I. (2026). Reddit's 'pulse' on US inflation: forecasting with large language models. Journal of Applied Econometrics, forthcoming. Working-paper version: Banca d'Italia, Questioni di Economia e Finanza (Occasional Papers) No. 1028, June 2026, doi:10.32057/0.QEF.2026.1028.

The fine-tuning and full-corpus inference code lives at andrea-dm/reddit-pulse.

Model lineage

Stage	Model	Notes
Base	google/gemma-2-2b	Pre-trained checkpoint
This checkpoint	andreadm/reddit-pulse-gemma2_2b-xqdora	QDoRA+/xQDoRA+ PEFT adapter on Reddit titles, three-way head down / neutral / up

Labels

id	label	encoding used in the paper's corpus files
0	down	-1
1	neutral	+0
2	up	+1

How to use

The adapter is loaded on top of the 4-bit quantized base model, exactly as it was trained (bitsandbytes and peft required). config.json in this repository carries the three-way head, the label names and the pad token, so no argument beyond the repository name is needed:

import torch
from peft import PeftModel
from transformers import (
    AutoConfig,
    AutoModelForSequenceClassification,
    AutoTokenizer,
    BitsAndBytesConfig,
)

name = "andreadm/reddit-pulse-gemma2_2b-xqdora"
tokenizer = AutoTokenizer.from_pretrained(name)
tokenizer.padding_side = "left"
config = AutoConfig.from_pretrained(name)
base = AutoModelForSequenceClassification.from_pretrained(
    "google/gemma-2-2b",
    config=config,
    dtype=torch.bfloat16,
    quantization_config=BitsAndBytesConfig(
        load_in_4bit=True,
        bnb_4bit_quant_type="nf4",
        bnb_4bit_use_double_quant=True,
        bnb_4bit_compute_dtype=torch.bfloat16,
    ),
    device_map="auto",
)
model = PeftModel.from_pretrained(base, name).eval()
encoding = {"down": -1, "neutral": 0, "up": 1}

texts = [
    "Inflation expectations drop to lowest level since 2021, NY Fed survey shows",
    "What is the difference between CPI and PCE?",
]
with torch.inference_mode():
    batch = tokenizer(texts, padding=True, truncation=True, max_length=1024, return_tensors="pt")
    ids = model(**batch.to(model.device)).logits.argmax(dim=-1).tolist()

print([encoding[config.id2label[i]] for i in ids])  # [-1, 0]

Do not hand the repository name to AutoModelForSequenceClassification directly: transformers' adapter shortcut rebuilds this DoRA adapter with different logits than the trained model.

Intended use and limitations

Intended use. Labelling large volumes of short, informal, English, economy-related texts (Reddit titles and comments, headlines, social media posts) with a directional inflation signal that is then aggregated over time — the paper's use case. The model is a building block for a high-frequency indicator, not a stand-alone oracle.

Limitations.

• Single predictions are noisy. The value of the model comes from averaging thousands of predictions per period, where idiosyncratic errors wash out. Do not rely on any single label.
• Domain and register. Trained on r/economy, r/Economics and r/wallstreetbets titles about US inflation from 2008 to 2022. Other countries, other registers and post-2022 vocabulary are out of distribution.
• Class imbalance. down is the minority class of the gold set and the hardest one; the loss was class-balanced during training, but down recall remains lower.
• Short texts. Fine-tuned on titles (median 11 words, max 52). Long comments are truncated and were not seen during training.
• Direction, not stance or sentiment. The model does not say whether the author wants inflation to move, nor whether the news is good or bad; only which way prices are said to be going.

Training data

The gold set is a hand-labelled sample of 1,383 Reddit submission titles from r/economy, r/Economics and r/wallstreetbets, dated February 2008 to December 2022. Labels were produced by a human-in-the-loop protocol (manual annotation assisted by zero-shot LLaMA-70B, a fine-tuned LLaMA-8B classifier and ChatGPT-assisted adjudication of disagreements) described in the paper. The gold set itself is not redistributed here.

label	titles	share
neutral	623	45.0 %
up	537	38.8 %
down	223	16.1 %

Training procedure

Seed protocol and model selection

The paper's protocol asks how sensitive the classifier is to which titles it is trained on, so it separates two sources of randomness:

• 19 split seeds each draw a different stratified 71 / 19 / 10 % train / validation / test partition (982 / 263 / 138 titles) of the same gold set.
• One fixed seed governs everything else: adapter and head initialisation, batch shuffling and dropout. Every split therefore trains the same model the same way on different data.

Each split is fine-tuned independently; the checkpoint with the median test weighted-F1 across the runs (the upper median) is kept and the others discarded. The result is a typical run, not the best one. This checkpoint is split seed 3266123502.

Hyperparameters

Base model	4-bit NF4, double-quantized (bitsandbytes), frozen
Adapters	DoRA (use_dora), rank r = 4, alpha = 32, dropout = 0.05, on k_proj, o_proj, q_proj, v_proj; classification head trained in full
Optimizer	AdamW (fused) with LoRA+ (adapter B matrices at 5x the base learning rate)
Learning rate	0.0001, cosine decay, no warm-up
Objective	Cross-entropy with balanced class weights (sklearn.utils.class_weight)
Batch size	64 (train and eval), dynamic padding to multiples of 8
Checkpoint	best epoch by validation weighted-F1 (load_best_model_at_end)
Precision	bf16 mixed precision
Weight decay	0.01
Gradient accumulation	8 micro-batches per optimizer step (effective batch 512)
Epochs	up to 40, early stopping with patience 5 on validation weighted-F1
Gradient checkpointing	off
Max sequence length	1024 tokens (truncation only)

The exact TrainingArguments are in training_args.json and the governing configuration extract in training_config.yml.

Evaluation

Selected checkpoint (split seed 3266123502)

split	accuracy	F1 weighted	F1 macro	precision macro	recall macro	ROC-AUC
validation	0.719	0.718	0.697	0.708	0.700	0.869
test	0.719	0.717	0.692	0.697	0.698	0.856

Split sensitivity across the 19 seeds

Test metrics of every run, sorted by weighted F1; the selected checkpoint is marked.

seed	accuracy	F1 weighted	F1 macro	precision macro	recall macro	ROC-AUC
1389303030	0.806	0.806	0.789	0.791	0.788	0.915
2928142788	0.770	0.767	0.743	0.761	0.730	0.888
1361883482	0.763	0.763	0.736	0.736	0.736	0.903
2786505123	0.748	0.750	0.724	0.717	0.734	0.891
4280088979	0.741	0.736	0.689	0.706	0.679	0.885
4203596092	0.734	0.732	0.677	0.681	0.674	0.908
477284336	0.734	0.730	0.682	0.692	0.677	0.890
1329496050	0.727	0.730	0.704	0.696	0.717	0.874
1245093080	0.719	0.721	0.694	0.688	0.701	0.888
3266123502	0.719	0.717	0.692	0.697	0.698	0.856	selected
3154447144	0.719	0.710	0.655	0.690	0.643	0.841
4054871397	0.712	0.704	0.647	0.666	0.639	0.862
2565555162	0.705	0.702	0.673	0.691	0.664	0.863
2078237541	0.712	0.700	0.661	0.683	0.667	0.866
239080115	0.719	0.697	0.605	0.654	0.613	0.817
709964709	0.691	0.689	0.641	0.647	0.638	0.826
107935903	0.683	0.681	0.629	0.632	0.628	0.824
228277762	0.662	0.663	0.627	0.627	0.628	0.823
3144693271	0.655	0.658	0.634	0.627	0.650	0.855

The full tables are in evaluation/.

Files

file	content
adapter_config.json	PEFT adapter configuration (base model, rank, target modules)
adapter_model.safetensors	DoRA adapter weights and the three-way classification head (base weights are not redistributed)
config.json	architecture, three-way head and label map
tokenizer.json	tokenizer
tokenizer_config.json	tokenizer settings (pad token, padding side)
evaluation/seeds_test_metrics.csv	test metrics of every split seed
evaluation/seeds_validation_metrics.csv	validation metrics of every split seed
training_args.json	the transformers.TrainingArguments of the selected run
training_config.yml	dataset split, hyperparameters, seed list and label map from the project config
README.md	this card

Reproducing

git clone https://github.com/andrea-dm/reddit-pulse && cd reddit-pulse
uv venv && uv pip install -e .
reddit run --model gemma2_2b --gpu 0     # every split seed, median selection, corpus labelling
reddit upload --model gemma2_2b          # this repository, from the selected checkpoint

The gold set (data/labelled.xlsx) and the subreddit corpus are not part of the repository; see the paper for the data-construction stages.

Citation

If you use this model, please cite the paper it was built for:

@article{delmonaco2026reddit,
  title   = {Reddit's `pulse' on {US} inflation: forecasting with large language models},
  author  = {Del Monaco, Andrea and Longo, Luigi and Marcucci, Juri and Tafani, Irene},
  journal = {Journal of Applied Econometrics},
  year    = {2026},
  note    = {forthcoming},
}

@techreport{delmonaco2026reddit_qef,
  title       = {Reddit's `pulse' on {US} inflation: forecasting with large language models},
  author      = {Del Monaco, Andrea and Longo, Luigi and Marcucci, Juri and Tafani, Irene},
  institution = {Banca d'Italia},
  series      = {Questioni di Economia e Finanza (Occasional Papers)},
  number      = {1028},
  year        = {2026},
  month       = jun,
  doi         = {10.32057/0.QEF.2026.1028},
}

License

The base model is released under the Gemma Terms of Use, which this adapter inherits; the terms are kept on Google's page and not redistributed here. The base repository ships no license file to carry along.

The views expressed in the paper are those of the authors and do not necessarily reflect those of the Bank of Italy, the Eurosystem, or the European Commission.