FinxGemma-Lite

FinxGemma-Lite is google/embeddinggemma-2 fine-tuned on raw card-transaction merchant descriptors: short, noisy strings such as SQ *SHAKE SHACK, AMZN MKTP US*Z58J27N20 and WAL-MART #2791. It is meant for retrieval-based transaction categorisation: embed a new transaction, find the most similar descriptor you've already labelled, and reuse its category.

It comes out of a two-part study of whether embedding similarity can be trusted for this job.

• Code, notebooks and every result: GitHub: Surya-sourav/FinxGemma-lite.
• The full write-up: REPORT.md.

An honest summary. FinxGemma-Lite matches the base model; it doesn't beat it. For merchants it has never seen, it gets 51.1% against the base model's 50.9%, which is within noise. What did work in the same study is a small calibrated trust model on top of retrieval, which predicts whether each answer is right. It raised the AUROC for telling right from wrong answers from 0.73 to 0.90. The trust model is described in the GitHub repo; it is not part of this checkpoint.

Model details

Base model	google/embeddinggemma-2, revision 914f7f8, text encoder only
Parameters	271M
Output	768-dimensional, L2-normalised embeddings; compare them with cosine similarity
Max sequence length	32 tokens, enough for any descriptor plus its prompt
Prompts	query for the new transaction, document for the labelled descriptors (the base model's retrieval prompts)
Precision	float32 (the base model card warns against float16)
Licence	Apache 2.0, the same as the base model

How to use

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("suryeah/FinxGemma-Lite")

# descriptors you've already categorised (in practice: your whole history)
labelled = {
    "SQ *BLUE BOTTLE COFFEE": "restaurants",
    "SHELL OIL 57444": "service stations",
    "STAPLES 00103010": "office supplies",
}

new = model.encode(["TST* LA COLOMBE COFFEE"], prompt_name="query")
known = model.encode(list(labelled), prompt_name="document")
scores = model.similarity(new, known)[0]
best = int(scores.argmax())
print(list(labelled.values())[best], float(scores[best]))

The similarity score is not a confidence: for well-known merchants it couldn't separate right answers from wrong ones (AUROC 0.40). If you need to know when to trust an answer, put a calibrated classifier on top. The repo shows one that uses neighbour agreement, merchant history and a TF-IDF cross-check.

Intended use

• Intended: research and prototyping of retrieval-based categorisation for card and bank transaction descriptors, and nearest-neighbour lookup of merchants you've already seen.
• Not intended: automated financial decisions without a confidence layer and human review. It is also not a general-purpose sentence embedding model; use the base model for that.

Training

• Data: 60,000 triplets mined from State of Delaware employee credit-card transactions dated before 2025-01-16 (data.delaware.gov, public domain). There are three kinds, each with a hard negative mined with the base model:
  • 30,185 noise pairs: a descriptor with a processor prefix (SQ *, TST* , PAYPAL *), a store number, or cut to 12 characters, against the clean descriptor;
  • 27,988 category pairs: a descriptor against the closest descriptor from a different merchant with the same category;
  • 1,827 variant pairs: two spellings of the same merchant.
• Held out: 5% of merchants were kept out of training entirely.
• Recipe: CachedMultipleNegativesRankingLoss (batch 64, computed 32 texts at a time), learning rate 2e-5, 1 epoch, 10% warm-up, float32, seed 42. Training prompts match the inference prompts. It took 40 minutes on one T4, with a peak of 6.5 GB of GPU memory.

Evaluation

The test set is 20,000 transactions from 4 Sep 2025 to 30 Jun 2026. Each was matched against the 154,361 distinct descriptors seen before that date, and the category was copied from the closest one. Accuracy is the share of right categories, split by how often the transaction's merchant had appeared before:

merchant seen before	queries	base model	char n-gram TF-IDF	FinxGemma-Lite
never	1,960	50.9%	43.1%	51.1%
once	335	92.2%	91.9%	92.5%
2–5 times	848	92.1%	92.0%	92.5%
6+ times	16,857	95.9%	95.4%	96.0%
all	20,000	91.2%	90.1%	91.4%

• Other measures:
  • Seen merchants are matched to the right merchant 99.5% of the time (base: 99.1%).
  • New-merchant detection from low similarity reaches an AUROC of 0.974 (base: 0.966).
• The trust model, from the same study:
  • It separates right from wrong answers far better than the cosine: AUROC 0.90 vs 0.73, and 0.81 vs 0.40 for merchants seen 6+ times.
  • Answering only the 80% of queries it's most confident about is 98.2% accurate, against 95.7% for the base model with calibrated cosine.

The hypotheses were fixed before the run. Each counts as supported only if the 95% paired-bootstrap interval of the improvement excludes zero (1,000 resamples):

	hypothesis	result	difference [95% CI]
RQ1	fine-tuning helps never-seen merchants without hurting known ones	not supported	new +0.002 [-0.013, +0.015]; known +0.001 [+0.000, +0.002]
RQ2	mined hard negatives beat in-batch negatives (never-seen)	not supported	+0.003 [-0.008, +0.013]
RQ3	trust model AUROC beats calibrated cosine, overall and 6+	supported	all +0.173 [+0.159, +0.186]; 6+ +0.407 [+0.382, +0.431]
RQ4	lower AURC and higher accuracy at 80% / 90% coverage (P vs B2)	supported	AURC -0.039 [-0.041, -0.034]; @80 +0.025 [+0.023, +0.028]; @90 +0.013 [+0.010, +0.015]

Limitations

• One dataset. It was trained and tested only on US public-sector purchase cards. Other card programmes, countries and languages may behave differently.
• Noisy labels. Categories are card-network merchant category descriptions, which are noisy and sometimes ambiguous. A hotel and its restaurant share one name, and 42% of the remaining errors are of this kind.
• Merchant names are inferred. The data has no merchant ID, so "same merchant" is decided by a simple rule.
• One training run. The planned extra seeds turned out identical, because sentence-transformers 6.1 seeds its batch sampler with a fixed 0, so the variance between seeds wasn't measured. The source notebooks now fix this.

Citation

@misc{parida2026finxgemmalite,
  author = {Surya Parida},
  title  = {FinxGemma-Lite: does an embedding model know when it is wrong about a card transaction?},
  year   = {2026},
  url    = {https://github.com/Surya-sourav/FinxGemma-lite}
}