grepac-e5-small

intfloat/multilingual-e5-small fine-tuned on CPU to find the right article of the EU Common Agricultural Policy (CAP) regulations for a question. It powers grePAC. Code and training pipeline: github.com/stonarini/grepac.

Usage

Same query: / passage: prefixes as the base model.

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("stonarini/grepac-e5-small")
q = model.encode(["query: When will I receive my CAP payments?"], normalize_embeddings=True)
p = model.encode(["passage: Article 44 Payment to beneficiaries\n..."], normalize_embeddings=True)
print(q @ p.T)

Training

The training data is generated, with no hand-written examples:

• a local LLM (Mistral 7B Instruct v0.3) writes questions for each paragraph, plus rule-based queries from titles and definitions
• each question gets a hard negative, a paragraph from another article that looks similar

~3000 training examples over +200 articles. MultipleNegativesRankingLoss, 2 epochs, batch 32, learning rate 2e-5, 73 minutes on a 4-core laptop CPU.

Evaluation

60 hand-written and hand-checked questions, each with the articles that answer it. Scored per article.

	right article first	in top 3	in top 5	MRR@10
base model	0.667	0.833	0.867	0.750
grepac-e5-small	0.717	0.900	0.900	0.797

The test is too small to be conclusive: 11 questions improved and 8 got worse. Real searches from the site will be the next test set.

License

MIT, like the base model. Regulation texts: EUR-Lex, © European Union, reused under the Commission's reuse policy.