UPC-HLE Cross-Encoder Models

Welcome to the UPC-HLE organization on HuggingFace!
This repository hosts the cross-encoder models developed for our submission to SemEval-2025 Task 7: Multilingual and Cross-lingual Fact-Checked Claim Retrieval. Our approach combines dense Text Embedding Models (TEMs) with a neural Cross-Encoder re-ranking stage to retrieve fact-checked claims for social-media posts in a multilingual setting. The dataset is a subset of MultiClaim Dataset.

🔍 Organization Overview

The SemEval-2025 Task 7 challenge requires retrieving previously fact-checked claims relevant to user queries (social-media posts) across multiple languages. Our solution uses a two-step pipeline:

1. Initial Retrieval
   Encode the query (post) and all fact-check documents with a bi-encoder TEM to get dense embeddings. Retrieve the top 100 candidates by cosine similarity.

2. Cross-Encoder Re-Ranking
   Fine-tune a transformer-based Cross-Encoder (Jina Reranker v2) on triplets of (post, candidate, label) using hard-negative sampling. Re-score the 100 candidates to produce the final ranked list.

This yields strong improvements over lexical baselines and standalone embeddings, especially in cross-lingual retrieval.

📦 Models in This Organization

These are the best performing models in the final solution for the shared task. The bi-encoder retrieval is performed using Snowflake/snowflake-arctic-embed-l-v2.0 and, afterwards, the re-ranking is done using the models available in this repository, which use Jina V2 as base Cross Encoder.

Languages Available

• English (eng)
• French (fra)
• German (deu)
• Portuguese (por)
• Thai (tha)
• Malay (msa)
• Arabic (ara)
• Polish (pol)
• Turkish (tur)
• Crosslingual

Thanks to its XLM-RoBERTa backbone, the model can encode sentences in any of these languages into the same 768-dimensional vector space. You can therefore use the same model instance to compute embeddings and cosine-similarity scores for queries and fact-check candidates across all test‐set languages, without loading separate language-specific checkpoints.

🚀 Usage

Install the 🤗 Sentence Transformers library and then load any model as follows:

from sentence_transformers import CrossEncoder
rerank_model = CrossEncoder(reranker_model_name, num_labels=1, max_length=1024, trust_remote_code=True)
ranked_vals = rerank_model.rank(str_candidate, ls_factchecks, show_progress_bar=False)

📈 Performance

All scores are reported as success@10 (the fraction of queries where at least one correct fact-check appears in the top 10).

Model Monolingual Cross-lingual mTEM-CE-Eng-SF 0.91 0.78 mTEM-CE-E5 0.91 0.75 mTEM-CE-Eng-E5 0.90 0.76 mTEM-LF-CE-Eng-E5 0.89 0.75 These results demonstrate the benefit of CE re-ranking over the baseline TEM alone ​

🛠 Training Details

Triplet Mining: – Retrieve 100 candidates per post by cosine similarity. – Label the gold fact-check as positive; sample up to 4 hard negatives where their TEM score ≥ 0.95 × positive score.

Hyperparameters (for mTEM-CE-Eng-SF): – Batch size: 32 – Epochs: 5 – Learning rate: 2 × 10⁻⁵ – Warmup steps: 10% of total – Candidates for re-ranking: 100

📚 Citation

If you use these models or our results, please cite:

@inproceedings{semeval2025task7upchle, title={UPC-HLE at SemEval-2025 Task 7: Multilingual Fact-Checked Claim Retrieval with Text Embedding Models and Cross-Encoder Re-Ranking}, author={Alberto Becerra-Tome, Agustín Conesa}, booktitle = {Proceedings of the 19th International Workshop on Semantic Evaluation}, series = {SemEval 2025}, year = {2025}, address = {Vienna, Austria}, month = {July}, pages = {} doi= {} }

📄 License

All models and code are released under the MIT License. See LICENSE for details.

Happy fact-checking!