GiovanniIacuzzo02/SemEval-2027-Task-1-subtrack-2a

RETECO Sub-track 2a: Conversational Dense Retrieval

A domain-aware dense bi-encoder for retrieving supporting passages from multi-turn conversations.

Overview

This repository contains a fine-tuned dense encoder for RETECO SemEval-2027 Task 1, Sub-track 2a (Conversational Retrieval). Given a target user turn, its preceding dialogue context, and a domain-specific passage collection, the system encodes the query and candidate passages into vectors that can be compared to rank potentially relevant evidence.

Conversational retrieval differs from ordinary single-turn search: a later turn can contain ellipsis, anaphora, or implicit references whose meaning depends on earlier dialogue. The query representation therefore needs to preserve the current information need together with the relevant conversation history.

The encoder is based on BAAI/bge-base-en-v1.5 and was trained with a contrastive retrieval objective. The project training pipeline supports BM25-mined hard negatives and in-batch negatives. The published encoder artifacts are intended to be used with the same pooling, normalization, and query-construction settings as the training configuration.

Scope. This repository provides the dense encoder component. It is not, by itself, a complete search service: corpus encoding, nearest-neighbour search, optional BM25 retrieval, rank fusion, and benchmark-output formatting belong to the surrounding retrieval pipeline.

Task definition

Item	Description
Benchmark	RETECO / RECOR, Track 2
Sub-track	2a — Conversational Retrieval
Input	Current target turn and dialogue history
Search space	The corresponding domain's document collection
Output	Ranked passage/document identifiers for each target turn
Official metric	Macro-averaged nDCG@10
Diagnostic metrics	Recall@K, MRR, per-domain and turn-depth breakdowns

See the official task definition and evaluation protocol.

Method

Architecture

flowchart TD
    A["Current target turn"] --> C["Query and context construction"]
    B["Conversation history"] --> C
    C --> D["Dense bi-encoder: BGE base English v1.5"]
    C --> E["BM25 retrieval"]
    D --> F["Ranked candidate lists"]
    E --> F
    F --> G["Reciprocal Rank Fusion (RRF)"]
    G --> H["Ranked passage IDs"]
    H --> I["nDCG@10 and retrieval diagnostics"]

Dense representation

The model starts from BGE base English v1.5. The project implementation supports configurable pooling and L2-normalized embeddings; the values in the training configuration used for the selected checkpoint should be treated as authoritative. When embeddings are L2-normalized, their dot product is equivalent to cosine similarity.

Contrastive training

The training pipeline optimizes query–positive-passage compatibility using a contrastive cross-entropy / InfoNCE-style objective. Negative examples can include in-batch negatives and BM25-mined hard negatives. Hard-negative mining is performed on the training portion to make negative candidates lexically plausible while avoiding intentional use of the held-out internal-validation conversations for training examples.

Conversation context

The query formatter is configurable. It can use the current query alone or combine it with dialogue history, subject to a token budget. For faithful reproduction, encode queries using the same formatting strategy and maximum sequence length that were used for training; do not assume that a raw follow-up sentence is a complete standalone query.

Loading the encoder

The example below assumes this repository contains a PEFT/LoRA adapter with adapter_config.json and adapter_model.safetensors. Replace the example repository ID with the exact ID of this model repository. If the repository instead contains full encoder weights, use the full-model loading path shown in its file list rather than the adapter-loading code below.

import torch
import torch.nn.functional as F
from peft import PeftModel
from transformers import AutoModel, AutoTokenizer

BASE_MODEL_ID = "BAAI/bge-base-en-v1.5"
ADAPTER_ID = "GiovanniIacuzzo02/SemEval-2027-Task-1-subtrack-2b"

# The tokenizer assets are included in the model repository.
tokenizer = AutoTokenizer.from_pretrained(ADAPTER_ID)
encoder = AutoModel.from_pretrained(BASE_MODEL_ID)
encoder = PeftModel.from_pretrained(encoder, ADAPTER_ID)

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
encoder.to(device)
encoder.eval()

@torch.inference_mode()
def encode(texts, max_length=256):
    batch = tokenizer(
        texts,
        padding=True,
        truncation=True,
        max_length=max_length,
        return_tensors="pt",
    ).to(device)
    outputs = encoder(**batch)

    # CLS pooling: use this only when it matches the checkpoint's training config.
    embeddings = outputs.last_hidden_state[:, 0]
    return F.normalize(embeddings, p=2, dim=-1)

# Supply the query already formatted according to the project's
# ContextAwareQueryFormatter, and passages as their indexed text.
query_text = "[formatted conversational query]"
passages = ["Passage text A", "Passage text B"]

query_embedding = encode([query_text])
passage_embeddings = encode(passages)
scores = query_embedding @ passage_embeddings.T
ranked_indices = scores[0].argsort(descending=True).tolist()

for rank, idx in enumerate(ranked_indices, start=1):
    print(rank, passages[idx], float(scores[0, idx]))

This is a minimal vector-encoding example, not a complete RETECO submission implementation. In particular, ensure that pooling_strategy, normalize_embeddings, query formatting, and token limits agree with the configuration used for the checkpoint. The full project pipeline is required to reproduce the competition's candidate retrieval, scoring, and output conventions.

Data and domains

The model is developed for the RECOR conversational retrieval data distributed as part of the official RETECO dataset. Track 2 covers these eleven domains:

• Biology
• Drones
• Earth Science
• Economics
• Hardware
• Law
• Medical Sciences
• Politics
• Psychology
• Robotics
• Sustainable Living

Use the official dataset and its train/dev split definitions. Do not use the hidden SemEval evaluation set for training or model selection.

Training and reproducibility

The project training code and configuration define the exact experiment. The principal stages are:

1. Load the RECOR conversations, domain corpus, and relevance judgments.
2. Split the available training conversations into train and internal-validation partitions.
3. Build context-aware queries and mine training hard negatives.
4. Optimize the bi-encoder with a contrastive objective.
5. Select the best checkpoint using internal validation and evaluate retrieval separately by domain.

For a reproducible experiment, record the checkpoint revision, seed, active domains, pooling strategy, normalization, query strategy, sequence lengths, negative-sampling settings, and evaluator/qrels revision.

Evaluation and results

The official retrieval metric is nDCG@10, macro-averaged over domains. Recall@K and MRR are useful diagnostics, but they do not replace the official score.

No official hidden-test score is claimed in this model card. Local experimental results should only be reported as a model score after they have been tied to this exact checkpoint, a named split, the full domain set, and a documented scorer version. Organizer-published BM25 reference results are available in the official baseline report; those baseline values are not results for this encoder.

Limitations

• Retrieval quality can vary substantially by domain and conversation depth.
• A query representation that drops relevant dialogue history can lose essential context.
• Dense retrieval requires encoding or indexing the target corpus; the adapter alone does not provide a document index.
• Any local validation score is specific to its split, corpus version, query formatter, and evaluation implementation.
• The model is an experimental research artifact and should be assessed on the intended corpus before operational use.

References

• RETECO — SemEval-2027 Task 1
• RETECO task definition
• RETECO evaluation protocol
• RETECO dataset on Hugging Face
• BGE base English v1.5 model card
• Ali et al. (2026), RECOR: Reasoning-focused Multi-turn Conversational Retrieval Benchmark, Findings of ACL 2026.