TEMPS: temporal module

TEMPS (Temporal Embedding Model for Precise Search) adds a temporal score to any dense retriever. This repository holds the trained temporal module from TEMPS: Temporal Sentence Embeddings for Temporal Information Retrieval (Hassani, Romero, Bouzeghoub, Jacquelinet, EMNLP 2026).

• Code: https://github.com/aldebaran-care/TEMPS
• Weights: temps-weights.pth (PyTorch state_dict, fp32)

Model description

Dense retrievers match queries and passages on topic but not on time, so they return passages that are on-topic yet temporally wrong. TEMPS attaches to a frozen semantic retriever and scores how well two anchored texts (text + anchor date) align in time.

For a text and its anchor date, the module outputs a diagonal Gaussian over time. Its layers are:

• a frozen all-MiniLM-L6-v2 backbone (up to 512 tokens)
• a sinusoidal anchor-date encoder
• attention pooling
• mean and variance heads

Two texts are compared with the KL-based score

ts(1, 2) = 1 / (1 + KL(p_1 || p_2))

which is fused with the semantic retriever's cosine score after per-query min-max normalization:

s(1, 2) = (1 - alpha) * cos(E_1, E_2) + alpha * ts(1, 2)

alpha is set per benchmark on a held-out validation split: 0.6 for TimeQA, 0.8 for TempReason and 0.3 for TS-Retriever.

Training

No hand-labeled temporal data is used. Each anchored temporal expression is resolved to an inclusive day interval [a, b] and moment-matched to a Gaussian (mu = (a + b + 1) / 2, sigma^2 = (b - a + 1)^2 / 12). The pair label is 1 / (1 + KL(p_I1 || p_I2)), and the module is trained with a CoSENT ranking loss to reproduce that ordering.

• Data: 60M automatically generated tuples (55M synthetic, 5M from Wikipedia text in the Multilingual MLM Temporal Tagging Resources)
• Setup: 3 epochs, global batch size 2,048, lr 3e-4, weight decay 0.01, 5% linear warmup, max 512 tokens, fp16, seed 42; about 4 hours on 8×A100-80GB

Usage

The model class lives in the code repository. Clone it and run from its root:

git clone https://github.com/aldebaran-care/TEMPS && cd TEMPS
uv sync

import torch
from torch.distributions import Normal, kl_divergence
from huggingface_hub import hf_hub_download
from transformers import AutoTokenizer

from temporal_embeddings.model.gauss_model import GaussModel
from temporal_embeddings.utils.positional_encoding import positional_encoding

backbone = "sentence-transformers/all-MiniLM-L6-v2"
model = GaussModel(backbone).eval()
model.load_state_dict(torch.load(hf_hub_download("aldebaran-research/temps", "temps-weights.pth"), map_location="cpu"))
tokenizer = AutoTokenizer.from_pretrained(backbone)

@torch.inference_mode()
def encode(texts, anchor_dates):
    batch = tokenizer(texts, padding=True, truncation=True, max_length=512, return_tensors="pt")
    return model(**batch, dates=positional_encoding(anchor_dates))

query = encode(["Who led the company last spring?"], ["2021-11-09"])
docs = encode(["In April 2021 she was appointed CEO.", "He ran the firm from 1998 to 2003."],
              ["2021-11-09", "2021-11-09"])

# Temporal score 1 / (1 + KL(query || doc)), KL averaged over dimensions; higher = better match
kl = kl_divergence(Normal(query.mu, query.std), Normal(docs.mu, docs.std)).mean(dim=-1)
print(1 / (1 + kl))  # tensor([0.7331, 0.7130])

To reproduce the paper's evaluation, uv run python scripts/download_model.py saves this checkpoint where the evaluation configs expect it; see the repository README.

Results

On the held-out test splits (MRR):

Benchmark	Best system	MRR	Gain over backbone
TimeQA (alpha=0.6)	SFR-Embedding-Mistral + TEMPS	0.5360	+31.9%
TempReason (alpha=0.8)	SFR-Embedding-Mistral + SUTime control	0.6901	+11.5%
TS-Retriever (alpha=0.3)	SFR-Embedding-Mistral + TEMPS	0.8240	+9.3%

On TS-Retriever, E5-base-v2 + TEMPS lifts R@1 from 19.92 to 25.39 and P@1 from 58.63 to 69.14 over TSContriever. On TempReason, the rule-based SUTime control outperforms TEMPS; see the paper for the full tables.

Limitations

• English only.
• Covers expressions that can be anchored and resolved to an interval: absolute dates, deictic expressions and offsets, present references, seasons, durations and intervals. Discourse-level reasoning and event–event relations are out of scope.
• A passage of up to 512 tokens is reduced to a single Gaussian, which loses information on long or temporally mixed passages.
• Passages with no temporal expression were held out of training, so their Gaussians are uncalibrated; the semantic score is what ranks them.
• Results come from a single training run (seed 42).

License

Apache-2.0, inherited from sentence-transformers/all-MiniLM-L6-v2. The code repository is MIT.

Citation

@inproceedings{hassani-etal-2026-temps,
  title     = {{TEMPS}: Temporal Sentence Embeddings for Temporal Information Retrieval},
  author    = {Hassani, Mourad and Romero, Julien and Bouzeghoub, Amel and Jacquelinet, Christian},
  booktitle = {Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
  year      = {2026},
  publisher = {Association for Computational Linguistics}
}

Acknowledgments

This project was provided with computing and storage resources by GENCI at IDRIS thanks to the grant 2025-105442 on the supercomputer Jean Zay's A100 partition.