ronke21/hebnli-neodictabert

NeoDictaBERT fine-tuned on HebNLI

dicta-il/neodictabert fine-tuned as a 3-class NLI classifier (entailment / neutral / contradiction) on HebNLI. It is one of the Strategy 1 denoisers of the HeRE paper: the premise is a Hebrew Wikipedia passage and the hypothesis is a verbalised Wikidata triple; the entailment probability decides whether the passage expresses the relation.

Training

AdamW, learning rate 2e-5, batch size 64, linear warm-up over 10% of steps, weight decay 0.01, maximum sequence length 250, 8,000 steps, evaluation every 200 steps; the checkpoint with the best validation macro-F1 was retained (step 5,600). Single run, seed 42.

Results

	HebNLI dev macro-F1	HebNLI test macro-F1	HeRE gold validation F1
NeoDictaBERT	0.861	0.878	0.771 (Strategy 1, template hypothesis); deployed on the full silver corpus as the encoder_nli__neodictabert signal

Labels

id2label: {"0": "היסק", "1": "סתירה", "2": "ניטרלי"}

Usage

from transformers import AutoTokenizer, AutoModelForSequenceClassification
tok = AutoTokenizer.from_pretrained("ronke21/hebnli-neodictabert")
model = AutoModelForSequenceClassification.from_pretrained("ronke21/hebnli-neodictabert", trust_remote_code=True)
enc = tok(premise, hypothesis, return_tensors="pt", truncation=True, max_length=250)
probs = model(**enc).logits.softmax(-1)

trust_remote_code=True is required (the checkpoint ships modeling_neobert.py).

License

The fine-tuned weights are released under the base model's license (cc-by-4.0). Paper resources: dataset https://huggingface.co/datasets/ronke21/HeRE, code https://github.com/Ronke21/HeRE.

Citation

Keinan, R., Cohen, A. D. N., and Tsarfaty, R. (2026). HeRE: A Novel Benchmark for Hebrew Relation Extraction via LLM-Guided Denoising of Knowledge Graph Alignments. In Proceedings of AACL 2026.