You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

VeriSci Claim Verifier

DeBERTa v3 small sequence classifier for evidence-aware scientific claim verification over SUPPORTS, REFUTES, and NOT_ENOUGH_INFO.

Intended Use

Given a scientific claim and retrieved evidence passage, classify whether the evidence supports, refutes, or does not provide enough information. This model is not a medical device, not a substitute for peer review, and should not be used for clinical, legal, or public-policy decisions without expert review.

Training Data

Primary verifier data: allenai/scifact_entailment train split.
Hard NEI examples: andreiaalexa/scifact-relevance-pairs title/train not_relevant pairs.
Validation/test policy: train-overlapping validation claims removed, then remaining validation rows partitioned by normalized claim key. Derived test was not used for training or model selection.

Evaluation

Validation accuracy: 0.4464 Validation macro F1: 0.4349

Derived test accuracy: 0.4024 Derived test macro F1: 0.3887

See evaluation/eval_summary.json and prediction JSONL files for class metrics, confusion matrices, and limitations.

Limitations

The derived test split is small and originates from SciFact validation, not an official hidden test set.
The model only judges the provided evidence; retrieval failures can change the end-to-end answer.
Raw softmax confidence is not a calibrated probability.
Training data is SciFact-derived and subject to license/domain constraints.

Downloads last month: 55

Safetensors

Model size

0.1B params

Tensor type

F32