Clamifision-v1

Clamifision-v1 is a binary text classifier that determines whether input text is medical/health-related or not. It's built as a routing gate for BotMed, a family of ML/DL models for the medical field -- Clamifision decides whether incoming text should be routed to BotMed's specialized medical models.

Model Details

Model Description

Clamifision-v1 is a fine-tuned ModernBERT-base encoder with a binary sequence classification head, distinguishing medical/health-related text (label 1) from non-medical text (label 0). It was trained on a curated, balanced dataset aggregated from multiple public sources spanning both formal (PubMed, MedQuAD, clinical transcriptions, Wikipedia) and casual/conversational (real patient-asked questions, template-generated symptom queries) medical text, alongside a diverse non-medical corpus (news, product reviews, forum discussion, encyclopedia articles).

Developed by: Bindupautra Jyotibrat
Shared by: BJyotibrat
Model type: Binary text classification (fine-tuned encoder-only transformer, sequence classification head, 2 labels)
Language(s) (NLP): English
License: GPL-3.0
Finetuned from model: answerdotai/ModernBERT-base

Model Sources

Repository: Jyotibrat/BotMed-Clamifision
Demo: BotMed Official Website

Uses

Direct Use

Classifying a piece of input text (e.g. a user query) as medical/health-related or not, in isolation -- for example, as a content filter or pre-processing step in an application that needs to detect medical topics.

Downstream Use

Intended as a routing/gating component within the BotMed pipeline: text classified as medical is forwarded to BotMed's specialized downstream medical models; text classified as non-medical is not.

Out-of-Scope Use

Not intended for medical diagnosis, treatment recommendations, or any clinical decision-making -- it only classifies whether text is about medicine/health, it does not evaluate medical content for accuracy or safety. Not validated on languages other than English. Not intended as a sole safety filter for high-stakes applications without human review.

Bias, Risks, and Limitations

An earlier version of this model, trained only on formal sources (PubMed, MedQuAD, MTSamples, Wikipedia), learned to recognize formal/clinical writing style rather than the underlying "is this medical" concept -- it classified short, casual, first-person medical queries (e.g. "my chest hurts, should I worry?") as non-medical with high confidence, despite strong accuracy on formal, source-matched test data. Training data was subsequently expanded with real patient-asked questions and synthetic casual queries to address this, but formal sources (particularly PubMed) still make up the largest share of the medical training class by volume, so degraded performance on informal/conversational medical text remains a plausible residual risk.

Recommendations

Test this model specifically on short, casual, first-person, informally-phrased text before relying on it in production -- that is the failure mode most likely to still exist to some degree. Users (both direct and downstream) should be made aware of the risks, biases, and limitations described above.

How to Get Started with the Model

Use the code below to get started with the model.

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

tokenizer = AutoTokenizer.from_pretrained("BJyotibrat/Clamifision-v1")
model = AutoModelForSequenceClassification.from_pretrained("BJyotibrat/Clamifision-v1")

inputs = tokenizer("I've had a headache for 3 days", return_tensors="pt")
with torch.no_grad():
    logits = model(**inputs).logits
    pred = torch.argmax(logits, dim=-1).item()

print(model.config.id2label[pred])

Training Details

Training Data

~40,000 rows, balanced 50/50 medical/non-medical, curated via the BotMed Clamifision Data Pipeline. Sources:

Medical: PubMed abstracts, MedQuAD, MTSamples clinical transcriptions, Wikipedia medical articles, real patient-asked questions (curaihealth/medical_questions_pairs), template-generated casual symptom queries
Non-medical: AG News, 20 Newsgroups, Amazon/Yelp reviews, Wikipedia non-medical articles

Split 80/10/10 into train/val/test, stratified by label and source.

Training Procedure

Preprocessing

Tokenized with the ModernBERT tokenizer, truncated/padded to a max sequence length of 256 tokens.

Training Hyperparameters

Training regime: fp16 mixed precision
Learning rate: 2e-5
Batch size: 16 (train), 32 (eval)
Epochs: 3
Weight decay: 0.01
Attention implementation: SDPA

Evaluation

Testing Data, Factors & Metrics

Testing Data

Held-out stratified test split (10% of the curated dataset, ~4,258 rows), stratified by label and source.

Metrics

Accuracy, precision, recall, and F1 (binary classification).

Validation Results

Results

The following results are from an evaluation run on a version of this model trained before the casual-register training data (real patient-asked questions + synthetic casual queries) was added -- they should be re-confirmed against the final released checkpoint before being relied on.

Metric	Score
Accuracy	0.9988
Precision	0.9981
Recall	0.9995
F1	0.9988

Confusion matrix (test set, n=4,258):

Summary

Very high accuracy on held-out test data drawn from the same sources as training, but this reflects the model's ability to recognize source-level writing style rather than confirmed generalization to real-world, casual, out-of-distribution medical queries -- see Bias, Risks, and Limitations above.