aryanator/substantive-news-gate-v2

Substantive News Gate (v2)

aryanator/substantive-news-gate-v2 is a lightweight BERT-based classifier fine-tuned to detect whether a news headline represents substantive real-world news versus routine, procedural, opinion, or noise content — specifically in the Indian news context.

This model is designed to be used as a semantic gating and ranking signal, not as a standalone binary truth oracle.

What “Substantive” Means

A headline is considered substantive if it reports a concrete action, decision, or event with real-world consequences, such as:

Government approvals, launches, schemes, regulations

Court rulings, legal directives, enforcement actions

Infrastructure, policy, or economic decisions

Major geopolitical or security events

Implemented changes (not just discussion)

Non-substantive headlines typically include:

Meetings, reviews, discussions without outcomes

Forecasts, opinions, analysis pieces

Explainers, lists, lifestyle or promotional content

Fact-checks of viral claims

Routine administrative updates

Model Details

Base model: aryanator/bert_mlm_longnews (domain-adapted bert-base-uncased)

Task: Binary classification (SUBSTANTIVE vs NON_SUBSTANTIVE)

Training data: ~3.5k Indian news headlines labeled via LLM-assisted annotation

Fine-tuning: LoRA (merged for inference)

Input: News headline (text)

Output: Class logits → semantic score

⚠️ Important Usage Note

This model is not calibrated as an absolute probability classifier.

Instead, treat the output as a relative semantic score and interpret it using tiers.

Recommended Interpretation score < 0.40 → NON_SUBSTANTIVE 0.40 ≤ score < 0.48 → SOFT_SUBSTANTIVE (process / in-motion) score ≥ 0.48 → HARD_SUBSTANTIVE (executed action)

This reflects the natural uncertainty of headline-only semantic judgments.

Recommended Production Use

This model works best when combined with freshness or recency scoring.

Example:

if tier == "HARD_SUBSTANTIVE": importance = freshness * 1.0 elif tier == "SOFT_SUBSTANTIVE": importance = freshness * 0.4 else: importance = 0

This mirrors how real-world news systems rank content — by combining simple, interpretable signals.

Example Inference from transformers import AutoTokenizer, AutoModelForSequenceClassification import torch

tokenizer = AutoTokenizer.from_pretrained("aryanator/substantive-news-gate-v2") model = AutoModelForSequenceClassification.from_pretrained( "aryanator/substantive-news-gate-v2" )

headline = "UP Cabinet approves sops for semiconductor sector" inputs = tokenizer(headline, return_tensors="pt", truncation=True)

with torch.no_grad(): logits = model(**inputs).logits probs = torch.softmax(logits, dim=-1)

semantic_score = probs[0, 1].item()

Intended Use Cases

News feed ranking

Noise filtering / gating

Editorial triage

Signal-over-noise news platforms

Pre-filtering for downstream event clustering

Not Intended For

Absolute importance scoring

Standalone editorial judgment

Long-form article classification

High-stakes moderation decisions

Philosophy

Rather than attempting to predict abstract “importance,” this model focuses on a narrow, well-defined semantic question:

Does this headline describe something that actually happened or was decided?

This constraint makes the model:

More reliable

Easier to debug

Safer to use in production

License & Notes

Trained for experimental / research use

Performance depends on headline phrasing and source conventions

Designed to degrade gracefully when uncertain