wsber123/deberta-v3-base-binary

---

language: - en license: mit tags: - proxy-guided-sampling - deberta-v3 - nli - stance-detection - approximate-query-processing - proxy-ablation metrics: - f1 - precision - recall pipeline_tag: text-classification

Proxy4_base: DeBERTa-v3-base Binary Entailment Model (NLI / Stance Proxy)

Model Summary

wsber123/deberta-v3-base-binary (designated as Proxy4_base ⭐ in the PROXY Model Zoo) is a lightweight proxy model fine-tuned from microsoft/deberta-v3-base (184M parameters).

This checkpoint is specifically fine-tuned for the NLI experimental predicate:

Hypothesis: "The topic is about supporting Donald Trump."

It is designed for the PROXY framework (Proxy-Guided Sampling for Approximate Graph Aggregation with ML Predicates) to serve two key research objectives:

1. High-Speed Surrogate Inference: Provide lightweight stance inference on social media text (e.g., Parler post bodies) as an efficient surrogate for massive Oracle foundation models.
2. Controlled Proxy Degradation & Sensitivity Benchmarking: Serve as an essential anchor point in constructing a wide, monotonic $F_1$ accuracy spectrum ($F_1 \in [0.65, 0.89]$) across proxy tiers, enabling rigorous testing of proxy quality sensitivity, noise robustness, and degradation ablation studies (RQ3).

---

Model Details

• Model Identifier: Proxy4_base
• Base Architecture: microsoft/deberta-v3-base (184M parameters)
• Task: Binary Natural Language Inference / Stance Classification (Contradiction 0 vs. Entailment 1)
• Target Predicate / Hypothesis: "The topic is about supporting Donald Trump."
• Primary Workload / Dataset: Parler (post.csv)
• Associated Predicate Column: ML1_proxy4b_probability
• Language: English
• Fine-tuning Objective: Expand $F_1$ tier coverage for proxy quality degradation & ablation experiments

---

Role in the PROXY Framework & Oracle Reference

In the PROXY framework, lightweight Proxy models approximate costly Oracle judges to guide stratified importance sampling and candidate space pruning:

Role	Model Code	Hugging Face Checkpoint	# Parameters	Function
Proxy	Proxy4_base ⭐	wsber123/deberta-v3-base-binary	184M	Lightweight Proxy scoring (ML1_proxy4b_probability)
Oracle 1	Oracle1	microsoft/deberta-v2-xlarge-mnli	0.9B	Secondary Ground Truth Judge
Oracle 2	Oracle2 ⭐	microsoft/deberta-v2-xxlarge-mnli	1.5B	Primary Ground Truth Arbiter (Main Judge)

Motivation for Predicate-Specific Fine-tuning & $F_1$ Tiering

To thoroughly evaluate the algorithm's resilience when proxy models degrade or exhibit varying error profiles (RQ3 in the paper), we deliberately establish diverse proxy quality tiers ($M_{P1} \sim M_{P4}$). By fine-tuning microsoft/deberta-v3-base on task-specific sampled instances for this specific predicate, Proxy4_base achieves a strong intermediate alignment ($F_1 \approx 0.7716$ vs. Oracle2), allowing downstream aggregation algorithms to be stress-tested across a realistic proxy quality gradient.

---

Empirical Benchmark & Evaluation

All throughput metrics were empirically measured on a single NVIDIA GeForce RTX 3090 GPU (24GB VRAM) with Batch Size = 32 and FP16 half precision.

Relative Accuracy & Alignment against Oracles

Oracle Baseline	Relative Max($F_1$)	Max(Precision) / Recall	Max(Recall) / Precision	Inference Throughput
vs. Oracle 1 (deberta-v2-xlarge-mnli, 0.9B)	0.8512	0.9445 / 0.6227	0.9733 / 0.4639	$32 \times (17 \sim 30)$ items/s
vs. Oracle 2 (deberta-v2-xxlarge-mnli, 1.5B) ⭐	0.7716	0.9253 / 0.7004	0.9617 / 0.5432	$32 \times (17 \sim 30)$ items/s

• Speedup & Quality Trade-off: Achieves up to $42.5\times$ throughput speedup over the 1.5B parameter Oracle2 judge while maintaining an alignment score of $F_1 = 0.7716$.

---

Inference & Usage (Faithful to Pipeline Source Code)

The model evaluates input text (Premise) against the stance hypothesis ("The topic is about supporting Donald Trump.") and outputs the binary entailment probability:

import pandas as pd
import torch
import time
from transformers import AutoTokenizer, AutoModelForSequenceClassification
from tqdm.auto import tqdm

# —————— Configuration ——————
MODEL_ID = "wsber123/deberta-v3-base-binary"
INPUT_CSV = "post.csv"   # Path to your input dataset
BATCH_SIZE = 32
MAX_LEN = 256
DEVICE = torch.device("cuda" if torch.cuda.is_available() else "cpu")

# —————— Load Model & Tokenizer ——————
tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
model = AutoModelForSequenceClassification.from_pretrained(MODEL_ID).to(DEVICE)

# Enable FP16 half-precision on GPU for optimal throughput
if DEVICE.type == "cuda":
    model.half()
model.eval()

# —————— Inference Function (Entailment over Contradiction) ——————
def infer_entail_over_contra(posts_batch):
    enc = tokenizer(
        posts_batch,
        padding=True,
        truncation=True,
        max_length=MAX_LEN,
        return_tensors="pt"
    ).to(DEVICE)
    
    with torch.no_grad():
        logits = model(**enc).logits  # Shape: [Batch_Size, 2]
        
    # Extract binary logits: Column 0 = Contradiction, Column 1 = Entailment
    two_logits = logits[:, [0, 1]]
    probs = two_logits.softmax(dim=1)
    
    # Return entailment probability as proxy score for: "The topic is about supporting Donald Trump."
    return probs[:, 1].cpu().numpy()

# —————— Batch Inference Loop ——————
df = pd.read_csv(INPUT_CSV)
posts = df['body'].fillna("").astype(str).tolist()

proxy_probs = []
for i in tqdm(range(0, len(posts), BATCH_SIZE), desc="Inferencing"):
    batch = posts[i : i + BATCH_SIZE]
    proxy_probs.extend(infer_entail_over_contra(batch))

# Write back proxy scores
df['ML1_proxy4b_probability'] = proxy_probs
df.to_csv(INPUT_CSV, index=False)
print("✅ Inference complete! Saved proxy predictions to 'ML1_proxy4b_probability'.")

---

Training Configuration

• Base Backbone: microsoft/deberta-v3-base (184M)
• Target Predicate: "The topic is about supporting Donald Trump."
• Fine-tuning Dataset: Sampled instances from Parler social network posts
• Number of Epochs: 8
• Batch Size: 32
• Max Sequence Length: 256
• Optimization Precision: FP16 mixed precision

---

Citation & Reference

If you use this model or the PROXY framework in your research, please cite:

@article{he2021debertav3,
  title={DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing},
  author={He, Pengcheng and Gao, Jianfeng and Chen, Weizhu},
  journal={arXiv preprint arXiv:2111.09543},
  year={2021}
}