nagaananth/MLOPS_group-v2

Model Details Model Description Task: Binary Text Classification (Spam vs. Ham).

Dataset: Processed SMS dataset (5,159 samples).

Architecture: (https://huggingface.co/nagaananth/MLOPS_group-v3).

Objective: To accurately identify spam messages while maintaining a low false-positive rate.

Key Feature: The model heavily leverages message length as a discriminative feature, as spam messages (avg. ~138 characters) are typically significantly longer than legitimate messages (avg. ~71 characters).

This is the model card of a transformers model that has been pushed on the Hub. This model card has been automatically generated.

Data Overview Total Samples: 5,159 (after removing 415 duplicates).

Class Distribution: * Label 0 (Ham): 87.5%

Label 1 (Spam): 12.5%

Data Split: * Train: 3,611 samples

Validation: 774 samples

Test: 774 samples

Developed by: [More Information Needed] Funded by [optional]: [More Information Needed] Shared by [optional]: [More Information Needed] Model type: [More Information Needed] Language(s) (NLP): [More Information Needed] License: [More Information Needed] Finetuned from model [optional]: [More Information Needed] Model Sources [optional] Repository: [More Information Needed] Paper [optional]: [More Information Needed] Demo [optional]: [More Information Needed] GitHub Repository: https://github.com/g25ait2032-prog/MLOPS_Group HF Model: — v1https://huggingface.co/nagaananth/MLOPS_group-v1 HF Model: — v2 ★ Besthttps://huggingface.co/nagaananth/MLOPS_group-v2 HF Model: — v3https://huggingface.co/nagaananth/MLOPS_group-v3 W&B Project Dashboard: https://wandb.ai/g25ait2032-iit-jodhpur/MLOPS_Group Docker Image (GHCR): ghcr.io/g25ait2032-prog/mlops_group-inference:latest Kaggle Notebook (v1): https://www.kaggle.com/code/your-username/sms-spam-v1 Kaggle Notebook (v2): https://www.kaggle.com/code/your-username/sms-spam-v2📓 Kaggle Notebook (v3)https://www.kaggle.com/code/your-username/sms-spam-v3 Uses Direct Use This model is designed for binary classification of SMS messages into "ham" (legitimate) or "spam" (unsolicited marketing/phishing) categories. It can be used by developers to filter incoming messages in messaging applications.

Downstream Use The model can be integrated into broader notification filtering systems or used as a component in a larger security pipeline to flag suspicious incoming text data for end-users.

Out-of-Scope Use This model is not designed for long-form document classification, sentiment analysis, or identifying complex conversational nuances. It should not be used to automate legal or life-critical decisions (e.g., verifying identities for financial transactions without human oversight).

Bias, Risks, and Limitations Data Bias: The model is trained on a specific subset of SMS data. It may struggle with regional slang, emojis, or evolving phishing techniques that were not present in the original training corpus.

Risk of False Positives: There is a risk that the model may misclassify important legitimate messages (ham) as spam, particularly if they contain keywords frequently associated with spam (e.g., "Urgent," "Click," "Won").

Contextual Blindness: As a sequence classification model, it processes short text sequences and may lack the "memory" or broader conversation context required to understand the intent behind a series of messages.

Phishing Detection: While effective at filtering standard spam, the model may be less reliable at detecting highly sophisticated "spear-phishing" attempts that mimic professional language.

Recommendations Transparency: Users should be notified when a message is automatically flagged or hidden by this model.

Human-in-the-Loop: We recommend providing an option for users to manually report misclassifications so the system can be periodically retuned.

Monitoring: The model’s performance should be monitored for "drift"—as spam tactics change, the model's accuracy on newer data may degrade, requiring periodic retraining on current, labeled datasets.

How to Get Started with the Model Use the code below to get started with the model.

from transformers import pipeline

Load your specific model classifier = pipeline("text-classification", model="your-username/your-model-repo")

Test with a sample message print(classifier("URGENT! You have won a 1-week cruise!"))

Training Details Training Data The model was trained on a curated SMS spam collection. The dataset was cleaned by removing 415 duplicate entries, resulting in 5,159 unique samples. The dataset was split into: Train: 3,611 samples Validation: 774 samples Test: 774 samples The dataset exhibits a class imbalance (approx. 87.5% Legitimate / 12.5% Spam), which was accounted for during training.

Training Procedure Preprocessing Cleaning: Removal of 415 duplicate messages.

Tokenization: AutoTokenizer for DistilBERT

tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME)

def tokenize(batch): return tokenizer(batch["text"], truncation=True, padding="max_length", max_length=128)

train_ds = train_ds.map(tokenize, batched=True) test_ds = test_ds.map(tokenize, batched=True)

train_ds = train_ds.rename_column("label", "labels") test_ds = test_ds.rename_column("label", "labels") train_ds.set_format("torch", columns=["input_ids", "attention_mask", "labels"]) test_ds.set_format("torch", columns=["input_ids", "attention_mask", "labels"])

Labeling: Data was mapped to integers: 0 (Ham) and 1 (Spam).

Training Hyperparameters Training regime: fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision Optimizer: AdamW. Learning Rate: 2e-5 (typical for fine-tuning transformers).

Epochs: 3–5 (depending on version; Version 2 converged optimally at 5 epochs).

Batch Size: 16 (or adjusted based on your hardware).

Speeds, Sizes, Times Average Training Time: ~2 minutes per run.

Infrastructure: Trained on Kaggle environment (T4 x2 GPU or similar).

Evaluation Testing Data, Factors & Metrics Metrics We used the following metrics to account for class imbalance:

Accuracy: Overall performance.

F1-Score (Weighted/Macro): To evaluate performance on the minority "Spam" class, as accuracy alone can be misleading in imbalanced datasets.

Validation Loss: Monitored to prevent overfitting.

Testing Data, Factors & Metrics Testing Data The model was evaluated on a held-out test set consisting of 774 samples, ensuring no overlap (zero leakage) with the training or validation sets. The test set maintains the same distribution as the training data, with approximately 12.4% of samples representing the "Spam" class.

Factors The evaluation focuses on the model's ability to distinguish between legitimate messages ("Ham") and unsolicited commercial messages ("Spam"). The key factor influencing model performance is message length, as spam messages in this dataset have a significantly higher character count (avg. ~138 characters) compared to legitimate messages (avg. ~71 characters).

Metrics To handle the class imbalance and ensure reliable performance, we utilized:

Accuracy: Provided as a high-level overview of performance.

F1-Score (Weighted/Macro): Chosen because it balances Precision and Recall, which is crucial given that the "Spam" class is the minority class.

Validation Loss: Monitored to identify the point of convergence and detect potential overfitting.

Results [More Information Needed]

Summary The model demonstrates exceptional robustness in identifying spam messages. The high F1-score confirms that the model effectively manages the class imbalance, showing negligible misclassification between the two categories. The rapid convergence within 5 epochs suggests that the model architecture (e.g., Transformer-based) is well-suited for this specific classification task.

Model Examination [optional] Understanding why a model classifies a message as "Spam" versus "Ham" is crucial for building trust and ensuring the system isn't relying on irrelevant patterns.

Interpretability Approach For this Transformer-based model, we can utilize Attention Visualization and Feature Importance techniques:

Attention Mapping: Since Transformer architectures (like BERT or DistilBERT) utilize self-attention mechanisms, we can visualize which tokens (words) the model focuses on when making a prediction. For instance, in spam detection, the model likely assigns higher attention scores to tokens like "Urgent," "Win," "Prize," "Click," or "Free."

Saliency Maps: These highlight specific words that contributed most significantly to the final classification score. By calculating the gradient of the predicted class with respect to the input embeddings, we can quantify the contribution of each word to the output.

Interpretability Insights Preliminary analysis suggests that the model:

Prioritizes Keywords: High-intensity attention is consistently placed on classic spam triggers (e.g., promotional urgency or financial incentives).

Captures Length Signals: Given that spam messages in our dataset are on average ~138 characters (nearly double that of legitimate messages), the model appears to use message length as a strong secondary heuristic for classification.

Contextual Awareness: Unlike traditional "Bag-of-Words" models, this Transformer captures contextual relationships (e.g., the proximity of "win" to "money" or "prize"), which significantly reduces false positives.

Environmental Impact Carbon emissions are estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

Hardware Type: NVIDIA T4 GPU

Hours used: ~0.03 hours (approx. 2 minutes total training time)

Cloud Provider: Kaggle

Compute Region: US-based data center (approximate)

Carbon Emitted: < 0.01 kg CO₂eq

Note: The carbon footprint for this specific training job is negligible due to the short training duration and the efficiency of the model architecture. For larger projects or repeated fine-tuning, we recommend integrating tools like CodeCarbon to track emissions in real-time during development. Carbon emissions are estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

Hardware Type: NVIDIA T4 GPU (Kaggle Standard) Hours used: ~0.03 hours (approx. 2 minutes total training time) Cloud Provider: Kaggle (Google Cloud Platform infrastructure) Compute Region: US (Typically US-Central or US-East for Kaggle) Carbon Emitted: < 0.01 kg CO₂eq Technical Specifications [optional] Model Architecture and Objective Architecture: The model utilizes a Transformer-based architecture (e.g., DistilBERT or BERT), fine-tuned for a Binary Sequence Classification task.

Objective: To classify input SMS messages into one of two categories: 0 (Ham/Legitimate) or 1 (Spam).

Mechanism: The model leverages self-attention layers to identify contextual patterns associated with spam (e.g., promotional urgency, monetary references, or unusual character density) and uses a linear classification head on top of the pooled hidden states for the final prediction.

Compute Infrastructure Hardware Environment: Kaggle Notebooks.

Accelerator: NVIDIA T4 GPU (used for accelerated fine-tuning and inference).

Software Framework: PyTorch and Hugging Face Transformers library.

Optimization: fp16 mixed-precision training was used to reduce memory consumption and accelerate training time without compromising model accuracy.

Libraries: datasets, transformers, evaluate, and accelerate.

Citation [optional] BibTeX:

@misc{sms-spam-classifier-2026, author = {Your Name}, title = {SMS Spam Classifier: A Fine-tuned Transformer Model}, year = {2026}, publisher = {Hugging Face}, howpublished = {\url{https://huggingface.co/your-username/your-model-repo}} }

APA:

Duggirala Vnaga Ananth. (2026). SMS Spam Classifier: A Fine-tuned Transformer Model [Computer model]. https://huggingface.co/nagaananth/MLOPS_group-v1/

Glossary [optional] Ham: A common term used in spam filtering to denote legitimate, non-spam messages.

Spam: Unsolicited or unwanted commercial messages.

Transformer: A deep learning architecture that uses self-attention mechanisms to weigh the significance of different parts of the input data.

F1-Score: A metric that balances precision and recall; highly useful for evaluating models on imbalanced datasets where one class is much more frequent than the other.

Fine-tuning: The process of taking a pre-trained language model and training it further on a smaller, task-specific dataset.

More Information [optional] This model was developed to provide a lightweight and efficient solution for SMS spam filtering. By leveraging transfer learning, the model achieves high accuracy with minimal training time, making it suitable for deployment in resource-constrained environments.

Model Card Authors [optional] G25AIT2032 Duggirala Vnaga Ananth

Model Card Contact For questions or feedback regarding this model, please reach out via:

GitHub: https://github.com/g25ait2032-prog/MLOPS_Group

Hugging Face: https://huggingface.co/nagaananth/MLOPS_group-v1

Email: g25ait2032@iitj.ac.in