cs1684models

Contents

FEVER - deberta-v3-base

• Path: fever/run1/
• Models: best_model/, best_model_calibrated/, final_model/
• Training logs: fever/run1/training_results.txt

IMDB - deberta-v3-base

• Path: imdb/run1/
• Models: best_model/, best_model_calibrated/, final_model/
• Training logs: imdb/run1/training_results.txt

JIGSAW - deberta-v3-base

• Path: jigsaw/run1/
• Models: best_model/, best_model_calibrated/, final_model/
• Training logs: jigsaw/run1/training_results.txt

Usage

Load Models

from models.baseline_models import SupervisedClassifier

# IMDb Sentiment Analysis
model = SupervisedClassifier(task_type="sentiment", num_labels=2, multilabel=False)
model.load_trained_model("limbo23/cs1684models", 
                        subfolder="imdb/run1/best_model_calibrated")

# FEVER Fact Verification
model = SupervisedClassifier(task_type="fact_verification", num_labels=3, multilabel=False)
model.load_trained_model("limbo23/cs1684models",
                        subfolder="fever/run1/best_model_calibrated")

# Jigsaw Toxicity Detection
model = SupervisedClassifier(task_type="toxicity", num_labels=2, multilabel=False)
model.load_trained_model("limbo23/cs1684models",
                        subfolder="jigsaw/run1/best_model_calibrated")

# Make predictions
result = model.predict_single("Your text here")
print(f"Label: {result['label']}, Confidence: {result['confidence']:.4f}")

Choose Model Version

Each dataset has three model versions:

• best_model_calibrated/ - Best validation performance with temperature scaling
• best_model/ - Best validation performance without calibration
• final_model/ - Model from final training epoch

# Use calibrated model
model.load_trained_model("limbo23/cs1684models", subfolder="imdb/run1/best_model_calibrated")

# Or use uncalibrated
model.load_trained_model("limbo23/cs1684models", subfolder="imdb/run1/best_model")

Datasets

Dataset	Task	Labels	Samples
IMDb	Sentiment Analysis	2 (negative/positive)	~25K
FEVER	Fact Verification	3 (refutes/nei/supports)	~145K
Jigsaw	Toxicity Detection	2 (toxic/non-toxic)	~160K

• IMDb: https://huggingface.co/datasets/stanfordnlp/imdb
• Jigsaw: https://www.kaggle.com/competitions/jigsaw-toxic-comment-classification-challenge
• FEVER: https://fever.ai/dataset/fever.html