lhbelfanti/drug-use-models

Drug Use Models (Spanish)

Binary classifiers for drug use detection in Spanish tweets

---

Drug Use Models (Spanish)

This repository contains all trained models for the Drug Use Modeler project: a binary text classifier that detects first-person drug use references in Spanish-language tweets. The models are trained and evaluated on the Drug Use Corpus (Spanish).

Overview

The experimental setup spans 48 configurations: 2 corpus variants × 3 preprocessing pipelines × 8 algorithms. The goal is to determine which combination best handles the ambiguity, irony, and obfuscation present in real-world social media drug discourse.

Corpus Variants

Variant	Description
pre-filtered-corpus	Cleaned dataset with noise and ambiguous tweets removed
raw-corpus	Full unfiltered dataset as collected from Twitter/X

Preprocessing Pipelines

Pipeline	Description
standard	Baseline tokenization and normalization
irony	Adds [IRONIA] token before sarcastic/ironic tweets
obfuscated	Named entities (substances, slang) masked via NER

Models

Model	Type	Representation
naive_bayes	Multinomial Naive Bayes	TF-IDF (5,000 features, bigrams)
logistic_regression	Logistic Regression	TF-IDF (5,000 features, bigrams)
svm	Linear SVC	TF-IDF (5,000 features, bigrams)
random_forest	Random Forest	TF-IDF (5,000 features, bigrams)
ffn	Feed-Forward Network (100→64→32→1)	Word2Vec 100-dim
cnn	TextCNN (filters 3/4/5)	Word2Vec 100-dim
rnn	BiLSTM (hidden=64, bidirectional)	Word2Vec 100-dim
bert_base	Fine-tuned BETO	dccuchile/bert-base-spanish-wwm-cased

---

Repository Structure

├── pre-filtered-corpus/
│   ├── naive_bayes/{standard,irony,obfuscated}/
│   │   ├── model.joblib
│   │   └── vectorizer.joblib
│   ├── logistic_regression/{standard,irony,obfuscated}/   (same structure)
│   ├── svm/{standard,irony,obfuscated}/                   (same structure)
│   ├── random_forest/{standard,irony,obfuscated}/         (same structure)
│   ├── ffn/{standard,irony,obfuscated}/
│   │   └── model.pt
│   ├── cnn/{standard,irony,obfuscated}/
│   │   └── model.pt
│   ├── rnn/{standard,irony,obfuscated}/
│   │   └── model.pt
│   ├── word2vec/{standard,irony,obfuscated}/
│   │   └── word2vec.model
│   └── bert_base/{standard,irony,obfuscated}/
│       ├── model/        (HuggingFace saved model)
│       └── tokenizer/    (HuggingFace tokenizer)
└── raw-corpus/           (same structure)

Note: For raw-corpus, Naive Bayes, SVM, and Random Forest models were not persisted — re-train inline with identical hyperparameters (seed=42) using the provided training splits.

---

Results

All metrics are macro-averaged over a balanced test set (225 NEGATIVE / 225 POSITIVE per split).

Pre-filtered Corpus

Standard

Model	Accuracy	Precision	Recall	F1
BERT (Base)	86.22%	86.25%	86.22%	86.22%
Naive Bayes	83.78%	83.79%	83.78%	83.78%
Logistic Regression	82.00%	82.14%	82.00%	81.98%
CNN	82.00%	82.14%	82.00%	81.98%
SVM	81.56%	81.63%	81.56%	81.54%
Random Forest	78.44%	78.54%	78.44%	78.43%
BiLSTM	78.44%	78.47%	78.44%	78.44%
FFN	76.89%	76.92%	76.89%	76.88%

Irony

Model	Accuracy	Precision	Recall	F1
BERT (Base)	85.33%	85.34%	85.33%	85.33%
Naive Bayes	83.33%	83.35%	83.33%	83.33%
Logistic Regression	82.00%	82.14%	82.00%	81.98%
SVM	81.78%	81.84%	81.78%	81.77%
CNN	80.67%	81.05%	80.67%	80.61%
Random Forest	79.78%	79.99%	79.78%	79.74%
BiLSTM	78.22%	78.33%	78.22%	78.20%
FFN	77.11%	77.14%	77.11%	77.11%

Obfuscated

Model	Accuracy	Precision	Recall	F1
Naive Bayes	83.11%	83.13%	83.11%	83.11%
Logistic Regression	81.78%	81.94%	81.78%	81.75%
SVM	81.56%	81.61%	81.56%	81.55%
CNN	80.89%	81.09%	80.89%	80.86%
BERT (Base)	80.44%	80.50%	80.44%	80.43%
Random Forest	79.56%	79.67%	79.56%	79.54%
BiLSTM	79.11%	79.34%	79.11%	79.07%
FFN	76.00%	76.10%	76.00%	75.98%

Raw Corpus

Standard

Model	Accuracy	Precision	Recall	F1
SVM	82.22%	82.48%	82.22%	82.19%
BERT (Base)	81.78%	81.78%	81.78%	81.78%
Naive Bayes	80.67%	80.74%	80.67%	80.66%
Logistic Regression	80.00%	80.35%	80.00%	79.94%
Random Forest	76.89%	76.97%	76.89%	76.87%
BiLSTM	76.44%	77.06%	76.44%	76.31%
CNN	76.22%	76.95%	76.22%	76.06%
FFN	76.00%	76.54%	76.00%	75.88%

Irony

Model	Accuracy	Precision	Recall	F1
BERT (Base)	82.44%	82.45%	82.44%	82.44%
SVM	82.44%	82.63%	82.44%	82.42%
Naive Bayes	80.44%	80.48%	80.44%	80.44%
Logistic Regression	80.22%	80.54%	80.22%	80.17%
BiLSTM	78.22%	78.22%	78.22%	78.22%
CNN	77.56%	78.09%	77.56%	77.45%
FFN	76.89%	77.15%	76.89%	76.83%
Random Forest	76.67%	76.82%	76.67%	76.63%

Obfuscated

Model	Accuracy	Precision	Recall	F1
SVM	83.11%	83.32%	83.11%	83.08%
BERT (Base)	82.00%	82.02%	82.00%	82.00%
Naive Bayes	81.11%	81.22%	81.11%	81.10%
Logistic Regression	80.67%	81.05%	80.67%	80.61%
Random Forest	77.78%	78.05%	77.78%	77.72%
CNN	76.44%	76.75%	76.44%	76.38%
BiLSTM	75.56%	76.15%	75.56%	75.42%
FFN	73.33%	74.65%	73.33%	72.97%

Best Models Summary

Corpus	Best Configuration	F1
Pre-filtered	BERT (Base) — Standard	86.22%
Raw	SVM — Obfuscated	83.08%

---

How to Use

Sklearn models (Naive Bayes, Logistic Regression, SVM, Random Forest)

import joblib
from huggingface_hub import hf_hub_download

# Download model and vectorizer
model_path = hf_hub_download(
    repo_id="lhbelfanti/drug-use-models",
    filename="pre-filtered-corpus/svm/standard/model.joblib"
)
vectorizer_path = hf_hub_download(
    repo_id="lhbelfanti/drug-use-models",
    filename="pre-filtered-corpus/svm/standard/vectorizer.joblib"
)

model = joblib.load(model_path)
vectorizer = joblib.load(vectorizer_path)

texts = ["me tomé una línea antes de salir"]
X = vectorizer.transform(texts)
print(model.predict(X))  # [1] → POSITIVE

BERT (BETO)

from transformers import AutoTokenizer, AutoModelForSequenceClassification
from huggingface_hub import snapshot_download
import torch

# Download model and tokenizer
model_dir = snapshot_download(
    repo_id="lhbelfanti/drug-use-models",
    allow_patterns="pre-filtered-corpus/bert_base/standard/**"
)

tokenizer = AutoTokenizer.from_pretrained(f"{model_dir}/pre-filtered-corpus/bert_base/standard/tokenizer")
model = AutoModelForSequenceClassification.from_pretrained(f"{model_dir}/pre-filtered-corpus/bert_base/standard/model")
model.eval()

text = "me tomé una línea antes de salir"
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=128)
with torch.no_grad():
    logits = model(**inputs).logits
pred = torch.argmax(logits, dim=1).item()
print("POSITIVE" if pred == 1 else "NEGATIVE")

---

Training Details

• Dataset: lhbelfanti/drug-use-corpus
• Split: 80% train / 20% test (stratified, seed=42)
• TF-IDF: max_features=5000, ngram_range=(1,2)
• Word2Vec: 100 dimensions, window=5, min_count=1, trained in-domain
• BERT base: dccuchile/bert-base-spanish-wwm-cased, fine-tuned 3 epochs, lr=2e-5, batch=16
• Framework: scikit-learn 1.5, PyTorch 2.2.2, Transformers 4.40.2

---

Related Resources

• Dataset: lhbelfanti/drug-use-corpus
• Code: github.com/lhbelfanti/drug-use-modeler

---

License

MIT