jackstev/hw2-distilbert-garment-type

DistilBERT Garment Type Classifier

Homework 2, Problem 3.

What it does

Reads a short product-style clothing description and predicts the garment type:

Label	Type
0	top
1	bottom
2	outerwear
3	dress
4	footwear

Data

• Source: leixiang25/24679-hw1-text-garments, revision 49d58b5a0942. A classmate wrote 100 original descriptions (~190 characters each) from retail product photos.
• Splits used as published: train = 70 originals + 1,289 noisy copies (character swaps/deletions, word swaps, WordNet synonyms), validation = 15 originals, test = 15 originals, 3 per class in each holdout. No original crosses splits.
• Preprocessing: DistilBERT uncased tokenizer, max length 64 tokens, dynamic padding. No other cleaning, so the model also sees the typo copies during training.

Training

• Base: distilbert-base-uncased, full fine-tune with a new 5-way head. Every weight is updated, which is what AutoGluon calls optim.peft = None; the partial options (bit_fit, norm_fit) were not needed with this much text.
• Up to 6 epochs, learning rate 5e-5 with linear decay and 10% warmup, batch size 16, weight decay 0.01, seed 24679, mixed precision on GPU.
• Early stopping on validation loss with patience 2. Kept epoch 1 of 3.
• Compute: Tesla T4, 35 s of training.

Results

Split	Accuracy	Macro F1	Errors
Validation (15)	1.000	1.000	0
Test (15)	1.000	1.000	0

Per-class on test:

Class	Precision	Recall	F1
top	1.00	1.00	1.00
bottom	1.00	1.00	1.00
outerwear	1.00	1.00	1.00
dress	1.00	1.00	1.00
footwear	1.00	1.00	1.00

Stress test: with the garment noun replaced by "item" in the 30 validation and test descriptions, accuracy is 0.833 (macro F1 0.792). The gap between this and the normal score shows how much the model relies on the noun itself. Confusion matrices for both are in the repo.

Limitations

• 15 test examples is very small; one error is 6.7 points of accuracy.
• The descriptions all follow the same listing template written by one person, and the garment noun nearly gives away the answer. Expect worse results on real, messier product listings.
• Only 5 broad types. Edge cases follow the author's labels (for example, a jumpsuit counts as a dress, a zip pullover as outerwear).
• The noisy copies are not independent examples.

Ethics

The texts are the dataset author's own fictional product descriptions with no brands or personal data. The model is not meant for anything beyond coursework.

How to use

from transformers import pipeline
clf = pipeline("text-classification", model="jackstev/hw2-distilbert-garment-type")
clf("Relaxed crewneck sweatshirt in heather gray fleece, with dropped shoulders and ribbed cuffs.")

License

Apache 2.0, the same as the DistilBERT base model. The dataset card assigns no license, so check with its author before reusing the data.

AI usage disclosure

I used Claude (Anthropic) to help write this notebook and draft this model card. I chose the dataset, ran the training, and checked the results and the card against the outputs.