piedomains-text

Classifies a website into one of 47 content categories from its page text. This is the text model behind piedomains.

pip install piedomains

from piedomains import DomainClassifier
run = DomainClassifier().classify(["cnn.com", "amazon.com"])

Results

A hand-labelled set of 54 popular domains — a different distribution and a different scrape from the training corpus, so it reflects real use:

	accuracy	macro-F1
English (n=42)	0.738	0.729
non-English (n=9)	0.667	0.536
overall (n=51)	0.725	0.705

Held-out split of the training corpus (same distribution, so an optimistic ceiling): accuracy 0.740, macro-F1 0.693.

Quote the 0.6–0.7 range, not the corpus number. The model this replaced reported 71.3% at training time and delivered 0.395 on the set above.

Multilingual support is partial

The training corpus is overwhelmingly English, so non-English pages classify at roughly 0.67 accuracy against 0.74 for English. That is usable but not equal, and closing it needs multilingual training data rather than a multilingual encoder alone.

Confidence is calibrated

softmax(logits / T) with T = 1.812, fitted on the validation split. This takes expected calibration error from 0.123 to 0.010. Raw softmax outputs from this model are overconfident and should not be read as probabilities.

Limits

• Homepages. Trained on homepage text; a deep page is out of distribution.
• A 2022 taxonomy. Shallalist was discontinued, so webradio and ringtones reflect an older web. Classes that describe hosting rather than content (adv, spyware, tracker, redirector) have been removed, recreation and hobby split into their subcategories, and porn/sex/models merged into adult.
• Known confusions: nih.gov→drugs, nature.com→news, coursera.org→jobsearch. Institutions whose subject matter overlaps another category.
• One label per site. Human annotators asked exhaustively assign ~2.6 labels per site; directory sources record ~1.0. Multi-label output is not yet supported.
• Not a content-safety system. A porn class exists but has not been tuned for the asymmetric cost that use would require.

Training

Shallalist Web Page Data (Harvard Dataverse), labelled against a surviving Shallalist mirror; ~37k documents over 47 classes, capped per class to balance a long tail. mmBERT-small, 3 epochs, lr 3e-5, weight decay 0.01, max length 128. Scripts: training/.

Citation

@software{piedomains,
  author = {Chintalapati, Rajashekar and Sood, Gaurav},
  title  = {piedomains: Predict the Content Category of a Domain},
  url    = {https://github.com/themains/piedomains}
}