ccm/24-679-distilbert-haiku

DistilBERT for Haiku Perspective Classification

This model classifies the perspective expressed in an English haiku. It adapts distilbert-base-uncased with AutoGluon MultiModal's norm_fit method, updating a small subset of the encoder's parameters and a classification head. It was developed for the Fall 2026 offering of 24-679 at Carnegie Mellon University to demonstrate parameter-efficient fine-tuning on a small text dataset.

Label	Meaning
0	AI or machine perspective
1	Human perspective

These labels describe perspective, not authorship. A human can write from a machine's perspective, and an AI can write from a human's perspective. The model does not determine who wrote a poem or whether a student used AI.

How the model was adapted

We used optim.peft = "norm_fit". In the DistilBERT encoder, this trains normalization parameters and bias parameters while keeping the other pretrained weights frozen. The complete classification head is also trainable in this setup. This allows the encoder's representations to adapt while updating less than one tenth of one percent of the model's parameters. See the AutoGluon description of norm_fit.

Parameter count	Value
Total	66,364,418
Trainable	62,978
Frozen	66,301,440
Trainable share	0.095%

The counts and fine-tuning mode are recorded in automm/model_notes.json. The repository contains the complete AutoMM prediction bundle: preprocessing objects, tokenizer, encoder, and classification head. Load it with MultiModalPredictor, as shown below.

Training data

The model uses ccm/2026-24679-text-dataset, a collection of paired poems from a course survey. Each retained response supplied a human-perspective poem and an AI-or-machine-perspective poem. Labels come from the requested survey perspective; they are not independent expert judgments of the text.

Split	Original poems	Augmented variants	Total
Training	30	117	147
Validation	6	0	6
Test	8	0	8

Original survey responses were split before augmentation, keeping both poems from a response together. Augmented variants remain in training with their parent poem. The grouping is by survey response and does not establish that every response came from a different person.

Stored training variants use character swaps, character deletions, within-line word swaps, and WordNet synonym replacement. AutoMM's additional random text augmentation was disabled with model.hf_text.text_trivial_aug_maxscale = 0.0. The input feature is haiku; label is the target. Label descriptions and provenance fields are excluded from the model's input.

Training configuration

Setting	Value
Encoder	distilbert-base-uncased, CLS pooling
Adaptation	norm_fit plus a trainable classification head
Optimizer and loss	AdamW; cross-entropy
Learning rate	2e-5, with layerwise decay of 0.9
Schedule	Cosine decay; 10% warmup
Weight decay	0.01
Maximum input length	128 subword tokens
Effective batch size	8
Epochs	5
Validation	At the end of each epoch
Checkpoint selection	Best validation accuracy; one checkpoint retained
Random seed	24679
Precision	Mixed FP16 on one GPU

The export records Python 3.13.15, AutoGluon MultiModal 1.6.1, PyTorch 2.11.0, and Transformers 5.14.1. The full configuration is in automm/config.yaml, with package versions in automm/requirements.txt. A dataset commit was not recorded in the export, so the dataset link alone does not guarantee exact reproduction if its contents change.

Evaluation and limitations

The saved bundle records a best validation accuracy of 66.7%, corresponding to 4 of 6 validation poems. This score was used to select the checkpoint. It is recorded in automm/assets.json. A held-out test score is not recorded in the exported metadata, so none is claimed here.

With only six validation poems and eight test poems, one changed prediction moves accuracy by 16.7 or 12.5 percentage points, respectively. These splits support a classroom demonstration, but do not establish general performance on new writers, topics, or styles.

The model may rely on explicit machine or human vocabulary, misunderstand figurative language, and struggle with ambiguous or mixed perspectives. Augmentation can alter meaning even when a label is retained. Longer inputs are truncated, and performance outside English haiku has not been established. Use this model for teaching and exploration; it is unsuitable for authorship detection or decisions about students.

Load and predict

Use a fresh environment matching the recorded Python and package versions. These commands download the complete AutoMM bundle at the artifact revision documented by this card:

python -m pip install --upgrade pip huggingface_hub
hf download ccm/24-679-distilbert-haiku --revision 1735063e9952b01b193ac2a32497e15c4f4a3aae --include "automm/*" --local-dir haiku_model
python -m pip install --use-pep517 -r haiku_model/automm/requirements.txt

If repository access requires authentication, run hf auth login before downloading. --use-pep517 avoids the legacy seqeval metadata-build issue encountered during installation. Then run this Python example. The example poem is an inference input, not a labeled evaluation example.

import pandas as pd
from autogluon.multimodal import MultiModalPredictor


def main():
    # Restore the saved preprocessing, encoder, and classification head together.
    predictor = MultiModalPredictor.load("haiku_model/automm")
    predictor.set_num_gpus(0)  # CPU inference; use 1 for an available CUDA GPU.

    poems = pd.DataFrame({
        "haiku": ["Rain taps on the glass\nI hold a warm cup of tea\nClouds drift past my door"]
    })
    label_names = {0: "AI/machine perspective", 1: "Human perspective"}
    labels = predictor.predict(poems).astype(int)
    results = poems.assign(label=labels, perspective=labels.map(label_names))
    print(results.to_string(index=False))


if __name__ == "__main__":
    main()

The exported format uses AutoMM loading rather than the Transformers pipeline() API. For an application, load the predictor once and reuse it for successive requests. The repository also includes an optional HaikuClassifier wrapper for a Python backend. AutoMM restores serialized Python objects; load only artifacts and revisions you trust.

Licensing

No license has been declared for this fine-tuned repository or its course dataset. This card does not assign one. Consult the base model's terms and the dataset's permissions before reuse.