YAML Metadata Warning: empty or missing yaml metadata in repo card (https://huggingface.co/docs/hub/model-cards#model-card-metadata)

Lora finetuning on Wikipedia-10, applying counter factual data augmentation (CDA)

Dataset: Wikipedia-10
Target modules = ["query", "value"]

{
    "epoch": 2.8509064991759097,
    "total_flos": 3.390811584548659e+16,
    "train_loss": 1.3963899144530296,
    "train_runtime": 5715.6632,
    "train_samples": 22449,
    "train_samples_per_second": 11.197,
    "train_steps_per_second": 0.35
}

Training script: https://github.com/ao9000/bias-bench/blob/main/experiments/run_clm.py

Downloads last month: 27

Safetensors

Model size

124M params

Tensor type

BF16

Inference Providers NEW

Text Generation

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support