Hugging Face's logo

amang1802
/
Llama3.2-1B-summary-length-exp4

Text Generation
Transformers
Safetensors
llama
conversational
text-generation-inference
Inference Endpoints
Model card Files Community

Model Card for Model ID

  • Summary Length PPO experiment #2
  • No KL divergence in loss

Model Details

  • Dataset size: 1024
  • Epochs: 2
  • Batch Size: 4 * 8 (w / Gradient Accumulation)

Optimizer args: Torch AdamW default, except

  • LR = 0.0001

Outcomes

Only outputs one word "relationship"

Downloads last month
30
Safetensors
Model size
1.24B params
Tensor type
BF16
·
Inference Providers NEW
Text Generation
This model is not currently available via any of the supported Inference Providers.

Collection including amang1802/Llama3.2-1B-summary-length-exp4