\n--- tags: - Causal Language modeling - CLM model_index: - name: MechDistilGPT2 results: - task: name: Causal Language modeling type: Causal Language modeling
MechDistilGPT2
This model is fine-tuned on text scraped from Mechanical/Automotive pdf books.
Base model is DistilGPT2(https://huggingface.co/gpt2) (the smallest version of GPT2)
Fine-Tuning
- Default Training Args
- Epochs = 3
- Training set = 200k sentences
- Validation set = 40k sentences
Framework versions
- Transformers 4.7.0.dev0
- Pytorch 1.8.1+cu111
- Datasets 1.6.2
- Tokenizers 0.10.2