File size: 694 Bytes
0552091 6ddf34d 0552091 6ddf34d 95efdc1 6818297 95efdc1 55aa619 ae0fc7b 55aa619 ae0fc7b 6ddf34d cae61c0 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 |
\n---
tags:
- Causal Language modeling
- CLM
model_index:
- name: MechDistilGPT2
results:
- task:
name: Causal Language modeling
type: Causal Language modeling
---
## MechDistilGPT2
This model is fine-tuned on text scraped from 100+ Mechanical/Automotive pdf books.
Base model is DistilGPT2(https://huggingface.co/gpt2) (the smallest version of GPT2)
## Fine-Tuning
* Default Training Args
* Epochs = 3
* Training set = 200k sentences
* Validation set = 40k sentences
## Framework versions
* Transformers 4.7.0.dev0
* Pytorch 1.8.1+cu111
* Datasets 1.6.2
* Tokenizers 0.10.2
## References
https://github.com/huggingface/notebooks/blob/master/examples/language_modeling.ipynb |