MechDistilGPT2

Model Details
Uses
Risks, Limitations and Biases
Training
Environmental Impact
How to Get Started With the Model

Model Details

Model Description: This model is fine-tuned on text scraped from 100+ Mechanical/Automotive pdf books.
Developed by: Ashwin
Model Type: Causal Language modeling
Language(s): English
License: [More Information Needed]
Parent Model: See the DistilGPT2model for more information about the Distilled-GPT2 base model.
Resources for more information:
- Research Paper
- GitHub Repo

Uses

Direct Use

The model can be used for tasks including topic classification, Causal Language modeling and text generation

Misuse and Out-of-scope Use

The model should not be used to intentionally create hostile or alienating environments for people. In addition, the model was not trained to be factual or true representations of people or events, and therefore using the model to generate such content is out-of-scope for the abilities of this model.

Risks, Limitations and Biases

CONTENT WARNING: Readers should be aware this section contains content that is disturbing, offensive, and can propagate historical and current stereotypes.

Significant research has explored bias and fairness issues with language models (see, e.g., Sheng et al. (2021) and Bender et al. (2021)).

Training

Training Data

This model is fine-tuned on text scraped from 100+ Mechanical/Automotive pdf books.

Training Procedure

Fine-Tuning

Default Training Args
Epochs = 3
Training set = 200k sentences
Validation set = 40k sentences

Framework versions

Transformers 4.7.0.dev0
Pytorch 1.8.1+cu111
Datasets 1.6.2
Tokenizers 0.10.2

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

Hardware Type: [More information needed]
Hours used: [More information needed]
Cloud Provider: [More information needed]
Compute Region: [More information needed"]
Carbon Emitted: [More information needed]

How to Get Started With the Model

from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("geralt/MechDistilGPT2")

model = AutoModelForCausalLM.from_pretrained("geralt/MechDistilGPT2")

geralt
/

MechDistilGPT2