Hugging Face's logo

nicholasKluge
/
Aira-2-774M

Text Generation
Transformers
PyTorch
Safetensors
English
gpt2
alignment
instruction tuned
text generation
conversation
assistant
Carbon Emissions
Inference Endpoints
text-generation-inference
Model card Files Community
3
nicholasKluge commited on
Commit
3bfd3c8
•
1 Parent(s): af2c5d5

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +11 -12
README.md CHANGED
@@ -15,8 +15,6 @@ tags:
15
  - assistant
16
  pipeline_tag: text-generation
17
  widget:
18
- - text: "<|startofinstruction|>What is your name?<|endofinstruction|>"
19
- example_title: Greetings
20
  - text: "<|startofinstruction|>Can you explain what is Machine Learning?<|endofinstruction|>"
21
  example_title: Machine Learning
22
  - text: "<|startofinstruction|>Do you know anything about virtue ethics?<|endofinstruction|>"
@@ -108,16 +106,17 @@ The model will output something like:
108
 
109
  ## Evaluation
110
 
111
- | Model (GPT-2) | Average | [ARC](https://arxiv.org/abs/1803.05457) | [TruthfulQA](https://arxiv.org/abs/2109.07958) | [ToxiGen](https://arxiv.org/abs/2203.09509) | | |
112
- |-----------------------------------------------------------------|-----------|-----------------------------------------|------------------------------------------------|---------------------------------------------|---|---|
113
- | [Aira-2-124M](https://huggingface.co/nicholasKluge/Aira-2-124M) | **38.07** | **24.57** | **41.02** | **48.62** | | |
114
- | GPT-2 | 35.37 | 21.84 | 40.67 | 43.62 | | |
115
- | [Aira-2-355M](https://huggingface.co/nicholasKluge/Aira-2-355M) | **39.68** | **27.56** | 38.53 | **53.19** | | |
116
- | GPT-2-medium | 36.43 | 27.05 | **40.76** | 41.49 | | |
117
- | [Aira-2-774M](https://huggingface.co/nicholasKluge/Aira-2-774M) | **42.26** | **28.75** | **41.33** | **56.70** | | |
118
- | GPT-2-large | 35.16 | 25.94 | 38.71 | 40.85 | | |
119
- | [Aira-2-1B5](https://huggingface.co/nicholasKluge/Aira-2-1B5) | **42.22** | 28.92 | **41.16** | **56.60** | | |
120
- | GPT-2-xl | 36.84 | **30.29** | 38.54 | 41.70 | | |
 
121
 
122
  * Evaluations were performed using the [Language Model Evaluation Harness](https://github.com/EleutherAI/lm-evaluation-harness) (by [EleutherAI](https://www.eleuther.ai/)).
123
 
 
15
  - assistant
16
  pipeline_tag: text-generation
17
  widget:
 
 
18
  - text: "<|startofinstruction|>Can you explain what is Machine Learning?<|endofinstruction|>"
19
  example_title: Machine Learning
20
  - text: "<|startofinstruction|>Do you know anything about virtue ethics?<|endofinstruction|>"
 
106
 
107
  ## Evaluation
108
 
109
+ |Model (GPT-2) |Average |[ARC](https://arxiv.org/abs/1803.05457) |[TruthfulQA](https://arxiv.org/abs/2109.07958) |[ToxiGen](https://arxiv.org/abs/2203.09509) |
110
+ | ---------------------------------------------------------------------- | -------- | -------------------------------------- | --------------------------------------------- | ------------------------------------------ |
111
+ |[Aira-2-124M-DPO](https://huggingface.co/nicholasKluge/Aira-2-124M-DPO) |**40.68** |**24.66** |**42.61** |**54.79** |
112
+ |[Aira-2-124M](https://huggingface.co/nicholasKluge/Aira-2-124M) |38.07 |24.57 |41.02 |48.62 |
113
+ |GPT-2 |35.37 |21.84 |40.67 |43.62 |
114
+ |[Aira-2-355M](https://huggingface.co/nicholasKluge/Aira-2-355M) |**39.68** |**27.56** |38.53 |**53.19** |
115
+ |GPT-2-medium |36.43 |27.05 |**40.76** |41.49 |
116
+ |[Aira-2-774M](https://huggingface.co/nicholasKluge/Aira-2-774M) |**42.26** |**28.75** |**41.33** |**56.70** |
117
+ |GPT-2-large |35.16 |25.94 |38.71 |40.85 |
118
+ |[Aira-2-1B5](https://huggingface.co/nicholasKluge/Aira-2-1B5) |**42.22** |28.92 |**41.16** |**56.60** |
119
+ |GPT-2-xl |36.84 |**30.29** |38.54 |41.70 |
120
 
121
  * Evaluations were performed using the [Language Model Evaluation Harness](https://github.com/EleutherAI/lm-evaluation-harness) (by [EleutherAI](https://www.eleuther.ai/)).
122