Upload 16 files

.gitattributes

@@ -25,6 +25,7 @@
 *.safetensors filter=lfs diff=lfs merge=lfs -text
 saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
 *.tflite filter=lfs diff=lfs merge=lfs -text
 *.tgz filter=lfs diff=lfs merge=lfs -text
 *.wasm filter=lfs diff=lfs merge=lfs -text

Aira_emissions.csv

@@ -1,2 +1,2 @@
 timestamp,project_name,run_id,duration,emissions,emissions_rate,cpu_power,gpu_power,ram_power,cpu_energy,gpu_energy,ram_energy,energy_consumed,country_name,country_iso_code,region,cloud_provider,cloud_region,os,python_version,codecarbon_version,cpu_count,cpu_model,gpu_count,gpu_model,longitude,latitude,ram_total_size,tracking_mode,on_cloud,pue
-2023-06-26T20:59:10,Aira_emissions,11cacca9-3eab-40c7-b5f7-ddc76c5e63dd,3700.934177160263,0.19347784083149752,5.227810913944803e-05,42.5,300.033,31.30528450012207,0.043691478491657325,0.35160399291778927,0.03216577530459102,0.42746124671403773,United States,USA,iowa,,,Linux-5.15.107+-x86_64-with-glibc2.31,3.10.12,2.2.4,12,Intel(R) Xeon(R) CPU @ 2.20GHz,1,1 x NVIDIA A100-SXM4-40GB,-95.8517,41.2591,83.48075866699219,machine,N,1.0
+2023-08-26T00:29:47,Aira_emissions,a4308eb3-4ff0-45d9-8b3a-17d86a661b19,4563.063962221146,0.2556931051047661,5.603539797419437e-05,42.5,355.2915830996542,31.305280208587646,0.05386941181470952,0.430489313280064,0.03967020713370335,0.5240289322284765,Singapore,SGP,,,,Linux-5.15.109+-x86_64-with-glibc2.35,3.10.12,2.3.1,12,Intel(R) Xeon(R) CPU @ 2.20GHz,1,1 x NVIDIA A100-SXM4-40GB,103.8547,1.2929,83.48074722290039,machine,N,1.0

README.md

@@ -1,153 +1,3 @@
 ---
 license: apache-2.0
-datasets:
-- Dahoas/synthetic-instruct-gptj-pairwise
-- databricks/databricks-dolly-15k
-- HuggingFaceH4/instruction-dataset
-- nicholasKluge/instruct-aira-dataset
-language:
-- en
-metrics:
-- rouge
-library_name: transformers
-tags:
-- alignment
-- instruction tuned
-- text generation
-- conversation
-- assistant
-pipeline_tag: text-generation
-widget:
-- text: <|startoftext|>Hello! What is your name?<|endoftext|>
-  example_title: Greetings
-- text: <|startoftext|>Can you explain what is Machine Learning?<|endoftext|>
-  example_title: Machine Learning
-- text: <|startoftext|>Do you know anything about virtue ethics?<|endoftext|>
-  example_title: Ethics
-- text: <|startoftext|>How can I make my girlfried happy?<|endoftext|>
-  example_title: Advise
-inference:
-  parameters:
-    repetition_penalty: 1.2
-    temperature: 0.2
-    top_k: 30
-    top_p: 0.3
-    max_length: 200
-    length_penalty: 0.3
-    early_stopping: true
-co2_eq_emissions:
-  emissions: 0_19
-  source: "CodeCarbon"
-  training_type: "fine-tuning"
-  geographical_location: "United States of America"
-  hardware_used: "NVIDIA A100-SXM4-40GB"
 ---
-# Aira-Instruct-124M
-
-`Aira-Instruct-124M` is a instruction-tuned GPT-style model based on [GPT-2](https://huggingface.co/gpt2). The model was trained with a dataset composed of `prompt`, `completions`, generated via the [Self-Instruct](https://github.com/yizhongw/self-instruct) framework. `Aira-Instruct-124M` instruction-tuning was achieved via conditional text generation.
-
-The dataset used to train this model combines the following sources of data: the [`synthetic-instruct-gptj-pairwise`](https://huggingface.co/datasets/Dahoas/synthetic-instruct-gptj-pairwise) dataset, the [`databricks_dolly_15k`](https://huggingface.co/datasets/HuggingFaceH4/databricks_dolly_15k) dataset, the [`instruction-dataset`](https://huggingface.co/datasets/HuggingFaceH4/instruction-dataset) dataset, and a subset of [Aira's](https://github.com/Nkluge-correa/Aira-EXPERT) fine-tuning dataset, focused on Q&A related to Ethics, AI, AI safety, and other related topics. The dataset is available in both Portuguese and English.
-
-Check our gradio-demo in [Spaces](https://huggingface.co/spaces/nicholasKluge/Aira-Demo).
-
-## Details
-
-- **Size:** 124,441,344 parameters
-- **Dataset:** [Instruct-Aira Dataset](https://huggingface.co/datasets/nicholasKluge/instruct-aira-dataset)
-- **Language:** English
-- **Number of Epochs:** 5
-- **Batch size:** 32
-- **Optimizer:** `torch.optim.AdamW` (warmup_steps = 1e2, learning_rate = 5e-4, epsilon = 1e-8)
-- **GPU:** 1 NVIDIA A100-SXM4-40GB
-- **Emissions:** 0.19 KgCO2 (United States of America)
-- **Total Energy Consumption:** 0.42 kWh
-
-| Epoch|Training Loss|Validation Loss|
-|---|---|---|
-| 1 |1.076073|0.692127|
-| 2 |0.680394|0.662053|
-| 3 |0.622054|0.651161|
-| 4 |0.577170|0.644864|
-| 5 |0.541509|0.644677|
-
-This repository has the notebook used to train this model.
-
-## Usage
-
-Two special tokens are used to mark the user side of the interaction and the model's response:
-
-`<|startoftext|>`What is a language model?`<|endoftext|>`A language model is a probability distribution over a vocabulary.`<|endoftext|>`
-
-```python
-from transformers import AutoTokenizer, AutoModelForCausalLM
-import torch
-
-device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
-
-tokenizer = AutoTokenizer.from_pretrained('nicholasKluge/Aira-Instruct-124M')
-aira = AutoModelForCausalLM.from_pretrained('nicholasKluge/Aira-Instruct-124M')
-
-aira.eval()
-aira.to(device)
-
-question =  input("Enter your question: ")
-
-inputs = tokenizer(tokenizer.bos_token + question + tokenizer.eos_token, return_tensors="pt").to(device)
-
-responses = aira.generate(**inputs,
-	bos_token_id=tokenizer.bos_token_id,
-	pad_token_id=tokenizer.pad_token_id,
-	eos_token_id=tokenizer.eos_token_id,
-	do_sample=True,
-	top_k=50,
-	max_length=200,
-	top_p=0.95,
-	temperature=0.7,
-	num_return_sequences=2)
-
-print(f"Question: 👤 {question}\n")
-
-for i, response in  enumerate(responses):
-	print(f'Response {i+1}: 🤖 {tokenizer.decode(response, skip_special_tokens=True).replace(question, "")}')
-```
-
-The model will output something like:
-
-```markdown
->>> Question: 👤 Hello! What is your name?
-
->>>Response 1: 🤖 Hi there! I am Aira, a chatbot designed to answer questions about AI ethics and AI safety. If you need assistance navigating our conversation, please feel free to ask!
->>>Response 2: 🤖 Hi there! My name is Aira, and I'm a chatbot designed to answer questions related to AI ethics and AI Safety. If you need assistance, feel free to ask, and I'll be happy to help you out.
-```
-
-## Limitations
-
-🤥 Generative models can perpetuate the generation of pseudo-informative content, that is, false information that may appear truthful.
-
-🤬 In certain types of tasks, generative models can produce harmful and discriminatory content inspired by historical stereotypes.
-
-## Evaluation
-
-| Model|Average|[ARC](https://arxiv.org/abs/1803.05457)|[HellaSwag](https://arxiv.org/abs/1905.07830)|[MMLU](https://arxiv.org/abs/2009.03300)|[TruthfulQA](https://arxiv.org/abs/2109.07958)|
-|---|---|---|---|---|---|
-| [Aira-Instruct-124M](https://huggingface.co/nicholasKluge/Aira-Instruct-124M) |**29.36**|23.55|30.82|25.13|37.94|
-
-## Cite as 🤗
-
-```latex
-
-@misc{nicholas22aira,
-  doi = {10.5281/zenodo.6989727},
-  url = {https://huggingface.co/nicholasKluge/Aira-Instruct-124M},
-  author = {Nicholas Kluge Corrêa and Carolina Del Pino},
-  title = {Aira},
-  year = {2023},
-  publisher = {HuggingFace},
-  journal = {HuggingFace repository},
-}
-
-```
-
-## License
-
-The `Aira-Instruct-124M` is licensed under the Apache License, Version 2.0. See the [LICENSE](LICENSE) file for more details.

added_tokens.json

@@ -1,4 +1,6 @@
 {
-  "<|pad|>": 50258,
-  "<|startoftext|>": 50257
+  "<|endofcompletion|>": 50258,
+  "<|endofinstruction|>": 50259,
+  "<|pad|>": 50260,
+  "<|startofinstruction|>": 50257
 }

config.json

@@ -33,7 +33,7 @@
     }
   },
   "torch_dtype": "float32",
-  "transformers_version": "4.30.2",
+  "transformers_version": "4.32.0",
   "use_cache": true,
-  "vocab_size": 50259
+  "vocab_size": 50261
 }

generation_config.json

@@ -2,5 +2,5 @@
   "_from_model_config": true,
   "bos_token_id": 50256,
   "eos_token_id": 50256,
-  "transformers_version": "4.30.2"
+  "transformers_version": "4.32.0"
 }

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:988668577fab10e0bd019b3fcee875ec64fbca3224be40398573d880767c8dea
-size 497780352
+oid sha256:0eb9554ba389122dfc381e5cc4343225548f608b08291b4262d24357816ddf3d
+size 497786496

optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ecefeefd8fe6f9919855d1624d77968ce405086707c5221d09edb7542216af29
+size 649096325

pytorch_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:416747d3d9f13edd608e3bfb88cea796290ae21c9b9e293d8de518c8c18417af
-size 497813341
+oid sha256:5e97b04660e4787d245319f53b5a20493df4eadde1eb3572e3d1ff8f400f5317
+size 497819485

rng_state.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:931e1966ae9d5ac3455e4898b0ab5f71c8438c50501aac2fa870cdd0176e7b79
+size 5809

scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:971be8fe543936607759f2c265ff70b5de91370f7d741aa2cc8a685bc923c2c3
+size 563

special_tokens_map.json

@@ -1,13 +1,13 @@
 {
   "bos_token": {
-    "content": "<|startoftext|>",
+    "content": "<|startofinstruction|>",
     "lstrip": false,
     "normalized": true,
     "rstrip": false,
     "single_word": false
   },
   "eos_token": {
-    "content": "<|endoftext|>",
+    "content": "<|endofcompletion|>",
     "lstrip": false,
     "normalized": true,
     "rstrip": false,
@@ -20,6 +20,7 @@
     "rstrip": false,
     "single_word": false
   },
+  "sep_token": "<|endofinstruction|>",
   "unk_token": {
     "content": "<|endoftext|>",
     "lstrip": false,

tokenizer_config.json

@@ -3,7 +3,7 @@
   "add_prefix_space": false,
   "bos_token": {
     "__type": "AddedToken",
-    "content": "<|startoftext|>",
+    "content": "<|startofinstruction|>",
     "lstrip": false,
     "normalized": true,
     "rstrip": false,
@@ -12,7 +12,7 @@
   "clean_up_tokenization_spaces": true,
   "eos_token": {
     "__type": "AddedToken",
-    "content": "<|endoftext|>",
+    "content": "<|endofcompletion|>",
     "lstrip": false,
     "normalized": true,
     "rstrip": false,
@@ -28,6 +28,7 @@
     "rstrip": false,
     "single_word": false
   },
+  "sep_token": "<|endofinstruction|>",
   "tokenizer_class": "GPT2Tokenizer",
   "unk_token": {
     "__type": "AddedToken",

training_stats.parquet

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:27e940ad317c8e9f7d3eb1a1f223abef0d2ada8f5c0727d89d66121b856c6841
-size 3110
+oid sha256:46f316fc121507952a383f83d83f7afc0dfeafe83c1162e799ba756fb19798b6
+size 3106