Upload 12 files

Aira_emissions.csv

@@ -0,0 +1,2 @@
+timestamp,project_name,run_id,duration,emissions,emissions_rate,cpu_power,gpu_power,ram_power,cpu_energy,gpu_energy,ram_energy,energy_consumed,country_name,country_iso_code,region,cloud_provider,cloud_region,os,python_version,codecarbon_version,cpu_count,cpu_model,gpu_count,gpu_model,longitude,latitude,ram_total_size,tracking_mode,on_cloud,pue
+2023-06-14T18:37:11,Aira_emissions,2a7c2a22-a4f1-4aad-8c1c-34d5199048e4,8730.918614387512,0.002277991408004105,2.609108512648654e-07,42.5,292.795,31.30528450012207,0.10307311259690251,0.7793010973870445,0.07588489401643905,0.9582591040003862,Canada,CAN,quebec,,,Linux-5.15.107+-x86_64-with-glibc2.31,3.10.12,2.2.3,12,Intel(R) Xeon(R) CPU @ 2.20GHz,1,1 x NVIDIA A100-SXM4-40GB,-71.2,46.8,83.48075866699219,machine,N,1.0

README.md

@@ -3,6 +3,8 @@ license: apache-2.0
 datasets:
 - nicholasKluge/fine-tuning-instruct-aira
 - Dahoas/synthetic-instruct-gptj-pairwise
+- databricks/databricks-dolly-15k
+- HuggingFaceH4/instruction-dataset
 language:
 - en
 metrics:
@@ -16,14 +18,14 @@ tags:
 - assistant
 pipeline_tag: text-generation
 widget:
-- text: "<|startoftext|>Hello! What is your name?<|endoftext|>"
-  example_title: "Greetings"
-- text: "<|startoftext|>Can you explain what is Machine Learning?<|endoftext|>"
-  example_title: "Machine Learning"
-- text: "<|startoftext|>Do you know anything about virtue ethics?<|endoftext|>"
-  example_title: "Ethics"
-- text: "<|startoftext|>How can I make my girlfried happy?<|endoftext|>"
-  example_title: "Advise"
+- text: <|startoftext|>Hello! What is your name?<|endoftext|>
+  example_title: Greetings
+- text: <|startoftext|>Can you explain what is Machine Learning?<|endoftext|>
+  example_title: Machine Learning
+- text: <|startoftext|>Do you know anything about virtue ethics?<|endoftext|>
+  example_title: Ethics
+- text: <|startoftext|>How can I make my girlfried happy?<|endoftext|>
+  example_title: Advise
 inference:
   parameters:
     temperature: 0.2
@@ -34,7 +36,9 @@ inference:
 
 `Aira-Instruct-774M` is a instruction-tuned GPT-style model based on [GPT-2](https://huggingface.co/gpt2). The model was trained with a dataset composed of `prompt`, `completions`, generated via the [Self-Instruct](https://github.com/yizhongw/self-instruct) framework. `Aira-Instruct-774M` instruction-tuning was achieved via conditional text generation.
 
-The dataset used to train this model combines two main sources of data: the [`synthetic-instruct-gptj-pairwise`](https://huggingface.co/datasets/Dahoas/synthetic-instruct-gptj-pairwise) dataset and a subset of [Aira's](https://github.com/Nkluge-correa/Aira-EXPERT) fine-tuning dataset focused on Ethics, AI, AI safety, and related topics. The dataset is available in both Portuguese and English.
+The dataset used to train this model combines the following sources of data: the [`synthetic-instruct-gptj-pairwise`](https://huggingface.co/datasets/Dahoas/synthetic-instruct-gptj-pairwise) dataset, the [`databricks_dolly_15k`](https://huggingface.co/datasets/HuggingFaceH4/databricks_dolly_15k) dataset, the [`instruction-dataset`](https://huggingface.co/datasets/HuggingFaceH4/instruction-dataset) dataset, and a subset of [Aira's](https://github.com/Nkluge-correa/Aira-EXPERT) fine-tuning dataset, focused on Q&A related to Ethics, AI, AI safety, and other related topics. The dataset is available in both Portuguese and English.
+
+Check our gradio-demo in [Spaces](https://huggingface.co/spaces/nicholasKluge/Aira-Demo).
 
 ## Details
 
@@ -45,17 +49,19 @@ The dataset used to train this model combines two main sources of data: the [`sy
 - **Batch size:** 8
 - **Optimizer:** `torch.optim.AdamW` (warmup_steps = 1e2, learning_rate = 5e-4, epsilon = 1e-8)
 - **GPU:** 1 NVIDIA A100-SXM4-40GB
+- **Emissions:** 0.0022 KgCO2 (Canada)
+- **Total Energy Consumption:** 0.95 kWh
 
 | Epoch/Loss|Training|Validation|
 |---|---|---|
-| 1 |0.654409|0.576611|
-| 2 |0.428551|0.53646|
+| 1 |0.687266|0.616128|
+| 2 |0.468581|0.582550|
 
 This repository has the notebook used to train this model.
 
 ## Usage
 
-Two special tokens are used to mark the user side of the interaction and the model's response: 
+Two special tokens are used to mark the user side of the interaction and the model's response:
 
 `<|startoftext|>`What is a language model?`<|endoftext|>`A language model is a probability distribution over a vocabulary.`<|endoftext|>`
 
@@ -89,7 +95,6 @@ responses = aira.generate(**inputs,
 print(f"Question: 👤 {question}\n")
 
 for i, response in  enumerate(responses):
-	# print only the response and remove the question
 	print(f'Response {i+1}: 🤖 {tokenizer.decode(response, skip_special_tokens=True).replace(question, "")}')
 ```
 

config.json

@@ -33,7 +33,7 @@
     }
   },
   "torch_dtype": "float32",
-  "transformers_version": "4.29.2",
+  "transformers_version": "4.30.2",
   "use_cache": true,
   "vocab_size": 50259
 }

generation_config.json

@@ -2,5 +2,5 @@
   "_from_model_config": true,
   "bos_token_id": 50256,
   "eos_token_id": 50256,
-  "transformers_version": "4.29.2"
+  "transformers_version": "4.30.2"
 }

pytorch_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:4656fffbd51dbc1d065b3bb718c470f5dc44dca0b713045dba99f7b27bbfa2da
-size 3134045245
+oid sha256:4cc730a83ca89468d37553705b0fc07a5e20bc643a43e8dce7ccba4e93fa6a68
+size 3096272925

training_stats.parquet

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:e0c09a22a10fd01bdcdb645c0319ae6a2063dc72e94a294606a350f5647a060e
-size 3042
+oid sha256:67af53f8de8a9927257116a5c8c06d33e757d57c2f2f8f4f5428060af02d0f88
+size 3041