RuterNorway commited on
Commit
069ca3f
1 Parent(s): d63493c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +203 -1
README.md CHANGED
@@ -1,3 +1,205 @@
1
  ---
2
- license: llama2
 
 
 
 
 
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ language:
3
+ - en
4
+ - no
5
+ pipeline_tag: text-generation
6
+ inference: false
7
+ tags:
8
+ - pytorch
9
+ - llama
10
+ - llama-2
11
+ - norwegian
12
+ - norsk
13
+ datasets:
14
+ - NbAiLab/norwegian-alpaca
15
+ - RuterNorway/OpenOrcaNo-15k
16
  ---
17
+
18
+ # Llama 2 7b Chat Norwegian LoRA adaptor
19
+ **This is the LoRA adaptor for the Llama 2 7b Chat Norwegian model, and requires the original base model to run**
20
+
21
+ Go [here](#demo-script) for a demo inference script and Google Colab implementation
22
+
23
+ Llama-2-7b-chat-norwegian is a variant of [Meta](https://huggingface.co/meta-llama)´s [Llama 2 7b Chat](https://huggingface.co/meta-llama/Llama-2-7b-chat-hf) model, finetuned on a mix of norwegian datasets created in [Ruter AI Lab](https://ruter.no) the summer of 2023.
24
+
25
+ The model is tuned to understand and generate text in Norwegian. It's trained for one epoch on norwegian-alpaca + 15000 samples of machine-translated data from OpenOrca. A small subset of custom-made instructional data is also included.
26
+
27
+ For other versions of this model see:
28
+ * [Llama-2-7b-chat-norwegian](https://huggingface.co/RuterNorway/Llama-2-7b-chat-norwegian)
29
+ * [Llama-2-7b-chat-norwegian-LoRa](https://huggingface.co/RuterNorway/Llama-2-7b-chat-norwegian-LoRa)
30
+ * [Llama-2-7b-chat-norwegian-GPTQ](https://huggingface.co/RuterNorway/Llama-2-7b-chat-norwegian-GPTQ)
31
+
32
+
33
+ ## Data
34
+ * Norwegian alpaca
35
+ * 15k Norwegian OpenOrcra
36
+ * Small subset of custom made instructional data
37
+
38
+ ## Intended Use
39
+ This model is intended for commercial and research use in Norwegian and can be used as an assistant-like chat.
40
+
41
+ ## Prompt Template
42
+ Llama2 Chat uses a new prompt format:
43
+
44
+ ```
45
+ <s>[INST] <<SYS>>
46
+ You are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature.
47
+ If a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information. Please answer in the same language as the user.
48
+ <</SYS>>
49
+ This is a test question[/INST] This is a answer </s><s>
50
+ ```
51
+ See also the original implementation [here](https://github.com/facebookresearch/llama/blob/main/llama/generation.py#L213).
52
+
53
+ We also implemented the alpaca prompt format, which the model supports.:
54
+ ```
55
+ ### Instruction:
56
+ Summarize following text.
57
+ ### Input:
58
+ Text to be summarized
59
+ ### Response:
60
+ ```
61
+
62
+ ## Why this model?
63
+ As a Norwegian company, we understand firsthand the pressing need for powerful language models tailored to specific languages. Our primary focus is on the Norwegian linguistic landscape. In the age of digitization, languages that lack robust, open-source models can risk becoming marginalized. This is why we're introducing this open-source Norwegian model. We believe that by making such resources freely accessible, we can democratize information, foster innovation, and create a more inclusive digital ecosystem. Our aspiration is for this model to serve as a foundational resource for future specialized Norwegian models. Ultimately, our goal is to bolster the Norwegian NLP community and facilitate the smoother integration of Norwegian models into diverse projects.
64
+
65
+ ## Limitations
66
+ * This is an LLM, not a knowledge model. It can not be expected to have more information about Norway than the basemodel.
67
+ * It will generally preform better on tasks that involves summarization, question answering and chat, than on tasks that requires more knowledge about Norway, specific domains, or tasks where the model can answer freely.
68
+ * The data used for training is machine translated, and may contain grammatical errors and other errors.
69
+ * The model is released as is, and would in most cases need prompt tuning to achieve optimal results.
70
+
71
+
72
+ ## License
73
+ Llama 2 is licensed under the LLAMA 2 [Community License](https://ai.meta.com/resources/models-and-libraries/llama-downloads/), Copyright © Meta Platforms, Inc. All Rights Reserved.
74
+ See the original [model card](https://huggingface.co/meta-llama/Llama-2-7b) for more information.
75
+
76
+ From [norwegian-alpaca](https://huggingface.co/NbAiLab/norwegian-alpaca) we also note that "the current version uses OpenAI's gpt-3.5-turbo; hence, this dataset cannot be used to create models that compete in any way against OpenAI."
77
+
78
+
79
+ ## Disclaimer
80
+ * The model is available "as is". Ruter As takes no responsibility for further use.
81
+ * During testing, it seems that the safeguards implemented by Meta, still work as expected in this model. However, we want to point to the Ethical Considerations and Limitations from the origenal model card:
82
+ ```
83
+ Llama 2 is a new technology that carries risks with use. Testing conducted to date has been in English, and has not covered, nor could it cover all scenarios.
84
+ For these reasons, as with all LLMs, Llama 2’s potential outputs cannot be predicted in advance, and the model may in some instances produce inaccurate, biased or other objectionable responses to user prompts.
85
+ Therefore, before deploying any applications of Llama 2, developers should perform safety testing and tuning tailored to their specific applications of the model.
86
+ Please see the Responsible Use Guide available at https://ai.meta.com/llama/responsible-use-guide/
87
+ ```
88
+ ## Credits
89
+ This model was made at Ruters AI Lab - summer of 2023 as part of their AI initiative.
90
+
91
+ The team wants to thank the support we got from the entire Ruter organization, and especially the Data Science team.
92
+
93
+ ___
94
+ # Llama 2 7b Chat Norwegian LoRA-adapter (Norsk)
95
+ **Dette er LoRA-adapteren for Llama 2 7b Chat Norwegian modellen, og krever den orginale basismodellen for å kjøre**
96
+ Llama-2-7b-chat-norwegian er en versjon av [Meta](https://huggingface.co/meta-llama) sin [Llama 2 7b Chat](https://huggingface.co/meta-llama/Llama-2-7b-chat-hf) model, finetuned på en kombinasjon av diverse norske datasett. Modellen ble laget i [Ruter AI Lab](https://ruter.no) 2023.
97
+
98
+ Modellen er finetuned til å forstå og generere tekst på Norsk. Den er trent i én epoch med norwegian-alpaca + et utvalg av 15000 maskinoversatt data fra OpenOrca. Det består og av et lite sett med selvlagde instruksjonsdata.
99
+
100
+ Andre versjoner av modellen:
101
+
102
+ * [Llama-2-7b-chat-norwegian](https://huggingface.co/RuterNorway/Llama-2-7b-chat-norwegian)
103
+ * [Llama-2-7b-chat-norwegian-LoRa](https://huggingface.co/RuterNorway/Llama-2-7b-chat-norwegian-LoRa)
104
+ * [Llama-2-7b-chat-norwegian-GPTQ](https://huggingface.co/RuterNorway/Llama-2-7b-chat-norwegian-GPTQ)
105
+
106
+
107
+ ## Data
108
+ * Norwegian alpaca
109
+ * 15k Norwegian OpenOrcra
110
+ * Lite sett med selvlagde instruksjonsdata
111
+
112
+
113
+ ## Prompt Mal
114
+ Llama2 Chat bruker et nytt prompt format:
115
+ ```
116
+ <s>[INST] <<SYS>>
117
+ You are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature.
118
+ If a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information. Please answer in the same language as the user.
119
+ <</SYS>>
120
+ This is a test question[/INST] This is a answer </s><s>
121
+ ```
122
+ Se orgianl implementasjon [her](https://github.com/facebookresearch/llama/blob/main/llama/generation.py#L213).
123
+
124
+ Vi har også implementert alpaca prompt formatet, som også er støttet av modellen.
125
+ ```
126
+ ### Instruction:
127
+ Summarize following text.
128
+ ### Input:
129
+ Text to be summarized
130
+ ### Response:
131
+ ```
132
+
133
+ ## Hvorfor denne modellen?
134
+ Som et norsk selskap forstår vi selv det presserende behovet for kraftige språkmodeller tilpasset spesifikke språk. Vårt primære fokus er på det norske språkområdet. I den digitale alderen risikerer språk som mangler robuste, åpne kildekodemodeller å bli marginalisert. Dette er grunnen til at vi nå introduserer denne åpne kildekodemodellen for norsk. Vi tror at ved å gjøre disse ressursene tilgjengelige gratis, kan vi demokratisere informasjonen, fremme innovasjon og skape et mer inkluderende digitalt økosystem. Vår ambisjon er at denne modellen skal tjene som en grunnleggende ressurs for fremtidige spesialiserte norske modeller. Vårt mål er å styrke det norske NLP-miljøet og gjøre det enklere å innlemme norske modeller i ulike prosjekter.
135
+
136
+ ## Begrensninger
137
+ * Dette er en LLM, ikke en kunnskapsmodell. Den kan ikke forventes å ha mer informasjon om Norge enn basismodellen.
138
+ * Den vil generelt prestere bedre på oppgaver som innebærer oppsummering, spørsmålsbesvarelse og chat, enn på oppgaver som krever mer kunnskap om Norge, spesifikke domener, eller oppgaver hvor modellen kan svare fritt.
139
+ * Dataene som brukes til trening er maskinoversatt, og kan inneholde grammatiske feil. Vi har kun gjort en rask manuell sjekk av dataene.
140
+ * Modellen er utgitt som den er, og vil i de fleste tilfeller trenge "prompt tuning" for å oppnå ønskede resultater.
141
+
142
+ ## Lisens
143
+ Llama 2 er lisensiert under LLAMA 2 [Community License](https://ai.meta.com/resources/models-and-libraries/llama-downloads/), Copyright © Meta Platforms, Inc. All Rights Reserved.
144
+ Se det orginale [modell kortet](https://huggingface.co/meta-llama/Llama-2-7b) for mer informasjon.
145
+
146
+
147
+ Fra [norwegian-alpaca](https://huggingface.co/NbAiLab/norwegian-alpaca) vil vi gjøre oppmerksomme på at "the current version uses OpenAI's gpt-3.5-turbo; hence, this dataset cannot be used to create models that compete in any way against OpenAI."
148
+
149
+
150
+ ## Ansvarsfraskrivelse
151
+ * Modellen tilgjengeliggjøres «som den er». Ruter As tar ikke noe ansvar for videre bruk.
152
+ * Under testingen virket det som sikkerhetstiltakene implementert av Meta fortsatt fungerer som forventet for denne modellen. Vi gjør derimot oppmerksom på de etiske betraktiningene og begrensningene fra det orignale modellkortet:
153
+ ```
154
+ Llama 2 is a new technology that carries risks with use. Testing conducted to date has been in English, and has not covered, nor could it cover all scenarios.
155
+ For these reasons, as with all LLMs, Llama 2’s potential outputs cannot be predicted in advance, and the model may in some instances produce inaccurate, biased or other objectionable responses to user prompts.
156
+ Therefore, before deploying any applications of Llama 2, developers should perform safety testing and tuning tailored to their specific applications of the model.
157
+ Please see the Responsible Use Guide available at https://ai.meta.com/llama/responsible-use-guide/
158
+ ```
159
+
160
+
161
+ # Demo script
162
+ This is a minimal example of how to use the adapter for inference
163
+
164
+ You can also check out our [colab implementation](https://colab.research.google.com/drive/1mAx2K-kX_YzymXE9vXNQmJ5IFeMB-CN8?usp=sharing).
165
+
166
+ ```python
167
+ import torch
168
+ from peft import PeftModel
169
+ from transformers import LlamaTokenizer, LlamaForCausalLM
170
+
171
+
172
+ MODEL = 'meta-llama/Llama-2-7b-chat-hf'
173
+ ADAPTER = 'RuterNorway/Llama-2-7b-chat-norwegian-LoRa'
174
+ HF_TOKEN = '...'
175
+
176
+
177
+ prompt = """
178
+ ### Instruction
179
+ Hva heter du?
180
+ ### Answer
181
+ """
182
+
183
+
184
+ tokenizer = LlamaTokenizer.from_pretrained(MODEL, legacy=False, use_auth_token=HF_TOKEN)
185
+
186
+ base_model = LlamaForCausalLM.from_pretrained(
187
+ MODEL,
188
+ device_map='auto',
189
+ load_in_8bit=True,
190
+ torch_dtype=torch.float16,
191
+ use_auth_token=HF_TOKEN,
192
+ )
193
+ model = PeftModel.from_pretrained(
194
+ base_model, ADAPTER, torch_dtype=torch.float16, is_trainable=False
195
+ )
196
+
197
+
198
+ with torch.no_grad():
199
+ output_tensors = model.generate(
200
+ input_ids=tokenizer(prompt, return_tensors="pt").input_ids.cuda(),
201
+ max_new_tokens=128
202
+ )[0]
203
+
204
+ tokenizer.decode(output_tensors, skip_special_tokens=True).split('### Answer')[-1]
205
+ ```