mhenrichsen commited on
Commit
db81ea7
1 Parent(s): 10c7d98

Upload folder using huggingface_hub

Browse files
.gitattributes CHANGED
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ tokenizer.json filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,493 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: transformers
3
+ ---
4
+ # Re-uploaded model. Original readme below.
5
+
6
+
7
+ # Gemma Model Card
8
+
9
+ **Model Page**: [Gemma](https://ai.google.dev/gemma/docs)
10
+
11
+ This model card corresponds to the 7B instruct version of the Gemma model. You can also visit the model card of the [2B base model](https://huggingface.co/google/gemma-2b), [7B base model](https://huggingface.co/google/gemma-7b), and [2B instruct model](https://huggingface.co/google/gemma-2b-it).
12
+
13
+ **Resources and Technical Documentation**:
14
+
15
+ * [Responsible Generative AI Toolkit](https://ai.google.dev/responsible)
16
+ * [Gemma on Kaggle](https://www.kaggle.com/models/google/gemma)
17
+ * [Gemma on Vertex Model Garden](https://console.cloud.google.com/vertex-ai/publishers/google/model-garden/335?version=gemma-7b-it-gg-hf)
18
+
19
+ **Terms of Use**: [Terms](https://www.kaggle.com/models/google/gemma/license/consent)
20
+
21
+ **Authors**: Google
22
+
23
+ ## Model Information
24
+
25
+ Summary description and brief definition of inputs and outputs.
26
+
27
+ ### Description
28
+
29
+ Gemma is a family of lightweight, state-of-the-art open models from Google,
30
+ built from the same research and technology used to create the Gemini models.
31
+ They are text-to-text, decoder-only large language models, available in English,
32
+ with open weights, pre-trained variants, and instruction-tuned variants. Gemma
33
+ models are well-suited for a variety of text generation tasks, including
34
+ question answering, summarization, and reasoning. Their relatively small size
35
+ makes it possible to deploy them in environments with limited resources such as
36
+ a laptop, desktop or your own cloud infrastructure, democratizing access to
37
+ state of the art AI models and helping foster innovation for everyone.
38
+
39
+ ### Usage
40
+
41
+ Below we share some code snippets on how to get quickly started with running the model. First make sure to `pip install -U transformers`, then copy the snippet from the section that is relevant for your usecase.
42
+
43
+ #### Fine-tuning the model
44
+
45
+ You can find fine-tuning scripts and notebook under the [`examples/` directory](https://huggingface.co/google/gemma-7b/tree/main/examples) of [`google/gemma-7b`](https://huggingface.co/google/gemma-7b) repository. To adapt it to this model, simply change the model-id to `google/gemma-7b-it`.
46
+ In that repository, we provide:
47
+
48
+ * A script to perform Supervised Fine-Tuning (SFT) on UltraChat dataset using QLoRA
49
+ * A script to perform SFT using FSDP on TPU devices
50
+ * A notebook that you can run on a free-tier Google Colab instance to perform SFT on English quotes dataset
51
+
52
+
53
+ #### Running the model on a CPU
54
+
55
+
56
+ ```python
57
+ from transformers import AutoTokenizer, AutoModelForCausalLM
58
+
59
+ tokenizer = AutoTokenizer.from_pretrained("google/gemma-7b-it")
60
+ model = AutoModelForCausalLM.from_pretrained("google/gemma-7b-it")
61
+
62
+ input_text = "Write me a poem about Machine Learning."
63
+ input_ids = tokenizer(**input_text, return_tensors="pt")
64
+
65
+ outputs = model.generate(input_ids)
66
+ print(tokenizer.decode(outputs[0]))
67
+ ```
68
+
69
+
70
+ #### Running the model on a single / multi GPU
71
+
72
+
73
+ ```python
74
+ # pip install accelerate
75
+ from transformers import AutoTokenizer, AutoModelForCausalLM
76
+
77
+ tokenizer = AutoTokenizer.from_pretrained("google/gemma-7b-it")
78
+ model = AutoModelForCausalLM.from_pretrained("google/gemma-7b-it", device_map="auto")
79
+
80
+ input_text = "Write me a poem about Machine Learning."
81
+ input_ids = tokenizer(input_text, return_tensors="pt").to("cuda")
82
+
83
+ outputs = model.generate(**input_ids)
84
+ print(tokenizer.decode(outputs[0]))
85
+ ```
86
+
87
+
88
+ #### Running the model on a GPU using different precisions
89
+
90
+ * _Using `torch.float16`_
91
+
92
+ ```python
93
+ # pip install accelerate
94
+ from transformers import AutoTokenizer, AutoModelForCausalLM
95
+
96
+ tokenizer = AutoTokenizer.from_pretrained("google/gemma-7b-it")
97
+ model = AutoModelForCausalLM.from_pretrained("google/gemma-7b-it", device_map="auto", torch_dtype=torch.float16)
98
+
99
+ input_text = "Write me a poem about Machine Learning."
100
+ input_ids = tokenizer(input_text, return_tensors="pt").to("cuda")
101
+
102
+ outputs = model.generate(**input_ids)
103
+ print(tokenizer.decode(outputs[0]))
104
+ ```
105
+
106
+ * _Using `torch.bfloat16`_
107
+
108
+ ```python
109
+ # pip install accelerate
110
+ from transformers import AutoTokenizer, AutoModelForCausalLM
111
+
112
+ tokenizer = AutoTokenizer.from_pretrained("google/gemma-7b-it")
113
+ model = AutoModelForCausalLM.from_pretrained("google/gemma-7b-it", device_map="auto", torch_dtype=torch.bfloat16)
114
+
115
+ input_text = "Write me a poem about Machine Learning."
116
+ input_ids = tokenizer(input_text, return_tensors="pt").to("cuda")
117
+
118
+ outputs = model.generate(**input_ids)
119
+ print(tokenizer.decode(outputs[0]))
120
+ ```
121
+
122
+ #### Quantized Versions through `bitsandbytes`
123
+
124
+ * _Using 8-bit precision (int8)_
125
+
126
+ ```python
127
+ # pip install bitsandbytes accelerate
128
+ from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
129
+
130
+ quantization_config = BitsAndBytesConfig(load_in_8bit=True)
131
+
132
+ tokenizer = AutoTokenizer.from_pretrained("google/gemma-7b-it")
133
+ model = AutoModelForCausalLM.from_pretrained("google/gemma-7b-it", quantization_config=quantization_config)
134
+
135
+ input_text = "Write me a poem about Machine Learning."
136
+ input_ids = tokenizer(input_text, return_tensors="pt").to("cuda")
137
+
138
+ outputs = model.generate(**input_ids)
139
+ print(tokenizer.decode(outputs[0]))
140
+ ```
141
+
142
+ * _Using 4-bit precision_
143
+
144
+ ```python
145
+ # pip install bitsandbytes accelerate
146
+ from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
147
+
148
+ quantization_config = BitsAndBytesConfig(load_in_4bit=True)
149
+
150
+ tokenizer = AutoTokenizer.from_pretrained("google/gemma-7b-it")
151
+ model = AutoModelForCausalLM.from_pretrained("google/gemma-7b-it", quantization_config=quantization_config)
152
+
153
+ input_text = "Write me a poem about Machine Learning."
154
+ input_ids = tokenizer(input_text, return_tensors="pt").to("cuda")
155
+
156
+ outputs = model.generate(**input_ids)
157
+ print(tokenizer.decode(outputs[0]))
158
+ ```
159
+
160
+
161
+ #### Other optimizations
162
+
163
+ * _Flash Attention 2_
164
+
165
+ First make sure to install `flash-attn` in your environment `pip install flash-attn`
166
+
167
+ ```diff
168
+ model = AutoModelForCausalLM.from_pretrained(
169
+ model_id,
170
+ torch_dtype=torch.float16,
171
+ + attn_implementation="flash_attention_2"
172
+ ).to(0)
173
+ ```
174
+
175
+ ### Chat Template
176
+
177
+ The instruction-tuned models use a chat template that must be adhered to for conversational use.
178
+ The easiest way to apply it is using the tokenizer's built-in chat template, as shown in the following snippet.
179
+
180
+ Let's load the model and apply the chat template to a conversation. In this example, we'll start with a single user interaction:
181
+
182
+ ```py
183
+ from transformers import AutoTokenizer, AutoModelForCausalLM
184
+ import transformers
185
+ import torch
186
+
187
+ model_id = "gg-hf/gemma-7b-it"
188
+ dtype = torch.bfloat16
189
+
190
+ tokenizer = AutoTokenizer.from_pretrained(model_id)
191
+ model = AutoModelForCausalLM.from_pretrained(
192
+ model_id,
193
+ device_map="cuda",
194
+ torch_dtype=dtype,
195
+ )
196
+
197
+ chat = [
198
+ { "role": "user", "content": "Write a hello world program" },
199
+ ]
200
+ prompt = tokenizer.apply_chat_template(chat, tokenize=False, add_generation_prompt=True)
201
+ ```
202
+
203
+ At this point, the prompt contains the following text:
204
+
205
+ ```
206
+ <start_of_turn>user
207
+ Write a hello world program<end_of_turn>
208
+ <start_of_turn>model
209
+ ```
210
+
211
+ As you can see, each turn is preceeded by a `<start_of_turn>` delimiter and then the role of the entity
212
+ (either `user`, for content supplied by the user, or `model` for LLM responses). Turns finish with
213
+ the `<end_of_turn>` token.
214
+
215
+ You can follow this format to build the prompt manually, if you need to do it without the tokenizer's
216
+ chat template.
217
+
218
+ After the prompt is ready, generation can be performed like this:
219
+
220
+ ```py
221
+ inputs = tokenizer.encode(prompt, add_special_tokens=True, return_tensors="pt")
222
+ outputs = model.generate(input_ids=inputs.to(model.device), max_new_tokens=150)
223
+ ```
224
+
225
+ ### Inputs and outputs
226
+
227
+ * **Input:** Text string, such as a question, a prompt, or a document to be
228
+ summarized.
229
+ * **Output:** Generated English-language text in response to the input, such
230
+ as an answer to a question, or a summary of a document.
231
+
232
+ ## Model Data
233
+
234
+ Data used for model training and how the data was processed.
235
+
236
+ ### Training Dataset
237
+
238
+ These models were trained on a dataset of text data that includes a wide variety
239
+ of sources, totaling 6 trillion tokens. Here are the key components:
240
+
241
+ * Web Documents: A diverse collection of web text ensures the model is exposed
242
+ to a broad range of linguistic styles, topics, and vocabulary. Primarily
243
+ English-language content.
244
+ * Code: Exposing the model to code helps it to learn the syntax and patterns of
245
+ programming languages, which improves its ability to generate code or
246
+ understand code-related questions.
247
+ * Mathematics: Training on mathematical text helps the model learn logical
248
+ reasoning, symbolic representation, and to address mathematical queries.
249
+
250
+ The combination of these diverse data sources is crucial for training a powerful
251
+ language model that can handle a wide variety of different tasks and text
252
+ formats.
253
+
254
+ ### Data Preprocessing
255
+
256
+ Here are the key data cleaning and filtering methods applied to the training
257
+ data:
258
+
259
+ * CSAM Filtering: Rigorous CSAM (Child Sexual Abuse Material) filtering was
260
+ applied at multiple stages in the data preparation process to ensure the
261
+ exclusion of harmful and illegal content
262
+ * Sensitive Data Filtering: As part of making Gemma pre-trained models safe and
263
+ reliable, automated techniques were used to filter out certain personal
264
+ information and other sensitive data from training sets.
265
+ * Additional methods: Filtering based on content quality and safely in line with
266
+ [our policies](https://storage.googleapis.com/gweb-uniblog-publish-prod/documents/2023_Google_AI_Principles_Progress_Update.pdf#page=11).
267
+
268
+ ## Implementation Information
269
+
270
+ Details about the model internals.
271
+
272
+ ### Hardware
273
+
274
+ Gemma was trained using the latest generation of
275
+ [Tensor Processing Unit (TPU)](https://cloud.google.com/tpu/docs/intro-to-tpu) hardware (TPUv5e).
276
+
277
+ Training large language models requires significant computational power. TPUs,
278
+ designed specifically for matrix operations common in machine learning, offer
279
+ several advantages in this domain:
280
+
281
+ * Performance: TPUs are specifically designed to handle the massive computations
282
+ involved in training LLMs. They can speed up training considerably compared to
283
+ CPUs.
284
+ * Memory: TPUs often come with large amounts of high-bandwidth memory, allowing
285
+ for the handling of large models and batch sizes during training. This can
286
+ lead to better model quality.
287
+ * Scalability: TPU Pods (large clusters of TPUs) provide a scalable solution for
288
+ handling the growing complexity of large foundation models. You can distribute
289
+ training across multiple TPU devices for faster and more efficient processing.
290
+ * Cost-effectiveness: In many scenarios, TPUs can provide a more cost-effective
291
+ solution for training large models compared to CPU-based infrastructure,
292
+ especially when considering the time and resources saved due to faster
293
+ training.
294
+ * These advantages are aligned with
295
+ [Google's commitments to operate sustainably](https://sustainability.google/operating-sustainably/).
296
+
297
+ ### Software
298
+
299
+ Training was done using [JAX](https://github.com/google/jax) and [ML Pathways](https://blog.google/technology/ai/introducing-pathways-next-generation-ai-architecture).
300
+
301
+ JAX allows researchers to take advantage of the latest generation of hardware,
302
+ including TPUs, for faster and more efficient training of large models.
303
+
304
+ ML Pathways is Google's latest effort to build artificially intelligent systems
305
+ capable of generalizing across multiple tasks. This is specially suitable for
306
+ [foundation models](https://ai.google/discover/foundation-models/), including large language models like
307
+ these ones.
308
+
309
+ Together, JAX and ML Pathways are used as described in the
310
+ [paper about the Gemini family of models](https://arxiv.org/abs/2312.11805); "the 'single
311
+ controller' programming model of Jax and Pathways allows a single Python
312
+ process to orchestrate the entire training run, dramatically simplifying the
313
+ development workflow."
314
+
315
+ ## Evaluation
316
+
317
+ Model evaluation metrics and results.
318
+
319
+ ### Benchmark Results
320
+
321
+ These models were evaluated against a large collection of different datasets and
322
+ metrics to cover different aspects of text generation:
323
+
324
+ | Benchmark | Metric | 2B Params | 7B Params |
325
+ | ------------------------------ | ------------- | ----------- | --------- |
326
+ | [MMLU](https://arxiv.org/abs/2009.03300) | 5-shot, top-1 | 42.3 | 64.3 |
327
+ | [HellaSwag](https://arxiv.org/abs/1905.07830) | 0-shot |71.4 | 81.2 |
328
+ | [PIQA](https://arxiv.org/abs/1911.11641) | 0-shot | 77.3 | 81.2 |
329
+ | [SocialIQA](https://arxiv.org/abs/1904.09728) | 0-shot | 59.7 | 51.8 |
330
+ | [BooIQ](https://arxiv.org/abs/1905.10044) | 0-shot | 69.4 | 83.2 |
331
+ | [WinoGrande](https://arxiv.org/abs/1907.10641) | partial score | 65.4 | 72.3 |
332
+ | [CommonsenseQA](https://arxiv.org/abs/1811.00937) | 7-shot | 65.3 | 71.3 |
333
+ | [OpenBookQA](https://arxiv.org/abs/1809.02789) | | 47.8 | 52.8 |
334
+ | [ARC-e](https://arxiv.org/abs/1911.01547) | | 73.2 | 81.5 |
335
+ | [ARC-c](https://arxiv.org/abs/1911.01547) | | 42.1 | 53.2 |
336
+ | [TriviaQA](https://arxiv.org/abs/1705.03551) | 5-shot | 53.2 | 63.4 |
337
+ | [Natural Questions](https://github.com/google-research-datasets/natural-questions) | 5-shot | - | 23 |
338
+ | [HumanEval](https://arxiv.org/abs/2107.03374) | pass@1 | 22.0 | 32.3 |
339
+ | [MBPP](https://arxiv.org/abs/2108.07732) | 3-shot | 29.2 | 44.4 |
340
+ | [GSM8K](https://arxiv.org/abs/2110.14168) | maj@1 | 17.7 | 46.4 |
341
+ | [MATH](https://arxiv.org/abs/2108.07732) | 4-shot | 11.8 | 24.3 |
342
+ | [AGIEval](https://arxiv.org/abs/2304.06364) | | 24.2 | 41.7 |
343
+ | [BIG-Bench](https://arxiv.org/abs/2206.04615) | | 35.2 | 55.1 |
344
+ | ------------------------------ | ------------- | ----------- | --------- |
345
+ | **Average** | | **54.0** | **56.4** |
346
+
347
+ ## Ethics and Safety
348
+
349
+ Ethics and safety evaluation approach and results.
350
+
351
+ ### Evaluation Approach
352
+
353
+ Our evaluation methods include structured evaluations and internal red-teaming
354
+ testing of relevant content policies. Red-teaming was conducted by a number of
355
+ different teams, each with different goals and human evaluation metrics. These
356
+ models were evaluated against a number of different categories relevant to
357
+ ethics and safety, including:
358
+
359
+ * Text-to-Text Content Safety: Human evaluation on prompts covering safety
360
+ policies including child sexual abuse and exploitation, harassment, violence
361
+ and gore, and hate speech.
362
+ * Text-to-Text Representational Harms: Benchmark against relevant academic
363
+ datasets such as [WinoBias](https://arxiv.org/abs/1804.06876) and [BBQ Dataset](https://arxiv.org/abs/2110.08193v2).
364
+ * Memorization: Automated evaluation of memorization of training data, including
365
+ the risk of personally identifiable information exposure.
366
+ * Large-scale harm: Tests for "dangerous capabilities," such as chemical,
367
+ biological, radiological, and nuclear (CBRN) risks.
368
+
369
+ ### Evaluation Results
370
+
371
+ The results of ethics and safety evaluations are within acceptable thresholds
372
+ for meeting [internal policies](https://storage.googleapis.com/gweb-uniblog-publish-prod/documents/2023_Google_AI_Principles_Progress_Update.pdf#page=11) for categories such as child
373
+ safety, content safety, representational harms, memorization, large-scale harms.
374
+ On top of robust internal evaluations, the results of well known safety
375
+ benchmarks like BBQ, BOLD, Winogender, Winobias, RealToxicity, and TruthfulQA
376
+ are shown here.
377
+
378
+ | Benchmark | Metric | 2B Params | 7B Params |
379
+ | ------------------------------ | ------------- | ----------- | --------- |
380
+ | [RealToxicity](https://arxiv.org/abs/2009.11462) | average | 6.86 | 7.90 |
381
+ | [BOLD](https://arxiv.org/abs/2101.11718) | | 45.57 | 49.08 |
382
+ | [CrowS-Pairs](https://aclanthology.org/2020.emnlp-main.154/) | top-1 | 45.82 | 51.33 |
383
+ | [BBQ Ambig](https://arxiv.org/abs/2110.08193v2) | 1-shot, top-1 | 62.58 | 92.54 |
384
+ | [BBQ Disambig](https://arxiv.org/abs/2110.08193v2) | top-1 | 54.62 | 71.99 |
385
+ | [Winogender](https://arxiv.org/abs/1804.09301) | top-1 | 51.25 | 54.17 |
386
+ | [TruthfulQA](https://arxiv.org/abs/2109.07958) | | 44.84 | 31.81 |
387
+ | [Winobias 1_2](https://arxiv.org/abs/1804.06876) | | 56.12 | 59.09 |
388
+ | [Winobias 2_2](https://arxiv.org/abs/1804.06876) | | 91.10 | 92.23 |
389
+ | [Toxigen](https://arxiv.org/abs/2203.09509) | | 29.77 | 39.59 |
390
+ | ------------------------------ | ------------- | ----------- | --------- |
391
+
392
+
393
+ ## Usage and Limitations
394
+
395
+ These models have certain limitations that users should be aware of.
396
+
397
+ ### Intended Usage
398
+
399
+ Open Large Language Models (LLMs) have a wide range of applications across
400
+ various industries and domains. The following list of potential uses is not
401
+ comprehensive. The purpose of this list is to provide contextual information
402
+ about the possible use-cases that the model creators considered as part of model
403
+ training and development.
404
+
405
+ * Content Creation and Communication
406
+ * Text Generation: These models can be used to generate creative text formats
407
+ such as poems, scripts, code, marketing copy, and email drafts.
408
+ * Chatbots and Conversational AI: Power conversational interfaces for customer
409
+ service, virtual assistants, or interactive applications.
410
+ * Text Summarization: Generate concise summaries of a text corpus, research
411
+ papers, or reports.
412
+ * Research and Education
413
+ * Natural Language Processing (NLP) Research: These models can serve as a
414
+ foundation for researchers to experiment with NLP techniques, develop
415
+ algorithms, and contribute to the advancement of the field.
416
+ * Language Learning Tools: Support interactive language learning experiences,
417
+ aiding in grammar correction or providing writing practice.
418
+ * Knowledge Exploration: Assist researchers in exploring large bodies of text
419
+ by generating summaries or answering questions about specific topics.
420
+
421
+ ### Limitations
422
+
423
+ * Training Data
424
+ * The quality and diversity of the training data significantly influence the
425
+ model's capabilities. Biases or gaps in the training data can lead to
426
+ limitations in the model's responses.
427
+ * The scope of the training dataset determines the subject areas the model can
428
+ handle effectively.
429
+ * Context and Task Complexity
430
+ * LLMs are better at tasks that can be framed with clear prompts and
431
+ instructions. Open-ended or highly complex tasks might be challenging.
432
+ * A model's performance can be influenced by the amount of context provided
433
+ (longer context generally leads to better outputs, up to a certain point).
434
+ * Language Ambiguity and Nuance
435
+ * Natural language is inherently complex. LLMs might struggle to grasp subtle
436
+ nuances, sarcasm, or figurative language.
437
+ * Factual Accuracy
438
+ * LLMs generate responses based on information they learned from their
439
+ training datasets, but they are not knowledge bases. They may generate
440
+ incorrect or outdated factual statements.
441
+ * Common Sense
442
+ * LLMs rely on statistical patterns in language. They might lack the ability
443
+ to apply common sense reasoning in certain situations.
444
+
445
+ ### Ethical Considerations and Risks
446
+
447
+ The development of large language models (LLMs) raises several ethical concerns.
448
+ In creating an open model, we have carefully considered the following:
449
+
450
+ * Bias and Fairness
451
+ * LLMs trained on large-scale, real-world text data can reflect socio-cultural
452
+ biases embedded in the training material. These models underwent careful
453
+ scrutiny, input data pre-processing described and posterior evaluations
454
+ reported in this card.
455
+ * Misinformation and Misuse
456
+ * LLMs can be misused to generate text that is false, misleading, or harmful.
457
+ * Guidelines are provided for responsible use with the model, see the
458
+ [Responsible Generative AI Toolkit](http://ai.google.dev/gemma/responsible).
459
+ * Transparency and Accountability:
460
+ * This model card summarizes details on the models' architecture,
461
+ capabilities, limitations, and evaluation processes.
462
+ * A responsibly developed open model offers the opportunity to share
463
+ innovation by making LLM technology accessible to developers and researchers
464
+ across the AI ecosystem.
465
+
466
+ Risks identified and mitigations:
467
+
468
+ * Perpetuation of biases: It's encouraged to perform continuous monitoring
469
+ (using evaluation metrics, human review) and the exploration of de-biasing
470
+ techniques during model training, fine-tuning, and other use cases.
471
+ * Generation of harmful content: Mechanisms and guidelines for content safety
472
+ are essential. Developers are encouraged to exercise caution and implement
473
+ appropriate content safety safeguards based on their specific product policies
474
+ and application use cases.
475
+ * Misuse for malicious purposes: Technical limitations and developer and
476
+ end-user education can help mitigate against malicious applications of LLMs.
477
+ Educational resources and reporting mechanisms for users to flag misuse are
478
+ provided. Prohibited uses of Gemma models are outlined in the
479
+ [Gemma Prohibited Use Policy](https://ai.google.dev/gemma/prohibited_use_policy).
480
+ * Privacy violations: Models were trained on data filtered for removal of PII
481
+ (Personally Identifiable Information). Developers are encouraged to adhere to
482
+ privacy regulations with privacy-preserving techniques.
483
+
484
+ ### Benefits
485
+
486
+ At the time of release, this family of models provides high-performance open
487
+ large language model implementations designed from the ground up for Responsible
488
+ AI development compared to similarly sized models.
489
+
490
+ Using the benchmark evaluation metrics described in this document, these models
491
+ have shown to provide superior performance to other, comparably-sized open model
492
+ alternatives.
493
+
config.json ADDED
@@ -0,0 +1,28 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "/raid/younes/gg-converted-ckpt/gemma-7b-it",
3
+ "architectures": [
4
+ "GemmaForCausalLM"
5
+ ],
6
+ "attention_bias": false,
7
+ "attention_dropout": 0.0,
8
+ "bos_token_id": 2,
9
+ "eos_token_id": 1,
10
+ "head_dim": 256,
11
+ "hidden_act": "gelu",
12
+ "hidden_size": 3072,
13
+ "initializer_range": 0.02,
14
+ "intermediate_size": 24576,
15
+ "max_position_embeddings": 8192,
16
+ "model_type": "gemma",
17
+ "num_attention_heads": 16,
18
+ "num_hidden_layers": 28,
19
+ "num_key_value_heads": 16,
20
+ "pad_token_id": 0,
21
+ "rms_norm_eps": 1e-06,
22
+ "rope_scaling": null,
23
+ "rope_theta": 10000.0,
24
+ "torch_dtype": "bfloat16",
25
+ "transformers_version": "4.38.0.dev0",
26
+ "use_cache": true,
27
+ "vocab_size": 256000
28
+ }
generation_config.json ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ {
2
+ "_from_model_config": true,
3
+ "bos_token_id": 2,
4
+ "eos_token_id": 1,
5
+ "pad_token_id": 0,
6
+ "transformers_version": "4.38.0.dev0"
7
+ }
gitattributes ADDED
@@ -0,0 +1,37 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ *.7z filter=lfs diff=lfs merge=lfs -text
2
+ *.arrow filter=lfs diff=lfs merge=lfs -text
3
+ *.bin filter=lfs diff=lfs merge=lfs -text
4
+ *.bz2 filter=lfs diff=lfs merge=lfs -text
5
+ *.ckpt filter=lfs diff=lfs merge=lfs -text
6
+ *.ftz filter=lfs diff=lfs merge=lfs -text
7
+ *.gz filter=lfs diff=lfs merge=lfs -text
8
+ *.h5 filter=lfs diff=lfs merge=lfs -text
9
+ *.joblib filter=lfs diff=lfs merge=lfs -text
10
+ *.lfs.* filter=lfs diff=lfs merge=lfs -text
11
+ *.mlmodel filter=lfs diff=lfs merge=lfs -text
12
+ *.model filter=lfs diff=lfs merge=lfs -text
13
+ *.msgpack filter=lfs diff=lfs merge=lfs -text
14
+ *.npy filter=lfs diff=lfs merge=lfs -text
15
+ *.npz filter=lfs diff=lfs merge=lfs -text
16
+ *.onnx filter=lfs diff=lfs merge=lfs -text
17
+ *.ot filter=lfs diff=lfs merge=lfs -text
18
+ *.parquet filter=lfs diff=lfs merge=lfs -text
19
+ *.pb filter=lfs diff=lfs merge=lfs -text
20
+ *.pickle filter=lfs diff=lfs merge=lfs -text
21
+ *.pkl filter=lfs diff=lfs merge=lfs -text
22
+ *.pt filter=lfs diff=lfs merge=lfs -text
23
+ *.pth filter=lfs diff=lfs merge=lfs -text
24
+ *.rar filter=lfs diff=lfs merge=lfs -text
25
+ *.safetensors filter=lfs diff=lfs merge=lfs -text
26
+ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
27
+ *.tar.* filter=lfs diff=lfs merge=lfs -text
28
+ *.tar filter=lfs diff=lfs merge=lfs -text
29
+ *.tflite filter=lfs diff=lfs merge=lfs -text
30
+ *.tgz filter=lfs diff=lfs merge=lfs -text
31
+ *.wasm filter=lfs diff=lfs merge=lfs -text
32
+ *.xz filter=lfs diff=lfs merge=lfs -text
33
+ *.zip filter=lfs diff=lfs merge=lfs -text
34
+ *.zst filter=lfs diff=lfs merge=lfs -text
35
+ *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ tokenizer.json filter=lfs diff=lfs merge=lfs -text
37
+ gemma-7b-it.gguf filter=lfs diff=lfs merge=lfs -text
model-00001-of-00004.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:12ba358c3ef24e72169e09a5d39c5ddf0a1113d646680964f1efe3c6f5b55d16
3
+ size 4995496656
model-00002-of-00004.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b13b6710374a0b7023415525b89542dfeae66942f8dbf882036cdc7db2848aea
3
+ size 4982953168
model-00003-of-00004.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2d2f076a3bd913313c98f229a1ad250afdcd36977adb97875b4b07bd2e415a98
3
+ size 4982953200
model-00004-of-00004.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4a125907b39544cb387c145c37043a756e1ec38e5b2ea6dfabe3a789b31ce723
3
+ size 2113988336
model.safetensors.index.json ADDED
@@ -0,0 +1,261 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "metadata": {
3
+ "total_size": 17075361792
4
+ },
5
+ "weight_map": {
6
+ "model.embed_tokens.weight": "model-00001-of-00004.safetensors",
7
+ "model.layers.0.input_layernorm.weight": "model-00001-of-00004.safetensors",
8
+ "model.layers.0.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
9
+ "model.layers.0.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
10
+ "model.layers.0.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
11
+ "model.layers.0.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
12
+ "model.layers.0.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
13
+ "model.layers.0.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
14
+ "model.layers.0.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
15
+ "model.layers.0.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
16
+ "model.layers.1.input_layernorm.weight": "model-00001-of-00004.safetensors",
17
+ "model.layers.1.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
18
+ "model.layers.1.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
19
+ "model.layers.1.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
20
+ "model.layers.1.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
21
+ "model.layers.1.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
22
+ "model.layers.1.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
23
+ "model.layers.1.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
24
+ "model.layers.1.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
25
+ "model.layers.10.input_layernorm.weight": "model-00002-of-00004.safetensors",
26
+ "model.layers.10.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
27
+ "model.layers.10.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
28
+ "model.layers.10.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
29
+ "model.layers.10.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
30
+ "model.layers.10.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
31
+ "model.layers.10.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
32
+ "model.layers.10.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
33
+ "model.layers.10.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
34
+ "model.layers.11.input_layernorm.weight": "model-00002-of-00004.safetensors",
35
+ "model.layers.11.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
36
+ "model.layers.11.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
37
+ "model.layers.11.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
38
+ "model.layers.11.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
39
+ "model.layers.11.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
40
+ "model.layers.11.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
41
+ "model.layers.11.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
42
+ "model.layers.11.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
43
+ "model.layers.12.input_layernorm.weight": "model-00002-of-00004.safetensors",
44
+ "model.layers.12.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
45
+ "model.layers.12.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
46
+ "model.layers.12.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
47
+ "model.layers.12.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
48
+ "model.layers.12.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
49
+ "model.layers.12.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
50
+ "model.layers.12.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
51
+ "model.layers.12.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
52
+ "model.layers.13.input_layernorm.weight": "model-00002-of-00004.safetensors",
53
+ "model.layers.13.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
54
+ "model.layers.13.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
55
+ "model.layers.13.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
56
+ "model.layers.13.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
57
+ "model.layers.13.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
58
+ "model.layers.13.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
59
+ "model.layers.13.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
60
+ "model.layers.13.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
61
+ "model.layers.14.input_layernorm.weight": "model-00002-of-00004.safetensors",
62
+ "model.layers.14.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
63
+ "model.layers.14.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
64
+ "model.layers.14.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
65
+ "model.layers.14.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
66
+ "model.layers.14.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
67
+ "model.layers.14.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
68
+ "model.layers.14.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
69
+ "model.layers.14.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
70
+ "model.layers.15.input_layernorm.weight": "model-00003-of-00004.safetensors",
71
+ "model.layers.15.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
72
+ "model.layers.15.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
73
+ "model.layers.15.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
74
+ "model.layers.15.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
75
+ "model.layers.15.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
76
+ "model.layers.15.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
77
+ "model.layers.15.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
78
+ "model.layers.15.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
79
+ "model.layers.16.input_layernorm.weight": "model-00003-of-00004.safetensors",
80
+ "model.layers.16.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
81
+ "model.layers.16.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
82
+ "model.layers.16.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
83
+ "model.layers.16.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
84
+ "model.layers.16.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
85
+ "model.layers.16.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
86
+ "model.layers.16.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
87
+ "model.layers.16.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
88
+ "model.layers.17.input_layernorm.weight": "model-00003-of-00004.safetensors",
89
+ "model.layers.17.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
90
+ "model.layers.17.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
91
+ "model.layers.17.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
92
+ "model.layers.17.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
93
+ "model.layers.17.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
94
+ "model.layers.17.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
95
+ "model.layers.17.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
96
+ "model.layers.17.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
97
+ "model.layers.18.input_layernorm.weight": "model-00003-of-00004.safetensors",
98
+ "model.layers.18.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
99
+ "model.layers.18.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
100
+ "model.layers.18.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
101
+ "model.layers.18.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
102
+ "model.layers.18.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
103
+ "model.layers.18.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
104
+ "model.layers.18.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
105
+ "model.layers.18.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
106
+ "model.layers.19.input_layernorm.weight": "model-00003-of-00004.safetensors",
107
+ "model.layers.19.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
108
+ "model.layers.19.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
109
+ "model.layers.19.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
110
+ "model.layers.19.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
111
+ "model.layers.19.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
112
+ "model.layers.19.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
113
+ "model.layers.19.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
114
+ "model.layers.19.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
115
+ "model.layers.2.input_layernorm.weight": "model-00001-of-00004.safetensors",
116
+ "model.layers.2.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
117
+ "model.layers.2.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
118
+ "model.layers.2.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
119
+ "model.layers.2.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
120
+ "model.layers.2.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
121
+ "model.layers.2.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
122
+ "model.layers.2.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
123
+ "model.layers.2.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
124
+ "model.layers.20.input_layernorm.weight": "model-00003-of-00004.safetensors",
125
+ "model.layers.20.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
126
+ "model.layers.20.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
127
+ "model.layers.20.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
128
+ "model.layers.20.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
129
+ "model.layers.20.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
130
+ "model.layers.20.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
131
+ "model.layers.20.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
132
+ "model.layers.20.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
133
+ "model.layers.21.input_layernorm.weight": "model-00003-of-00004.safetensors",
134
+ "model.layers.21.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
135
+ "model.layers.21.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
136
+ "model.layers.21.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
137
+ "model.layers.21.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
138
+ "model.layers.21.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
139
+ "model.layers.21.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
140
+ "model.layers.21.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
141
+ "model.layers.21.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
142
+ "model.layers.22.input_layernorm.weight": "model-00003-of-00004.safetensors",
143
+ "model.layers.22.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
144
+ "model.layers.22.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
145
+ "model.layers.22.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
146
+ "model.layers.22.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
147
+ "model.layers.22.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
148
+ "model.layers.22.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
149
+ "model.layers.22.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
150
+ "model.layers.22.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
151
+ "model.layers.23.input_layernorm.weight": "model-00003-of-00004.safetensors",
152
+ "model.layers.23.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
153
+ "model.layers.23.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
154
+ "model.layers.23.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
155
+ "model.layers.23.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
156
+ "model.layers.23.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
157
+ "model.layers.23.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
158
+ "model.layers.23.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
159
+ "model.layers.23.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
160
+ "model.layers.24.input_layernorm.weight": "model-00004-of-00004.safetensors",
161
+ "model.layers.24.mlp.down_proj.weight": "model-00004-of-00004.safetensors",
162
+ "model.layers.24.mlp.gate_proj.weight": "model-00004-of-00004.safetensors",
163
+ "model.layers.24.mlp.up_proj.weight": "model-00004-of-00004.safetensors",
164
+ "model.layers.24.post_attention_layernorm.weight": "model-00004-of-00004.safetensors",
165
+ "model.layers.24.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
166
+ "model.layers.24.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
167
+ "model.layers.24.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
168
+ "model.layers.24.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
169
+ "model.layers.25.input_layernorm.weight": "model-00004-of-00004.safetensors",
170
+ "model.layers.25.mlp.down_proj.weight": "model-00004-of-00004.safetensors",
171
+ "model.layers.25.mlp.gate_proj.weight": "model-00004-of-00004.safetensors",
172
+ "model.layers.25.mlp.up_proj.weight": "model-00004-of-00004.safetensors",
173
+ "model.layers.25.post_attention_layernorm.weight": "model-00004-of-00004.safetensors",
174
+ "model.layers.25.self_attn.k_proj.weight": "model-00004-of-00004.safetensors",
175
+ "model.layers.25.self_attn.o_proj.weight": "model-00004-of-00004.safetensors",
176
+ "model.layers.25.self_attn.q_proj.weight": "model-00004-of-00004.safetensors",
177
+ "model.layers.25.self_attn.v_proj.weight": "model-00004-of-00004.safetensors",
178
+ "model.layers.26.input_layernorm.weight": "model-00004-of-00004.safetensors",
179
+ "model.layers.26.mlp.down_proj.weight": "model-00004-of-00004.safetensors",
180
+ "model.layers.26.mlp.gate_proj.weight": "model-00004-of-00004.safetensors",
181
+ "model.layers.26.mlp.up_proj.weight": "model-00004-of-00004.safetensors",
182
+ "model.layers.26.post_attention_layernorm.weight": "model-00004-of-00004.safetensors",
183
+ "model.layers.26.self_attn.k_proj.weight": "model-00004-of-00004.safetensors",
184
+ "model.layers.26.self_attn.o_proj.weight": "model-00004-of-00004.safetensors",
185
+ "model.layers.26.self_attn.q_proj.weight": "model-00004-of-00004.safetensors",
186
+ "model.layers.26.self_attn.v_proj.weight": "model-00004-of-00004.safetensors",
187
+ "model.layers.27.input_layernorm.weight": "model-00004-of-00004.safetensors",
188
+ "model.layers.27.mlp.down_proj.weight": "model-00004-of-00004.safetensors",
189
+ "model.layers.27.mlp.gate_proj.weight": "model-00004-of-00004.safetensors",
190
+ "model.layers.27.mlp.up_proj.weight": "model-00004-of-00004.safetensors",
191
+ "model.layers.27.post_attention_layernorm.weight": "model-00004-of-00004.safetensors",
192
+ "model.layers.27.self_attn.k_proj.weight": "model-00004-of-00004.safetensors",
193
+ "model.layers.27.self_attn.o_proj.weight": "model-00004-of-00004.safetensors",
194
+ "model.layers.27.self_attn.q_proj.weight": "model-00004-of-00004.safetensors",
195
+ "model.layers.27.self_attn.v_proj.weight": "model-00004-of-00004.safetensors",
196
+ "model.layers.3.input_layernorm.weight": "model-00001-of-00004.safetensors",
197
+ "model.layers.3.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
198
+ "model.layers.3.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
199
+ "model.layers.3.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
200
+ "model.layers.3.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
201
+ "model.layers.3.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
202
+ "model.layers.3.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
203
+ "model.layers.3.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
204
+ "model.layers.3.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
205
+ "model.layers.4.input_layernorm.weight": "model-00001-of-00004.safetensors",
206
+ "model.layers.4.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
207
+ "model.layers.4.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
208
+ "model.layers.4.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
209
+ "model.layers.4.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
210
+ "model.layers.4.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
211
+ "model.layers.4.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
212
+ "model.layers.4.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
213
+ "model.layers.4.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
214
+ "model.layers.5.input_layernorm.weight": "model-00001-of-00004.safetensors",
215
+ "model.layers.5.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
216
+ "model.layers.5.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
217
+ "model.layers.5.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
218
+ "model.layers.5.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
219
+ "model.layers.5.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
220
+ "model.layers.5.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
221
+ "model.layers.5.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
222
+ "model.layers.5.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
223
+ "model.layers.6.input_layernorm.weight": "model-00002-of-00004.safetensors",
224
+ "model.layers.6.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
225
+ "model.layers.6.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
226
+ "model.layers.6.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
227
+ "model.layers.6.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
228
+ "model.layers.6.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
229
+ "model.layers.6.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
230
+ "model.layers.6.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
231
+ "model.layers.6.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
232
+ "model.layers.7.input_layernorm.weight": "model-00002-of-00004.safetensors",
233
+ "model.layers.7.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
234
+ "model.layers.7.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
235
+ "model.layers.7.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
236
+ "model.layers.7.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
237
+ "model.layers.7.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
238
+ "model.layers.7.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
239
+ "model.layers.7.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
240
+ "model.layers.7.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
241
+ "model.layers.8.input_layernorm.weight": "model-00002-of-00004.safetensors",
242
+ "model.layers.8.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
243
+ "model.layers.8.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
244
+ "model.layers.8.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
245
+ "model.layers.8.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
246
+ "model.layers.8.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
247
+ "model.layers.8.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
248
+ "model.layers.8.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
249
+ "model.layers.8.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
250
+ "model.layers.9.input_layernorm.weight": "model-00002-of-00004.safetensors",
251
+ "model.layers.9.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
252
+ "model.layers.9.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
253
+ "model.layers.9.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
254
+ "model.layers.9.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
255
+ "model.layers.9.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
256
+ "model.layers.9.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
257
+ "model.layers.9.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
258
+ "model.layers.9.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
259
+ "model.norm.weight": "model-00004-of-00004.safetensors"
260
+ }
261
+ }
special_tokens_map.json ADDED
@@ -0,0 +1,46 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "additional_special_tokens": [
3
+ {
4
+ "content": "<start_of_turn>",
5
+ "lstrip": false,
6
+ "normalized": false,
7
+ "rstrip": false,
8
+ "single_word": false
9
+ },
10
+ {
11
+ "content": "<end_of_turn>",
12
+ "lstrip": false,
13
+ "normalized": false,
14
+ "rstrip": false,
15
+ "single_word": false
16
+ }
17
+ ],
18
+ "bos_token": {
19
+ "content": "<bos>",
20
+ "lstrip": false,
21
+ "normalized": false,
22
+ "rstrip": false,
23
+ "single_word": false
24
+ },
25
+ "eos_token": {
26
+ "content": "<eos>",
27
+ "lstrip": false,
28
+ "normalized": false,
29
+ "rstrip": false,
30
+ "single_word": false
31
+ },
32
+ "pad_token": {
33
+ "content": "<pad>",
34
+ "lstrip": false,
35
+ "normalized": false,
36
+ "rstrip": false,
37
+ "single_word": false
38
+ },
39
+ "unk_token": {
40
+ "content": "<unk>",
41
+ "lstrip": false,
42
+ "normalized": false,
43
+ "rstrip": false,
44
+ "single_word": false
45
+ }
46
+ }
tokenizer.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:05e97791a5e007260de1db7e1692e53150e08cea481e2bf25435553380c147ee
3
+ size 17477929
tokenizer.model ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6969e64047744a44bb3abfb5c50f8de0f7ed8b571d5444426ef931f651d1a0ef
3
+ size 4241111
tokenizer_config.json ADDED
@@ -0,0 +1,70 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_bos_token": true,
3
+ "add_eos_token": false,
4
+ "added_tokens_decoder": {
5
+ "0": {
6
+ "content": "<pad>",
7
+ "lstrip": false,
8
+ "normalized": false,
9
+ "rstrip": false,
10
+ "single_word": false,
11
+ "special": true
12
+ },
13
+ "1": {
14
+ "content": "<eos>",
15
+ "lstrip": false,
16
+ "normalized": false,
17
+ "rstrip": false,
18
+ "single_word": false,
19
+ "special": true
20
+ },
21
+ "2": {
22
+ "content": "<bos>",
23
+ "lstrip": false,
24
+ "normalized": false,
25
+ "rstrip": false,
26
+ "single_word": false,
27
+ "special": true
28
+ },
29
+ "3": {
30
+ "content": "<unk>",
31
+ "lstrip": false,
32
+ "normalized": false,
33
+ "rstrip": false,
34
+ "single_word": false,
35
+ "special": true
36
+ },
37
+ "106": {
38
+ "content": "<start_of_turn>",
39
+ "lstrip": false,
40
+ "normalized": false,
41
+ "rstrip": false,
42
+ "single_word": false,
43
+ "special": true
44
+ },
45
+ "107": {
46
+ "content": "<end_of_turn>",
47
+ "lstrip": false,
48
+ "normalized": false,
49
+ "rstrip": false,
50
+ "single_word": false,
51
+ "special": true
52
+ }
53
+ },
54
+ "additional_special_tokens": [
55
+ "<start_of_turn>",
56
+ "<end_of_turn>"
57
+ ],
58
+ "bos_token": "<bos>",
59
+ "chat_template": "{% if messages[0]['role'] == 'system' %}{{ raise_exception('System role not supported') }}{% endif %}{% for message in messages %}{% if (message['role'] == 'user') != (loop.index0 % 2 == 0) %}{{ raise_exception('Conversation roles must alternate user/assistant/user/assistant/...') }}{% endif %}{% if (message['role'] == 'assistant') %}{% set role = 'model' %}{% else %}{% set role = message['role'] %}{% endif %}{{ '<start_of_turn>' + role + '\n' + message['content'] | trim + '<end_of_turn>\n' }}{% endfor %}{% if add_generation_prompt %}{{'<start_of_turn>model\n'}}{% endif %}",
60
+ "clean_up_tokenization_spaces": false,
61
+ "eos_token": "<eos>",
62
+ "legacy": null,
63
+ "model_max_length": 1000000000000000019884624838656,
64
+ "pad_token": "<pad>",
65
+ "sp_model_kwargs": {},
66
+ "spaces_between_special_tokens": false,
67
+ "tokenizer_class": "GemmaTokenizer",
68
+ "unk_token": "<unk>",
69
+ "use_default_system_prompt": false
70
+ }