samta-kamboj commited on
Commit
48fe57e
·
verified ·
1 Parent(s): 2b9ce0c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +30 -30
README.md CHANGED
@@ -28,7 +28,7 @@ We hope this extensive release will accelerate research in Arabic NLP, and enabl
28
 
29
  ## Jais Family Details
30
 
31
- - **Developed by:** Core42 (Inception), Cerebras Systems.
32
  - **Language(s):** (NLP): Arabic (MSA) and English.
33
  - **Input:** Text only data.
34
  - **Output:** Model generates text.
@@ -38,19 +38,19 @@ We hope this extensive release will accelerate research in Arabic NLP, and enabl
38
 
39
  | **Pre-trained Model** | **Fine-tuned Model** | **Size (Parameters)** | **Context length (Tokens)** |
40
  |:---------------------|:--------|:-------|:-------|
41
- | [jais-family-30b-16k](https://huggingface.co/core42/jais-family-30b-16k) | [Jais-family-30b-16k-chat](https://huggingface.co/core42/jais-family-30b-16k-chat) | 30B | 16,384 |
42
- | [jais-family-30b-8k](https://huggingface.co/core42/jais-family-30b-8k) | [Jais-family-30b-8k-chat](https://huggingface.co/core42/jais-family-30b-8k-chat) | 30B | 8,192 |
43
- | [jais-family-13b ](https://huggingface.co/core42/jais-family-13b) | [Jais-family-13b-chat](https://huggingface.co/core42/jais-family-13b-chat) | 13B | 2,048 |
44
- | [jais-family-6p7b](https://huggingface.co/core42/jais-family-6p7b) | [Jais-family-6p7b-chat](https://huggingface.co/core42/jais-family-6p7b-chat) | 6.7B | 2,048 |
45
- | [jais-family-2p7b](https://huggingface.co/core42/jais-family-2p7b) | [Jais-family-2p7b-chat](https://huggingface.co/core42/jais-family-2p7b-chat) | 2.7B | 2,048 |
46
- | [jais-family-1p3b](https://huggingface.co/core42/jais-family-1p3b) | [Jais-family-1p3b-chat](https://huggingface.co/core42/jais-family-1p3b-chat) | 1.3B | 2,048 |
47
- | [jais-family-590m](https://huggingface.co/core42/jais-family-590m) | [Jais-family-590m-chat](https://huggingface.co/core42/jais-family-590m-chat) | 590M | 2,048 |
48
 
49
  | **Adapted pre-trained Model** | **Fine-tuned Model** | **Size (Parameters)** | **Context length (Tokens)** |
50
  |:---------------------|:--------|:-------|:-------|
51
- | [jais-adapted-70b](https://huggingface.co/core42/jais-adapted-70b) | [Jais-adapted-70b-chat](https://huggingface.co/core42/jais-adapted-70b-chat) | 70B | 4,096 |
52
- | [jais-adapted-13b](https://huggingface.co/core42/jais-adapted-13b) | [Jais-adapted-13b-chat](https://huggingface.co/core42/jais-adapted-13b-chat) | 13B | 4,096 |
53
- | [jais-adapted-7b](https://huggingface.co/core42/jais-adapted-7b) | [Jais-adapted-7b-chat](https://huggingface.co/core42/jais-adapted-7b-chat) | 7B | 4,096 |
54
 
55
  ### Model Architecture:
56
  <a name="model-architecture"></a>
@@ -73,10 +73,10 @@ Below is sample code to use the model. Note that the model requires a custom mod
73
  import torch
74
  from transformers import AutoTokenizer, AutoModelForCausalLM
75
 
76
- model_path = "core42/jais-family-590m-chat"
77
 
78
- prompt_eng = "### Instruction:Your name is 'Jais', and you are named after Jebel Jais, the highest mountain in UAE. You were made by 'Core42' in the UAE. You are a helpful, respectful, and honest assistant. Always answer as helpfully as possible, while being safe. Complete the conversation between [|Human|] and [|AI|]:\n### Input: [|Human|] {Question}\n[|AI|]\n### Response :"
79
- prompt_ar = "### Instruction:اسمك \"جيس\" وسميت على اسم جبل جيس اعلى جبل في الامارات. تم بنائك بواسطة Core42 في الإمارات. أنت مساعد مفيد ومحترم وصادق. أجب دائمًا بأكبر قدر ممكن من المساعدة، مع الحفاظ على البقاء أمناً. أكمل المحادثة بين [|Human|] و[|AI|] :\n### Input:[|Human|] {Question}\n[|AI|]\n### Response :"
80
 
81
  device = "cuda" if torch.cuda.is_available() else "cpu"
82
 
@@ -137,16 +137,16 @@ We extensively preprocess and deduplicate the training data. For Arabic, we used
137
 
138
  | **Pre-trained model** | **English data (tokens)** | **Arabic data (tokens)** | **Code data (tokens)** | **Total data (tokens)** |
139
  |-------------------------|---------------------------|--------------------------|------------------------|------------------------|
140
- | [jais-family-30b-16k](https://huggingface.co/core42/jais-family-30b-16k) | 980B | 490B | 196B | 1666B |
141
- | [jais-family-30b-8k](https://huggingface.co/core42/jais-family-30b-8k) | 882B | 441B | 177B | 1500B |
142
- | [jais-family-13b ](https://huggingface.co/core42/jais-family-13b) | 283B | 141B | 56B | 480B |
143
- | [jais-family-6p7b](https://huggingface.co/core42/jais-family-6p7b) | 283B | 141B | 56B | 480B |
144
- | [jais-family-2p7b](https://huggingface.co/core42/jais-family-2p7b) | 283B | 141B | 56B | 480B |
145
- | [jais-family-1p3b](https://huggingface.co/core42/jais-family-1p3b) | 283B | 141B | 56B | 480B |
146
- | [jais-family-590m](https://huggingface.co/core42/jais-family-590m) | 283B | 141B | 56B | 480B |
147
- | [jais-adapted-70b](https://huggingface.co/core42/jais-adapted-70b) | 33B | 334B | 4B | 371B |
148
- | [jais-adapted-13b](https://huggingface.co/core42/jais-adapted-13b) | 127B | 140B | 13B | 280B |
149
- | [jais-adapted-7b](https://huggingface.co/core42/jais-adapted-7b) | 18B | 19B | 2B | 39B |
150
 
151
  ### Finetuning data
152
 
@@ -280,14 +280,14 @@ English prompts were translated to Arabic by our in-house linguists.
280
  In the following, we compare the models in this release of the jais family against previously released versions:
281
 
282
  <p align="center">
283
- <img src="https://huggingface.co/core42/jais-family-30b-16k-chat/resolve/main/jais.png" alt="Jais-adapted GPT-4">
284
  </p>
285
  <p align="center">
286
  <em>GPT-4-as-a-judge evaluation of Jais in Arabic and English. Jais family models are significantly better than previous Jais at generations in both languages. </em>
287
  </p>
288
 
289
  <p align="center">
290
- <img src="https://huggingface.co/core42/jais-family-30b-16k-chat/resolve/main/jais-adapted.png" alt="Jais-adapted GPT-4">
291
  </p>
292
  <p align="center">
293
  <em>GPT-4-as-a-judge evaluation of adapted Jais in Arabic and English. The generation quality of Arabic is significantly enhanced, while achieving improvement in English when compared to Llama-2 instruct. </em>
@@ -296,7 +296,7 @@ In the following, we compare the models in this release of the jais family again
296
  Besides pairwise comparison, we also perform MT-bench style single-answer grading on a scale of 1 to 10.
297
 
298
  <p align="center">
299
- <img src="https://huggingface.co/core42/jais-family-30b-16k-chat/resolve/main/mt_bench.png" alt="MT-bench">
300
  </p>
301
  <p align="center">
302
  <em>MT-bench style single-answer grading evaluation of Jais and adapted Jais in Arabic and English. Comparisons are made between select corresponding models from earlier releases. The quality ratings of responses are generally improved, with significant enhancements in Arabic.</em>
@@ -347,7 +347,7 @@ The following are some example scenarios where the model should not be used.
347
 
348
  <!-- This section is meant to convey both technical and sociotechnical limitations. -->
349
 
350
- The Jais family is trained on publicly available data which was in part curated by Core42. We have employed different techniques to reduce bias in the model. While efforts have been made to minimize biases, it is likely that the model, as with all LLM models, will exhibit some bias.
351
 
352
  The fine-tuned variants are trained as an AI assistant for Arabic and English speakers. Chat models are limited to produce responses for queries in these two languages and may not produce appropriate responses to other language queries.
353
 
@@ -374,8 +374,8 @@ Through this release, we aim to make LLMs more accessible to Arabic NLP research
374
 
375
  @article{jaisfamilymodelcard,
376
  title={Jais Family Model Card},
377
- author={Core42 (Inception)},
378
  year={2024},
379
- url = {https://huggingface.co/core42/jais-family-30b-16k-chat/blob/main/README.md}
380
  }
381
  ```
 
28
 
29
  ## Jais Family Details
30
 
31
+ - **Developed by:** Inception, Cerebras Systems.
32
  - **Language(s):** (NLP): Arabic (MSA) and English.
33
  - **Input:** Text only data.
34
  - **Output:** Model generates text.
 
38
 
39
  | **Pre-trained Model** | **Fine-tuned Model** | **Size (Parameters)** | **Context length (Tokens)** |
40
  |:---------------------|:--------|:-------|:-------|
41
+ | [jais-family-30b-16k](https://huggingface.co/inceptionai/jais-family-30b-16k) | [Jais-family-30b-16k-chat](https://huggingface.co/inceptionai/jais-family-30b-16k-chat) | 30B | 16,384 |
42
+ | [jais-family-30b-8k](https://huggingface.co/inceptionai/jais-family-30b-8k) | [Jais-family-30b-8k-chat](https://huggingface.co/inceptionai/jais-family-30b-8k-chat) | 30B | 8,192 |
43
+ | [jais-family-13b ](https://huggingface.co/inceptionai/jais-family-13b) | [Jais-family-13b-chat](https://huggingface.co/inceptionai/jais-family-13b-chat) | 13B | 2,048 |
44
+ | [jais-family-6p7b](https://huggingface.co/inceptionai/jais-family-6p7b) | [Jais-family-6p7b-chat](https://huggingface.co/inceptionai/jais-family-6p7b-chat) | 6.7B | 2,048 |
45
+ | [jais-family-2p7b](https://huggingface.co/inceptionai/jais-family-2p7b) | [Jais-family-2p7b-chat](https://huggingface.co/inceptionai/jais-family-2p7b-chat) | 2.7B | 2,048 |
46
+ | [jais-family-1p3b](https://huggingface.co/inceptionai/jais-family-1p3b) | [Jais-family-1p3b-chat](https://huggingface.co/inceptionai/jais-family-1p3b-chat) | 1.3B | 2,048 |
47
+ | [jais-family-590m](https://huggingface.co/inceptionai/jais-family-590m) | [Jais-family-590m-chat](https://huggingface.co/inceptionai/jais-family-590m-chat) | 590M | 2,048 |
48
 
49
  | **Adapted pre-trained Model** | **Fine-tuned Model** | **Size (Parameters)** | **Context length (Tokens)** |
50
  |:---------------------|:--------|:-------|:-------|
51
+ | [jais-adapted-70b](https://huggingface.co/inceptionai/jais-adapted-70b) | [Jais-adapted-70b-chat](https://huggingface.co/inceptionai/jais-adapted-70b-chat) | 70B | 4,096 |
52
+ | [jais-adapted-13b](https://huggingface.co/inceptionai/jais-adapted-13b) | [Jais-adapted-13b-chat](https://huggingface.co/inceptionai/jais-adapted-13b-chat) | 13B | 4,096 |
53
+ | [jais-adapted-7b](https://huggingface.co/inceptionai/jais-adapted-7b) | [Jais-adapted-7b-chat](https://huggingface.co/inceptionai/jais-adapted-7b-chat) | 7B | 4,096 |
54
 
55
  ### Model Architecture:
56
  <a name="model-architecture"></a>
 
73
  import torch
74
  from transformers import AutoTokenizer, AutoModelForCausalLM
75
 
76
+ model_path = "inceptionai/jais-family-590m-chat"
77
 
78
+ prompt_eng = "### Instruction:Your name is 'Jais', and you are named after Jebel Jais, the highest mountain in UAE. You were made by 'Inception' in the UAE. You are a helpful, respectful, and honest assistant. Always answer as helpfully as possible, while being safe. Complete the conversation between [|Human|] and [|AI|]:\n### Input: [|Human|] {Question}\n[|AI|]\n### Response :"
79
+ prompt_ar = "### Instruction:اسمك \"جيس\" وسميت على اسم جبل جيس اعلى جبل في الامارات. تم بنائك بواسطة Inception في الإمارات. أنت مساعد مفيد ومحترم وصادق. أجب دائمًا بأكبر قدر ممكن من المساعدة، مع الحفاظ على البقاء أمناً. أكمل المحادثة بين [|Human|] و[|AI|] :\n### Input:[|Human|] {Question}\n[|AI|]\n### Response :"
80
 
81
  device = "cuda" if torch.cuda.is_available() else "cpu"
82
 
 
137
 
138
  | **Pre-trained model** | **English data (tokens)** | **Arabic data (tokens)** | **Code data (tokens)** | **Total data (tokens)** |
139
  |-------------------------|---------------------------|--------------------------|------------------------|------------------------|
140
+ | [jais-family-30b-16k](https://huggingface.co/inceptionai/jais-family-30b-16k) | 980B | 490B | 196B | 1666B |
141
+ | [jais-family-30b-8k](https://huggingface.co/inceptionai/jais-family-30b-8k) | 882B | 441B | 177B | 1500B |
142
+ | [jais-family-13b ](https://huggingface.co/inceptionai/jais-family-13b) | 283B | 141B | 56B | 480B |
143
+ | [jais-family-6p7b](https://huggingface.co/inceptionai/jais-family-6p7b) | 283B | 141B | 56B | 480B |
144
+ | [jais-family-2p7b](https://huggingface.co/inceptionai/jais-family-2p7b) | 283B | 141B | 56B | 480B |
145
+ | [jais-family-1p3b](https://huggingface.co/inceptionai/jais-family-1p3b) | 283B | 141B | 56B | 480B |
146
+ | [jais-family-590m](https://huggingface.co/inceptionai/jais-family-590m) | 283B | 141B | 56B | 480B |
147
+ | [jais-adapted-70b](https://huggingface.co/inceptionai/jais-adapted-70b) | 33B | 334B | 4B | 371B |
148
+ | [jais-adapted-13b](https://huggingface.co/inceptionai/jais-adapted-13b) | 127B | 140B | 13B | 280B |
149
+ | [jais-adapted-7b](https://huggingface.co/inceptionai/jais-adapted-7b) | 18B | 19B | 2B | 39B |
150
 
151
  ### Finetuning data
152
 
 
280
  In the following, we compare the models in this release of the jais family against previously released versions:
281
 
282
  <p align="center">
283
+ <img src="https://huggingface.co/inceptionai/jais-family-30b-16k-chat/resolve/main/jais.png" alt="Jais-adapted GPT-4">
284
  </p>
285
  <p align="center">
286
  <em>GPT-4-as-a-judge evaluation of Jais in Arabic and English. Jais family models are significantly better than previous Jais at generations in both languages. </em>
287
  </p>
288
 
289
  <p align="center">
290
+ <img src="https://huggingface.co/inceptionai/jais-family-30b-16k-chat/resolve/main/jais-adapted.png" alt="Jais-adapted GPT-4">
291
  </p>
292
  <p align="center">
293
  <em>GPT-4-as-a-judge evaluation of adapted Jais in Arabic and English. The generation quality of Arabic is significantly enhanced, while achieving improvement in English when compared to Llama-2 instruct. </em>
 
296
  Besides pairwise comparison, we also perform MT-bench style single-answer grading on a scale of 1 to 10.
297
 
298
  <p align="center">
299
+ <img src="https://huggingface.co/inceptionai/jais-family-30b-16k-chat/resolve/main/mt_bench.png" alt="MT-bench">
300
  </p>
301
  <p align="center">
302
  <em>MT-bench style single-answer grading evaluation of Jais and adapted Jais in Arabic and English. Comparisons are made between select corresponding models from earlier releases. The quality ratings of responses are generally improved, with significant enhancements in Arabic.</em>
 
347
 
348
  <!-- This section is meant to convey both technical and sociotechnical limitations. -->
349
 
350
+ The Jais family is trained on publicly available data which was in part curated by Inception. We have employed different techniques to reduce bias in the model. While efforts have been made to minimize biases, it is likely that the model, as with all LLM models, will exhibit some bias.
351
 
352
  The fine-tuned variants are trained as an AI assistant for Arabic and English speakers. Chat models are limited to produce responses for queries in these two languages and may not produce appropriate responses to other language queries.
353
 
 
374
 
375
  @article{jaisfamilymodelcard,
376
  title={Jais Family Model Card},
377
+ author={Inception},
378
  year={2024},
379
+ url = {https://huggingface.co/inceptionai/jais-family-30b-16k-chat/blob/main/README.md}
380
  }
381
  ```