anudeepadi commited on
Commit
3bc0a63
โ€ข
1 Parent(s): c01a7ad

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +36 -27
README.md CHANGED
@@ -1,41 +1,50 @@
1
- Telugu Gemma 7B Model
2
- This is a Telugu language model based on the Gemma 7B model, finetuned on a dataset of Telugu instructions and outputs using the Alpaca format. The model was trained 2x faster using Unsloth and Huggingface's TRL library.
3
-
4
- Model Details
5
- Developed by: Anudeep Adi
6
- Model architecture: Gemma 7B
7
- Language: Telugu
8
- License: Apache 2.0
9
- Base model: unsloth/gemma-7b-bnb-4bit
10
- Dataset: telugu_teknium_GPTeacher_general_instruct_filtered_romanized
11
- Finetuning steps: 60
12
- Finetuning dataset size: 43,614 examples
13
- Tags: text-generation-inference, transformers, unsloth, gemma, trl
14
- Usage
15
- The model can be used for Telugu language text generation tasks following an instruction and optional input prompt in the Alpaca format:
16
-
17
-
18
- Copy code
 
 
 
19
  Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.
20
 
21
  ### Instruction:
22
  {instruction}
23
 
24
- ### Input:
25
  {input}
26
 
27
  ### Response:
28
  {output}
29
- Replace {instruction}, {input} and {output} with the relevant text. The model will generate a continuation for the {output} field.
 
 
 
 
 
 
 
 
30
 
31
- Training Procedure
32
- The model was finetuned on the telugu_teknium_GPTeacher_general_instruct_filtered_romanized dataset which contains 43,614 examples of Telugu instructions and outputs in the Alpaca format.
33
 
34
- Finetuning was performed for 60 steps using Unsloth and the TRL library from Huggingface to enable 2x faster training. LoRA adapters were added to enable efficient finetuning of the 7B parameter model. Mixed precision training with bfloat16 was used.
35
 
36
- Unsloth also provides optimized inference code to run the model 2x faster.
37
 
38
- <img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>
39
 
40
- Limitations
41
- As the model was trained on a relatively small Telugu dataset, its knowledge and conversational abilities in Telugu are limited compared to larger language models. It may make factual errors or inconsistent statements. The model should be used as an experimental prototype only.
 
1
+ # ๐Ÿ‡ฎ๐Ÿ‡ณ Telugu Gemma 7B Model ๐Ÿ’ฌ
2
+
3
+ Welcome to the Telugu Gemma 7B model! This model brings the power of the Gemma 7B architecture to the Telugu language, enabling engaging conversational AI in Telugu. ๐ŸŒŸ
4
+
5
+ ## ๐Ÿš€ Model Highlights
6
+
7
+ - **Developed by:** Anudeep Adi ๐Ÿ‘จโ€๐Ÿ’ป
8
+ - **Model architecture:** Gemma 7B ๐Ÿง 
9
+ - **Language:** Telugu ๐Ÿ‡ฎ๐Ÿ‡ณ
10
+ - **License:** Apache 2.0 โš–๏ธ
11
+ - **Base model:** unsloth/gemma-7b-bnb-4bit ๐ŸŒฟ
12
+ - **Dataset:** telugu_teknium_GPTeacher_general_instruct_filtered_romanized ๐Ÿ“š
13
+ - **Finetuning steps:** 60 ๐Ÿƒโ€โ™‚๏ธ
14
+ - **Finetuning dataset size:** 43,614 examples ๐Ÿ“ˆ
15
+ - **Tags:** text-generation-inference, transformers, unsloth, gemma, trl ๐Ÿท๏ธ
16
+
17
+ ## ๐Ÿ—ฃ๏ธ Usage
18
+
19
+ Want to engage in Telugu conversations with an AI? This model makes it easy! Simply provide an instruction and optional input prompt in the Alpaca format:
20
+
21
+ ```
22
  Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.
23
 
24
  ### Instruction:
25
  {instruction}
26
 
27
+ ### Input:
28
  {input}
29
 
30
  ### Response:
31
  {output}
32
+ ```
33
+
34
+ Replace `{instruction}`, `{input}` and `{output}` with your desired Telugu text. Watch in amazement as the model generates a fluent continuation for the `{output}` field! โœจ
35
+
36
+ ## ๐Ÿ‹๏ธโ€โ™€๏ธ Training Procedure
37
+
38
+ This model didn't skip leg day! It was finetuned on the telugu_teknium_GPTeacher_general_instruct_filtered_romanized dataset containing a whopping 43,614 examples of Telugu instructions and outputs. ๐Ÿ’ช
39
+
40
+ But how did we make training lightning fast? By using Unsloth and the TRL library from Huggingface, enabling 2x faster training! โšก We also added LoRA adapters for efficient finetuning of the 7B parameter model and used mixed precision training with bfloat16.
41
 
42
+ And the best part? Unsloth provides optimized inference code to run the model 2x faster. Talk about having your cake and eating it too! ๐Ÿฐ
 
43
 
44
+ [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
45
 
46
+ ## โš ๏ธ Limitations
47
 
48
+ While this model is impressive, it's not perfect. It was trained on a relatively small Telugu dataset, so its knowledge and conversational abilities in Telugu are limited compared to larger language models. It may occasionally make factual errors or inconsistent statements.
49
 
50
+ So use this model as an experimental prototype and have fun chatting, but don't rely on it for mission-critical Telugu conversations just yet! ๐Ÿ˜‰