yifanxie commited on
Commit
27573dc
1 Parent(s): 8d393f0

Upload model card

Browse files
Files changed (1) hide show
  1. README.md +192 -196
README.md CHANGED
@@ -1,199 +1,195 @@
1
  ---
 
 
2
  library_name: transformers
3
- tags: []
 
 
 
 
 
 
4
  ---
5
-
6
- # Model Card for Model ID
7
-
8
- <!-- Provide a quick summary of what the model is/does. -->
9
-
10
-
11
-
12
- ## Model Details
13
-
14
- ### Model Description
15
-
16
- <!-- Provide a longer summary of what this model is. -->
17
-
18
- This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
19
-
20
- - **Developed by:** [More Information Needed]
21
- - **Funded by [optional]:** [More Information Needed]
22
- - **Shared by [optional]:** [More Information Needed]
23
- - **Model type:** [More Information Needed]
24
- - **Language(s) (NLP):** [More Information Needed]
25
- - **License:** [More Information Needed]
26
- - **Finetuned from model [optional]:** [More Information Needed]
27
-
28
- ### Model Sources [optional]
29
-
30
- <!-- Provide the basic links for the model. -->
31
-
32
- - **Repository:** [More Information Needed]
33
- - **Paper [optional]:** [More Information Needed]
34
- - **Demo [optional]:** [More Information Needed]
35
-
36
- ## Uses
37
-
38
- <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
39
-
40
- ### Direct Use
41
-
42
- <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
43
-
44
- [More Information Needed]
45
-
46
- ### Downstream Use [optional]
47
-
48
- <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
49
-
50
- [More Information Needed]
51
-
52
- ### Out-of-Scope Use
53
-
54
- <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
55
-
56
- [More Information Needed]
57
-
58
- ## Bias, Risks, and Limitations
59
-
60
- <!-- This section is meant to convey both technical and sociotechnical limitations. -->
61
-
62
- [More Information Needed]
63
-
64
- ### Recommendations
65
-
66
- <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
67
-
68
- Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
69
-
70
- ## How to Get Started with the Model
71
-
72
- Use the code below to get started with the model.
73
-
74
- [More Information Needed]
75
-
76
- ## Training Details
77
-
78
- ### Training Data
79
-
80
- <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
81
-
82
- [More Information Needed]
83
-
84
- ### Training Procedure
85
-
86
- <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
87
-
88
- #### Preprocessing [optional]
89
-
90
- [More Information Needed]
91
-
92
-
93
- #### Training Hyperparameters
94
-
95
- - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
96
-
97
- #### Speeds, Sizes, Times [optional]
98
-
99
- <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
100
-
101
- [More Information Needed]
102
-
103
- ## Evaluation
104
-
105
- <!-- This section describes the evaluation protocols and provides the results. -->
106
-
107
- ### Testing Data, Factors & Metrics
108
-
109
- #### Testing Data
110
-
111
- <!-- This should link to a Dataset Card if possible. -->
112
-
113
- [More Information Needed]
114
-
115
- #### Factors
116
-
117
- <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
118
-
119
- [More Information Needed]
120
-
121
- #### Metrics
122
-
123
- <!-- These are the evaluation metrics being used, ideally with a description of why. -->
124
-
125
- [More Information Needed]
126
-
127
- ### Results
128
-
129
- [More Information Needed]
130
-
131
- #### Summary
132
-
133
-
134
-
135
- ## Model Examination [optional]
136
-
137
- <!-- Relevant interpretability work for the model goes here -->
138
-
139
- [More Information Needed]
140
-
141
- ## Environmental Impact
142
-
143
- <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
144
-
145
- Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
146
-
147
- - **Hardware Type:** [More Information Needed]
148
- - **Hours used:** [More Information Needed]
149
- - **Cloud Provider:** [More Information Needed]
150
- - **Compute Region:** [More Information Needed]
151
- - **Carbon Emitted:** [More Information Needed]
152
-
153
- ## Technical Specifications [optional]
154
-
155
- ### Model Architecture and Objective
156
-
157
- [More Information Needed]
158
-
159
- ### Compute Infrastructure
160
-
161
- [More Information Needed]
162
-
163
- #### Hardware
164
-
165
- [More Information Needed]
166
-
167
- #### Software
168
-
169
- [More Information Needed]
170
-
171
- ## Citation [optional]
172
-
173
- <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
174
-
175
- **BibTeX:**
176
-
177
- [More Information Needed]
178
-
179
- **APA:**
180
-
181
- [More Information Needed]
182
-
183
- ## Glossary [optional]
184
-
185
- <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
186
-
187
- [More Information Needed]
188
-
189
- ## More Information [optional]
190
-
191
- [More Information Needed]
192
-
193
- ## Model Card Authors [optional]
194
-
195
- [More Information Needed]
196
-
197
- ## Model Card Contact
198
-
199
- [More Information Needed]
 
1
  ---
2
+ language:
3
+ - en
4
  library_name: transformers
5
+ tags:
6
+ - gpt
7
+ - llm
8
+ - large language model
9
+ - h2o-llmstudio
10
+ inference: false
11
+ thumbnail: https://h2o.ai/etc.clientlibs/h2o/clientlibs/clientlib-site/resources/images/favicon.ico
12
  ---
13
+ # Model Card
14
+ ## Summary
15
+
16
+ This model was trained using [H2O LLM Studio](https://github.com/h2oai/h2o-llmstudio).
17
+ - Base model: [google/gemma-2b](https://huggingface.co/google/gemma-2b)
18
+
19
+
20
+ ## Usage
21
+
22
+ To use the model with the `transformers` library on a machine with GPUs, first make sure you have the `transformers` library installed.
23
+
24
+ ```bash
25
+ pip install transformers==4.40.2
26
+ ```
27
+
28
+ Also make sure you are providing your huggingface token to the pipeline if the model is lying in a private repo.
29
+
30
+ - Either leave `token=True` in the `pipeline` and login to hugginface_hub by running
31
+
32
+ ```python
33
+ import huggingface_hub
34
+ huggingface_hub.login(<ACCESS_TOKEN>)
35
+ ```
36
+
37
+ - Or directly pass your <ACCESS_TOKEN> to `token` in the `pipeline`
38
+
39
+ ```python
40
+ from transformers import pipeline
41
+
42
+ generate_text = pipeline(
43
+ model="yifanxie/able-ara-1",
44
+ torch_dtype="auto",
45
+ trust_remote_code=True,
46
+ use_fast=True,
47
+ device_map={"": "cuda:0"},
48
+ token=True,
49
+ )
50
+
51
+ # generate configuration can be modified to your needs
52
+ # generate_text.model.generation_config.min_new_tokens = 2
53
+ # generate_text.model.generation_config.max_new_tokens = 486
54
+ # generate_text.model.generation_config.do_sample = False
55
+ # generate_text.model.generation_config.num_beams = 1
56
+ # generate_text.model.generation_config.temperature = float(0.0)
57
+ # generate_text.model.generation_config.repetition_penalty = float(1.0)
58
+
59
+ messages = [
60
+ {
61
+ "role": "system",
62
+ "content": "You are a friendly and polite chatbot.",
63
+ },
64
+ {"role": "user", "content": "Hi, how are you?"},
65
+ {"role": "assistant", "content": "I'm doing great, how about you?"},
66
+ {"role": "user", "content": "Why is drinking water so healthy?"},
67
+ ]
68
+
69
+ res = generate_text(
70
+ messages,
71
+ renormalize_logits=True
72
+ )
73
+ print(res[0]["generated_text"][-1]['content'])
74
+ ```
75
+
76
+ You can print a sample prompt after applying chat template to see how it is feed to the tokenizer:
77
+
78
+ ```python
79
+ print(generate_text.tokenizer.apply_chat_template(
80
+ messages,
81
+ tokenize=False,
82
+ add_generation_prompt=True,
83
+ ))
84
+ ```
85
+
86
+ You may also construct the pipeline from the loaded model and tokenizer yourself and consider the preprocessing steps:
87
+
88
+ ```python
89
+ from transformers import AutoModelForCausalLM, AutoTokenizer
90
+
91
+ model_name = "yifanxie/able-ara-1" # either local folder or huggingface model name
92
+ # Important: The prompt needs to be in the same format the model was trained with.
93
+ # You can find an example prompt in the experiment logs.
94
+ messages = [
95
+ {
96
+ "role": "system",
97
+ "content": "You are a friendly and polite chatbot.",
98
+ },
99
+ {"role": "user", "content": "Hi, how are you?"},
100
+ {"role": "assistant", "content": "I'm doing great, how about you?"},
101
+ {"role": "user", "content": "Why is drinking water so healthy?"},
102
+ ]
103
+
104
+ tokenizer = AutoTokenizer.from_pretrained(
105
+ model_name,
106
+ use_fast=True,
107
+ trust_remote_code=True,
108
+ )
109
+ model = AutoModelForCausalLM.from_pretrained(
110
+ model_name,
111
+ torch_dtype="auto",
112
+ device_map={"": "cuda:0"},
113
+ trust_remote_code=True,
114
+ )
115
+ model.cuda().eval()
116
+
117
+ # generate configuration can be modified to your needs
118
+ # model.generation_config.min_new_tokens = 2
119
+ # model.generation_config.max_new_tokens = 486
120
+ # model.generation_config.do_sample = False
121
+ # model.generation_config.num_beams = 1
122
+ # model.generation_config.temperature = float(0.0)
123
+ # model.generation_config.repetition_penalty = float(1.0)
124
+
125
+ inputs = tokenizer.apply_chat_template(
126
+ messages,
127
+ tokenize=True,
128
+ add_generation_prompt=True,
129
+ return_tensors="pt",
130
+ return_dict=True,
131
+ ).to("cuda")
132
+
133
+ tokens = model.generate(
134
+ input_ids=inputs["input_ids"],
135
+ attention_mask=inputs["attention_mask"],
136
+ renormalize_logits=True
137
+ )[0]
138
+
139
+ tokens = tokens[inputs["input_ids"].shape[1]:]
140
+ answer = tokenizer.decode(tokens, skip_special_tokens=True)
141
+ print(answer)
142
+ ```
143
+
144
+ ## Quantization and sharding
145
+
146
+ You can load the models using quantization by specifying ```load_in_8bit=True``` or ```load_in_4bit=True```. Also, sharding on multiple GPUs is possible by setting ```device_map=auto```.
147
+
148
+ ## Model Architecture
149
+
150
+ ```
151
+ GemmaForCausalLM(
152
+ (model): GemmaModel(
153
+ (embed_tokens): Embedding(256000, 2048, padding_idx=0)
154
+ (layers): ModuleList(
155
+ (0-17): 18 x GemmaDecoderLayer(
156
+ (self_attn): GemmaSdpaAttention(
157
+ (q_proj): Linear(in_features=2048, out_features=2048, bias=False)
158
+ (k_proj): Linear(in_features=2048, out_features=256, bias=False)
159
+ (v_proj): Linear(in_features=2048, out_features=256, bias=False)
160
+ (o_proj): Linear(in_features=2048, out_features=2048, bias=False)
161
+ (rotary_emb): GemmaRotaryEmbedding()
162
+ )
163
+ (mlp): GemmaMLP(
164
+ (gate_proj): Linear(in_features=2048, out_features=16384, bias=False)
165
+ (up_proj): Linear(in_features=2048, out_features=16384, bias=False)
166
+ (down_proj): Linear(in_features=16384, out_features=2048, bias=False)
167
+ (act_fn): PytorchGELUTanh()
168
+ )
169
+ (input_layernorm): GemmaRMSNorm()
170
+ (post_attention_layernorm): GemmaRMSNorm()
171
+ )
172
+ )
173
+ (norm): GemmaRMSNorm()
174
+ )
175
+ (lm_head): Linear(in_features=2048, out_features=256000, bias=False)
176
+ )
177
+ ```
178
+
179
+ ## Model Configuration
180
+
181
+ This model was trained using H2O LLM Studio and with the configuration in [cfg.yaml](cfg.yaml). Visit [H2O LLM Studio](https://github.com/h2oai/h2o-llmstudio) to learn how to train your own large language models.
182
+
183
+
184
+ ## Disclaimer
185
+
186
+ Please read this disclaimer carefully before using the large language model provided in this repository. Your use of the model signifies your agreement to the following terms and conditions.
187
+
188
+ - Biases and Offensiveness: The large language model is trained on a diverse range of internet text data, which may contain biased, racist, offensive, or otherwise inappropriate content. By using this model, you acknowledge and accept that the generated content may sometimes exhibit biases or produce content that is offensive or inappropriate. The developers of this repository do not endorse, support, or promote any such content or viewpoints.
189
+ - Limitations: The large language model is an AI-based tool and not a human. It may produce incorrect, nonsensical, or irrelevant responses. It is the user's responsibility to critically evaluate the generated content and use it at their discretion.
190
+ - Use at Your Own Risk: Users of this large language model must assume full responsibility for any consequences that may arise from their use of the tool. The developers and contributors of this repository shall not be held liable for any damages, losses, or harm resulting from the use or misuse of the provided model.
191
+ - Ethical Considerations: Users are encouraged to use the large language model responsibly and ethically. By using this model, you agree not to use it for purposes that promote hate speech, discrimination, harassment, or any form of illegal or harmful activities.
192
+ - Reporting Issues: If you encounter any biased, offensive, or otherwise inappropriate content generated by the large language model, please report it to the repository maintainers through the provided channels. Your feedback will help improve the model and mitigate potential issues.
193
+ - Changes to this Disclaimer: The developers of this repository reserve the right to modify or update this disclaimer at any time without prior notice. It is the user's responsibility to periodically review the disclaimer to stay informed about any changes.
194
+
195
+ By using the large language model provided in this repository, you agree to accept and comply with the terms and conditions outlined in this disclaimer. If you do not agree with any part of this disclaimer, you should refrain from using the model and any content generated by it.