alozowski commited on
Commit
8013f91
β€’
1 Parent(s): c8c4792

update and revised submission info

Browse files
Files changed (1) hide show
  1. src/display/about.py +26 -14
src/display/about.py CHANGED
@@ -179,35 +179,47 @@ EVALUATION_QUEUE_TEXT = f"""
179
 
180
  Models added here will be automatically evaluated on the πŸ€— cluster.
181
 
182
- ## Don't forget to read the FAQ and the About tabs for more information!
183
 
184
- ## First steps before submitting a model
 
 
 
 
 
 
 
 
 
 
185
 
186
- ### 1) Make sure you can load your model and tokenizer using AutoClasses:
187
  ```python
188
  from transformers import AutoConfig, AutoModel, AutoTokenizer
189
  config = AutoConfig.from_pretrained("your model name", revision=revision)
190
  model = AutoModel.from_pretrained("your model name", revision=revision)
191
  tokenizer = AutoTokenizer.from_pretrained("your model name", revision=revision)
192
  ```
193
- If this step fails, follow the error messages to debug your model before submitting it. It's likely your model has been improperly uploaded.
 
 
 
194
 
195
- Note: make sure your model is public!
196
- Note: if your model needs `use_remote_code=True`, we do not support this option yet but we are working on adding it, stay posted!
197
 
198
- ### 2) Convert your model weights to [safetensors](https://huggingface.co/docs/safetensors/index)
199
- It's a new format for storing weights which is safer and faster to load and use. It will also allow us to add the number of parameters of your model to the `Extended Viewer`!
200
-
201
- ### 3) Make sure your model has an open license!
202
  This is a leaderboard for Open LLMs, and we'd love for as many people as possible to know they can use your model πŸ€—
203
 
204
- ### 4) Fill up your model card
205
  When we add extra information about models to the leaderboard, it will be automatically taken from the model card
206
 
207
- ### 5) Select the correct precision
208
- Not all models are converted properly from `float16` to `bfloat16`, and selecting the wrong precision can sometimes cause evaluation error (as loading a `bf16` model in `fp16` can sometimes generate NaNs, depending on the weight range).
 
 
 
 
209
 
210
- <b>Note:</b> Please be advised that when submitting, git <b>branches</b> and <b>tags</b> will be strictly tied to the <b>specific commit</b> present at the time of submission. This ensures revision consistency.
211
  ## Model types
212
  {icons}
213
  """
 
179
 
180
  Models added here will be automatically evaluated on the πŸ€— cluster.
181
 
182
+ > **Important:** Don't forget to read the [FAQ](https://huggingface.co/docs/leaderboards/open_llm_leaderboard/faq) and [documentation](https://huggingface.co/docs/leaderboards/open_llm_leaderboard/about) for more information! πŸ“„
183
 
184
+ ## Submission Disclaimer
185
+ **By submitting a model, you acknowledge that:**
186
+ - We store information about who submitted each model in [Requests dataset](https://huggingface.co/datasets/open-llm-leaderboard/requests).
187
+ - This practice helps maintain the integrity of our leaderboard, prevent spam, and ensure responsible submissions.
188
+ - Your submission will be visible to the community and you may be contacted regarding your model.
189
+ - Please submit carefully and responsibly πŸ’›
190
+
191
+ ## First Steps Before Submitting a Model
192
+
193
+ ### 1. Ensure Your Model Loads with AutoClasses
194
+ Verify that you can load your model and tokenizer using AutoClasses:
195
 
 
196
  ```python
197
  from transformers import AutoConfig, AutoModel, AutoTokenizer
198
  config = AutoConfig.from_pretrained("your model name", revision=revision)
199
  model = AutoModel.from_pretrained("your model name", revision=revision)
200
  tokenizer = AutoTokenizer.from_pretrained("your model name", revision=revision)
201
  ```
202
+ Note:
203
+ - If this step fails, debug your model before submitting.
204
+ - Ensure your model is public.
205
+ - We are working on adding support for models requiring `use_remote_code=True`.
206
 
207
+ ### 2. Convert Weights to Safetensors
208
+ [Safetensors](https://huggingface.co/docs/safetensors/index) is a new format for storing weights which is safer and faster to load and use. It will also allow us to add the number of parameters of your model to the `Extended Viewer`!
209
 
210
+ ### 3. Verify Your Model Open License
 
 
 
211
  This is a leaderboard for Open LLMs, and we'd love for as many people as possible to know they can use your model πŸ€—
212
 
213
+ ### 4. Complete Your Model Card
214
  When we add extra information about models to the leaderboard, it will be automatically taken from the model card
215
 
216
+ ### 5. Select Correct Precision
217
+ Choose the right precision to avoid evaluation errors:
218
+ - Not all models convert properly from float16 to bfloat16.
219
+ - Incorrect precision can cause issues (e.g., loading a bf16 model in fp16 may generate NaNs).
220
+
221
+ > **Important:** When submitting, git branches and tags will be strictly tied to the specific commit present at the time of submission to ensure revision consistency.
222
 
 
223
  ## Model types
224
  {icons}
225
  """