More quality results
Browse files
    	
        README.md
    CHANGED
    
    | 
         @@ -185,6 +185,7 @@ Okay, the user is asking if I can talk to them. First, I need to clarify that I 
     | 
|
| 185 | 
         
             
            | mmlu_pro                         | -          | -                     |
         
     | 
| 186 | 
         
             
            | bbh                              | -          | -                     |
         
     | 
| 187 | 
         
             
            | **Reasoning**                    |                |                           |
         
     | 
| 
         | 
|
| 188 | 
         
             
            | gpqa_main_zeroshot               | -          | 27.46                     |
         
     | 
| 189 | 
         
             
            | **Multilingual**                 |                |                           |
         
     | 
| 190 | 
         
             
            | m_mmlu                           | -          | -                     |
         
     | 
| 
         @@ -203,7 +204,7 @@ Need to install lm-eval from source: https://github.com/EleutherAI/lm-evaluation 
     | 
|
| 203 | 
         | 
| 204 | 
         
             
            ## baseline
         
     | 
| 205 | 
         
             
            ```Shell
         
     | 
| 206 | 
         
            -
            lm_eval --model hf --model_args pretrained= 
     | 
| 207 | 
         
             
            ```
         
     | 
| 208 | 
         | 
| 209 | 
         
             
            ## int8 dynamic activation and int4 weight quantization (8da4w)
         
     | 
| 
         | 
|
| 185 | 
         
             
            | mmlu_pro                         | -          | -                     |
         
     | 
| 186 | 
         
             
            | bbh                              | -          | -                     |
         
     | 
| 187 | 
         
             
            | **Reasoning**                    |                |                           |
         
     | 
| 188 | 
         
            +
            | hellaswag                        | -          | 54.39                     |
         
     | 
| 189 | 
         
             
            | gpqa_main_zeroshot               | -          | 27.46                     |
         
     | 
| 190 | 
         
             
            | **Multilingual**                 |                |                           |
         
     | 
| 191 | 
         
             
            | m_mmlu                           | -          | -                     |
         
     | 
| 
         | 
|
| 204 | 
         | 
| 205 | 
         
             
            ## baseline
         
     | 
| 206 | 
         
             
            ```Shell
         
     | 
| 207 | 
         
            +
            lm_eval --model hf --model_args pretrained=HuggingFaceTB/SmolLM3-3B --tasks mmlu --device cuda:0 --batch_size auto
         
     | 
| 208 | 
         
             
            ```
         
     | 
| 209 | 
         | 
| 210 | 
         
             
            ## int8 dynamic activation and int4 weight quantization (8da4w)
         
     |