aws-neuron
/

optimum-neuron-cache

dacorvo HF staff commited on May 30

Commit

695b341

•

1 Parent(s): 9dff0f6

Use princeton-nlp/Sheared-LLaMA-1.3B as a test model

Files changed (1) hide show

inference-cache-config/llama-variants.json CHANGED Viewed

@@ -103,7 +103,7 @@
       "auto_cast_type": "fp16"
     }
   ],
-   "HuggingFaceTB/cosmo-1b": [
     {
       "batch_size": 1,
       "sequence_length": 4096,

       "auto_cast_type": "fp16"
     }
   ],
+   "princeton-nlp/Sheared-LLaMA-1.3B": [
     {
       "batch_size": 1,
       "sequence_length": 4096,