mlc-ai
/

TinyLlama-1.1B-Chat-v0.4-q4f32_1-MLC

Model card Files Files and versions Community

ruihanglai commited on Apr 15

Commit

7e97171

•

1 Parent(s): 8574133

Update mlc-chat-config.json

Files changed (1) hide show

mlc-chat-config.json +1 -1

mlc-chat-config.json CHANGED Viewed

@@ -14,7 +14,7 @@
     "num_key_value_heads": 4,
     "head_dim": 64,
     "tensor_parallel_shards": 1,
-    "max_batch_size": 1
   },
   "vocab_size": 32003,
   "context_window_size": 2048,

     "num_key_value_heads": 4,
     "head_dim": 64,
     "tensor_parallel_shards": 1,
+    "max_batch_size": 80
   },
   "vocab_size": 32003,
   "context_window_size": 2048,