mlc-ai
/

RedPajama-INCITE-Chat-3B-v1-q4f16_1-MLC

Model card Files Files and versions Community

ruihanglai commited on Apr 15

Commit

dcc48ed

•

1 Parent(s): f3a6bf4

Update mlc-chat-config.json

Files changed (1) hide show

mlc-chat-config.json +2 -1

mlc-chat-config.json CHANGED Viewed

@@ -15,7 +15,8 @@
     "head_dim": 80,
     "prefill_chunk_size": 2048,
     "tensor_parallel_shards": 1,
-    "ffn_out_dtype": "float32"
   },
   "vocab_size": 50432,
   "context_window_size": 2048,

     "head_dim": 80,
     "prefill_chunk_size": 2048,
     "tensor_parallel_shards": 1,
+    "ffn_out_dtype": "float32",
+    "max_batch_size": 80
   },
   "vocab_size": 50432,
   "context_window_size": 2048,