kmadsen

/

vicuna-7b-v1.1-q4f16_0

like 1

Hi Qing

I made a edit to the MLCChat application so that our fork will accept unaltered model directories from huggingface. This is what the configuration looks like, notice the added model_url_directory.

{
  "model_libs": [
    "RedPajama-INCITE-Chat-3B-v1-q4f16_0",
    "vicuna-7b-v1.1-q4f16_0"
  ],
  "model_list": [
    {
      "model_url": "https://huggingface.co/mlc-ai/mlc-chat-RedPajama-INCITE-Chat-3B-v1-q4f16_0/",
      "local_id": "RedPajama-INCITE-Chat-3B-v1-q4f16_0"
    },
    {
      "model_url": "https://huggingface.co/kmadsen/vicuna-7b-v1.1-q4f16_0/",
      "model_url_directory": "params/",
      "local_id": "vicuna-7b-v1.1-q4f16_0"
    }
  ],
  "add_model_samples": [
  ]
}

Ok, thanks for your comment. But my issues not related with name, I directly change name from vicuna-7b-v1.1-q4f16_0 to vicuna-v1-7b-q4f16_0 is also ok.

I checked ndarray-cache.json with default ndarray-cache.json seems a little different, my ndarray-cache.json first param as below:
"dataPath": "params_shard_0.bin",
"format": "raw-shard",
"nbytes": 65536000,
"records": [
{
"name": "param_0",

But default is :
"dataPath": "params_shard_0.bin",
"format": "raw-shard",
"nbytes": 65536000,
"records": [
{
"name": "param_67",

Default can work, but my generated params not work, will check further.
Thanks again for your comment.

Thanks,
Qing

The reason I uploaded this was to keep track of vicuna-7b-v1.1-q4f16_0-android.tar. For some reason it's not uploaded to https://github.com/mlc-ai/binary-mlc-llm-libs

I was facing similar issues as you except it was when using the pre-builds in https://huggingface.co/mlc-ai. This upload allows us to build the android app and load the params from a single place.

Am considering this model all temporary, but if you're looking to customize an android app for testing; then this model will allow you to skip the custom TVM+OpenCL. But you'll still need to prepare_libs.sh.