importance matrix
#1
by
sanjay920
- opened
Do you mind sharing how you generated the importance matrix?
sanjay920
changed discussion title from
important matrix
to importance matrix
also, how did you overcome this: https://huggingface.co/gorilla-llm/gorilla-openfunctions-v2/discussions/2
@sanjay920 Normally I would extract data relevant for the input/output of the model and generate a text file, however in this case since the model is mostly ingesting JSON anyway I used the file (linked in the README.md) as-is and it turned out to work very well. :) Then it is simply a case of running the following command:
./imatrix -m gorilla-openfunctions-v2.fp16.gguf -f gorilla_openfunctions_v1_train.json -o gorilla-openfunctions-v2.imatrix.dat -ngl 99 -c 4096 --chunks 256
I created a new chat_template as seen in PR #4 to fix the issues with the original one.
awesome! thanks for the info :)
sanjay920
changed discussion status to
closed