I want to know how gguf converts this model and why its model_family is llama.
#3
by
hzjane
- opened
I want to know how gguf converts this model. I tried to use baichuan2-7b-chat.Q4_0.gguf and found that its model_family is llama. Does this mean that I am running llama's model.
GGUFConfig["general.architecture"] is llama
Yes, as I understand it, llama is a model structure and gguf is a file format. gguf contains all the files needed for model inference, including the model structure, model parameters, model hyperparameters, and so on.