question about the gguf model

#5
by zz227892 - opened

The provided llava-v1.5-7b-mmproj-Q8_0.gguf. And get the following information during the inference. The provided llava-v1.5-7b-mmproj-Q8_0.gguf uses q5_1 quantization. @jartine

clip_model_load: model name: openai/clip-vit-large-patch14-336
clip_model_load: description: image encoder for LLaVA
clip_model_load: GGUF version: 3
clip_model_load: alignment: 32
clip_model_load: n_tensors: 377
clip_model_load: n_kv: 19
clip_model_load: ftype: q5_1

clip_model_load: CLIP using CPU backend
clip_model_load: text_encoder: 0
clip_model_load: vision_encoder: 1
clip_model_load: llava_projector: 1
clip_model_load: model size: 224.83 MB
clip_model_load: metadata size: 0.14 MB
clip_model_load: params backend buffer size = 224.83 MB (377 tensors)
clip_model_load: compute allocated memory: 32.89 MB

Sign up or log in to comment