Couples of warnings popping out
So I have cloned the main branch and I am trying to run the sample code in the description but some warnings which are shown below popped out:
CUDA extension not installed.
The model weights are not tied. Please use the tie_weights method before using the infer_auto_device function.
The safetensors archive passed at ./Llama-2-7B-GPTQ\gptq_model-4bit-128g.safetensors does not contain metadata. Make sure to save your model with the save_pretrained method. Defaulting to 'pt' metadata.
skip module injection for FusedLlamaMLPForQuantizedModel not support integrate without triton yet.
Even though output is generated eventually, it is really slow and the result is undesired:
nobody can tell me about AI
I'm not sure what you mean by "tell me about AI".
I'm not sure what you mean by "tell me about AI". I'm not sure what you mean by "tell me about AI". I'm not sure what you mean by "tell me about AI". I'm not sure what you mean by "tell me about AI". I'm not sure what you mean by "tell me about AI". I'm not sure what you mean by "tell me about AI". I'm not sure what you mean by "tell me about AI". I'm not sure what you mean by "tell me about AI". I'm not sure what you mean by "tell me about AI". I'm not sure what you mean by "tell me about AI". I'm not sure what you mean by "tell me about AI". I'm not sure what you mean by "tell me about AI". I'm not sure what you mean by "tell me about AI". I'm not sure what you mean by "tell me about AI". I'm not sure what you mean by "tell me about AI". I'm not sure what you mean by "tell me about AI". I'm not sure what you mean by "tell me about AI". I'm not sure what you mean by "tell me about AI". I'm not sure what you mean by "tell me about AI". I'm not sure what you mean by "tell me about AI". I'm not sure what you mean by "tell me about AI". I'm not sure what you mean by "tell me about AI". I'm not sure what you mean by "tell me about AI". I'm not sure what you mean by "tell me about AI". I'm not sure what you mean by "tell me about AI". I'm not sure what you mean by "tell me about AI". I'm not sure what you mean by "tell me about AI". I'm not sure what you mean by "tell me about AI". I'm not sure what you mean by "
I wonder if it is the problem related to the warnings...