bartowski

/

Meta-Llama-3-8B-Instruct-old-GGUF

like 45

Hi!
I am trying this simple code:

from llama_cpp import Llama
model_name = 'Meta-Llama-3-8B.Q4_K_M.gguf'
llm = Llama( model_path=f"llms/{model_name}", n_gpu_layers=20 )
response = llm.create_chat_completion( messages = [ { "role": "user", "content": "How are you?" } ] )
print(response)

And the answer is very crazy:

{'id': 'chatcmpl-f73666a5-b774-44c5-9b4a-69c43c050838', 'object': 'chat.completion', 'created': 1714041374, 'model': 'llms/Meta-Llama-3-8B.Q4_K_M.gguf', 'choices': [{'index': 0, 'message': {'role': 'assistant', 'content': "I am a student of computer science in Iran. I have been learning C++ for 2 years and I want to learn more about it.\nI'm looking for someone who can teach me C++ online, but I don't know where to start from.\n\nI'm sorry if my English is not good enough, because I'm using Google Translate.\n\nThanks\n\n1. ### Staff Emeritus Science Advisor Gold Member\n\nMessages:\n22,202\nWelcome to PF!\n\n2. ### ircuser Banned\n\nMessages:\n4,000\nYou can find a lot of tutorials on the internet. You could also try to get in contact with some local universities and ask if they have any online courses for C++.\n\n3. ### ircuser Banned\n\nMessages:\n4,000\nI'm not sure if this is what you're looking for but here's a link to a free course on Udemy: https://www.udemy.com/course/c-plus-plus-101/\n\n4. ### ircuser Banned\n\nMessages:\n4,000\nYou could also try to find some online courses on Coursera or Edx.\n\n5. ### ircuser Banned\n\nMessages:\n4,000\nI'm not sure if this is what you're looking for but here's a link to a free course on Udemy: https://www.udemy.com/course/c-plus-plus-101/\n\n6. ### ircuser Banned\n\nMessages:\n4,000\nYou could also try to find some online courses on Coursera or Edx.\n\n7. ### ircuser Banned\n\nMessages:\n4,000\nI'm not sure if this is what you're looking for but here's a link to a free course on Udemy: https://www.udemy.com/course/c-plus-plus-101/\n\n8. ### ircuser Banned\n\nMessages:\n4,000\nYou could also try to find some online courses on Coursera or Edx.\n\n9. ### ircuser Banned\n\nMessages:\n4,000\nI'm not sure if this is what you're looking for but here's a link to a free course on Udemy: https://www.udemy.com/course/c-plus-plus-101/\n\n10. ### ircuser Banned\n\nMessages:\n4,000\nYou could also try to find some online courses on Coursera or Edx.\n\n11. ### ircuser Banned\n\nMessages:\n4,000\nI'm not sure if this is what you"}, 'logprobs': None, 'finish_reason': 'length'}], 'usage': {'prompt_tokens': 15, 'completion_tokens': 497, 'total_tokens': 512}}

What I need to do?