用在RAG容易一直出現重複的文字
您好,
我使用的是Llama-3-Taiwan-8B-Instruct
基於RAG, 我透過LangChain將context跟query一起塞到input_data裡, 但偶爾llm會一直輸出完全無關且重複的文字, 而無法結束.
想請問這邊有什麼建議, 例如對prompt或參數上的修正？
試試加上 stop 的參數: ["<|end_of_text|>","<|eot_id|>"]
有加上stop參數了, 發生機率有下降, 但是偶爾還是會出現重複字串如下
最後列到50條, 跟13項一樣的句子
偶爾也會答非所問
我用的是ollama的q5_k_m版本, 但我想應該與這個無關？
https://ollama.com/cwchang/llama-3-taiwan-8b-instruct
Description: Hello, Currently i am using this model and have designed RAG system. Now we want to receive the response from the chatbot in the specific format. for example
"""prompt"" = ""Please provide details about the shelter located at Puxin Street, suitable for weaker population""
""Response"" = Here are the details of the shelter located at Beipu Junior High School:
Name: Beipu Junior High School
Address: 141 Puxin Street, Puwei Village, Beipu Township, Hsinchu County
Coordinates:
Latitude: 24.704277
Longitude: 121.047729
Location Details:
County: Hsinchu County
Township: Beipu Township
Village: Puwei Village
Accessibility:
Indoor Shelter: Yes
Outdoor Shelter: Yes
Status: Open"
so to receive the response in this kind of format, how can we define the prompt?
Thanks in advance.