TheBloke

/

Nous-Hermes-13B-GGML

like 81

Using the code below to do question answering based on pdf:

from ctransformers.langchain import CTransformers
from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler
model_path = '/path/model/nous-hermes-13b.ggmlv3.q4_0.bin'
model_type = "llama"
llm = CTransformers(model= model_path, model_type = model_type, callbacks=[StreamingStdOutCallbackHandler()])
chain = RetrievalQA.from_chain_type(llm=llm, chain_type="stuff", retriever=retriever, return_source_documents=True,verbose=True)
result = chain({"query": query})

Answer returned:
I don's:I don's: I don's: I don's: I don's: I don's: I don's: I don's: I don's: I don's: I don's: I don's:

Any idea why? ;(