mistralai

/

Mistral-7B-Instruct-v0.2

like 2.29k

Hi! I have actually been working on that these days. I had thought about writing a post on Medium describing the process but since I have never published anything before, it might take me some time.

In the meantime, I can give you some tips if you want, although you might have already figured it out.

My goal is creating a local chatbot with mistral 7b which connects my text data and mistral 7b and this agent must read german language very well. If you could provide something, I will be happy.

If you want to chat with your local text data, wouldn't it be better to implement a RAG pipeline instead of using agents?

I am actually not an expert on this topic and I am trying to learn how to connect a big llm and a data source but llm must work without any API connection. It must be downloaded on my own device. Yeah maybe it would be better on that way but I don't have really any idea..

Ok. Check this: https://medium.com/@thakermadhav/build-your-own-rag-with-mistral-7b-and-langchain-97d0c92fa146
It is a good starting point.

That is great thanks for your support :D

I'm having an error runing simple tools on agent.
The error is: LLM output produced both a final answer and a parse-able action:: Answer the following questions as best you can. You have access to the following tools...

Do you recognize this error?

I'm having an error runing simple tools on agent.
The error is: LLM output produced both a final answer and a parse-able action:: Answer the following questions as best you can. You have access to the following tools...

Do you recognize this error?

I'm almost certain it happens because you're not managing the 'stop words'. In each iteration, langchain sends the LLM a list of words that, if found in the output, must be removed along with all the text following them.

In the _call method of the LLM class, there's an optional 'stop' parameter. That's the list of words the model will receive.

def _call(
    self,
    prompt: str,
    stop: Optional[List[str]] = None,
    run_manager: Optional[CallbackManagerForLLMRun] = None,
    **kwargs: Any,
) -> str:

By the way, I think I will be able to publish the tutorial this weekend.

I'm having an error runing simple tools on agent.
The error is: LLM output produced both a final answer and a parse-able action:: Answer the following questions as best you can. You have access to the following tools...

Do you recognize this error?

I'm almost certain it happens because you're not managing the 'stop words'. In each iteration, langchain sends the LLM a list of words that, if found in the output, must be removed along with all the text following them.

In the _call method of the LLM class, there's an optional 'stop' parameter. That's the list of words the model will receive.

def _call(
    self,
    prompt: str,
    stop: Optional[List[str]] = None,
    run_manager: Optional[CallbackManagerForLLMRun] = None,
    **kwargs: Any,
) -> str:

By the way, I think I will be able to publish the tutorial this weekend.

Actually I set the stop by this way:

agent = create_react_agent(hf, tools, prompt=prompt_agent, stop_sequence=['\nFinal Answer'] )

agent_executor = AgentExecutor.from_agent_and_tools(
agent=agent,
verbose=True,
tools=tools,
max_iterations=2,
return_intermediate_steps=True,
#handle_parsing_errors=True,
memory=memory
)

while True:
response = agent_executor.invoke({"input":input(">>>")})
print(response["output"])

Well, stop words don't work like that. By the way, here is the tutorial link: https://medium.com/@jorgepardoserrano/building-a-langchain-agent-with-a-self-hosted-mistral-7b-a-step-by-step-guide-85eda2fbf6c2
It includes a link to the related colab. I hope it helps.

Just to comment that I have reviewed and edited the notebook. I have significantly improved the prompting, and now it is much more reliable.

Thank you @Yemeral I was able to solve my problem using your tutorial.