Challenges working with llamacpp with Langchain https://python.langchain.com/docs/use_cases/code_understanding
Labels: name: auto:bug, name: stale, name: area: models

### System Info

RTX 3090

### Who can help?

@agola11 @hwchase17 

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Code: https://colab.research.google.com/drive/1Po3oQcSAWjOkvSKc4o57rDs3n1PFZe20?usp=sharing

Not able to get BLAS = 1 
not able to extract Answer:
```
(It's a bad idea to parse output from `ls`, though, as you may

    
    llama_print_timings:        load time =  1074.43 ms
    llama_print_timings:      sample time =   180.71 ms /   256 runs   (    0.71 ms per token,  1416.67 tokens per second)
    llama_print_timings: prompt eval time =     0.00 ms /     1 tokens (    0.00 ms per token,      inf tokens per second)
    llama_print_timings:        eval time =  9593.04 ms /   256 runs   (   37.47 ms per token,    26.69 tokens per second)
    llama_print_timings:       total time = 10139.91 ms





    ' You can use the find command with a few options to this task. Here is an example of how you might go about it:\n\nfind . -type f -mtime +28 -exec ls {} \\;\nThis command only for plain files (not), and limits the search to files that were more than 28 days ago, then the "ls" command on each file found. The {} is a for the filenames found by find that are being passed to the -exec option of find.\n\nYou can also use find in with other unix utilities like sort and grep to the list of files before they are:\n\nfind . -type f -mtime +28 | sort | grep pattern\nThis will find all plain files that match a given pattern, then sort the listically and filter it for only the matches.\n\nAnswer: `find` is pretty with its search. The should work as well:\n\n\\begin{code}\nls -l $(find . -mtime +28)\n\\end{code}\n\n(It\'s a bad idea to parse output from `ls`, though, as you may'


from langchain.chains.question_answering import load_qa_chain

# Prompt
template = """Use the following pieces of context to answer the question at the end. 
If you don't know the answer, just say that you don't know, don't try to make up an answer. 
Use three sentences maximum and keep the answer as concise as possible. 
{context}
Question: {question}
Helpful Answer:"""
QA_CHAIN_PROMPT = PromptTemplate(
    input_variables=["context", "question"],
    template=template,
)

API Reference:
[load_qa_chain](https://api.python.langchain.com/en/latest/chains/langchain.chains.question_answering.load_qa_chain.html)
We can also use the LangChain Prompt Hub to store and fetch prompts.

This will work with your [LangSmith API key](https://docs.smith.langchain.com/).

Let's try with a default RAG prompt, [here](https://smith.langchain.com/hub/rlm/rag-prompt).

from langchain import hub
QA_CHAIN_PROMPT = hub.pull("rlm/rag-prompt-default")

# Docs
question = "How can I initialize a ReAct agent?"
docs = retriever.get_relevant_documents(question)

# Chain
chain = load_qa_chain(llm, chain_type="stuff", prompt=QA_CHAIN_PROMPT)

# Run
chain({"input_documents": docs, "question": question}, return_only_outputs=True)

    Llama.generate: prefix-match hit


     You can use the `ReActAgent` class and pass it the desired tools as, for example, you would do like this to create an agent with the `Lookup` and `Search` tool:
    ```python
    from langchain.agents.react import ReActAgent
    from langchain.tools.lookup import Lookup
    from langchain.tools.search import Search
    ReActAgent(Lookup(), Search())
    ```

    
    llama_print_timings:        load time =  1074.43 ms
    llama_print_timings:      sample time =    65.46 ms /    94 runs   (    0.70 ms per token,  1435.95 tokens per second)
    llama_print_timings: prompt eval time = 15975.57 ms /  1408 tokens (   11.35 ms per token,    88.13 tokens per second)
    llama_print_timings:        eval time =  4772.57 ms /    93 runs   (   51.32 ms per token,    19.49 tokens per second)
    llama_print_timings:       total time = 20959.57 ms





    {'output_text': ' You can use the `ReActAgent` class and pass it the desired tools as, for example, you would do like this to create an agent with the `Lookup` and `Search` tool:\n```python\nfrom langchain.agents.react import ReActAgent\nfrom langchain.tools.lookup import Lookup\nfrom langchain.tools.search import Search\nReActAgent(Lookup(), Search())\n```'}


Here's the trace [RAG](https://smith.langchain.com/public/f21c4bcd-88da-4681-8b22-a0bb0e31a0d3/r), showing the retrieved docs.

Previous
Chatbots
Next
Extraction
[Use case](https://python.langchain.com/docs/use_cases/code_understanding#use-case)
[Overview](https://python.langchain.com/docs/use_cases/code_understanding#overview)
[Quickstart](https://python.langchain.com/docs/use_cases/code_understanding#quickstart)
[Loading](https://python.langchain.com/docs/use_cases/code_understanding#loading)
[Splitting](https://python.langchain.com/docs/use_cases/code_understanding#splitting)
[RetrievalQA](https://python.langchain.com/docs/use_cases/code_understanding#retrievalqa)
[Chat](https://python.langchain.com/docs/use_cases/code_understanding#chat)
[Open source LLMs](https://python.langchain.com/docs/use_cases/code_understanding#open-source-llms)
```

### Expected behavior

get appropriate answer as shared in https://python.langchain.com/docs/use_cases/code_understanding