Cannot use LangChain Tools with GPT4ALL and LLaMA model
Labels: name: area: agent, name: auto:bug, name: area: models

### System Info

LangChain 0.0.292
Python 3.11.5
GPT4ALL with LLAMA q4_0 3b model running on CPU

### Who can help?

@agola11

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [X] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

My code is:
```from langchain.llms import GPT4All
from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler
from langchain.agents import AgentType, initialize_agent
from langchain.tools import Tool
from pydantic import BaseModel, Field

class JokeInput(BaseModel):
    confidence: float = Field(default=0.0)

import pyjokes

import langchain
langchain.debug = True
import os
os.environ["LANGCHAIN_TRACING"] = "true" # If you want to trace the execution of the program, set to "true"

# There are many CallbackHandlers supported, such as
# from langchain.callbacks.streamlit import StreamlitCallbackHandler
callbacks = [StreamingStdOutCallbackHandler()]
model = GPT4All(model=".cache/llama-2-7b-chat.ggmlv3.q4_0.bin", n_threads=6)

# Generate text. Tokens are streamed through the callback manager.
tools = []
tools.append(
    Tool.from_function(
        func=pyjokes.get_joke,
        name="Joke",
        description="useful for when you need to tell the user a joke",
        args_schema=JokeInput
    )
)

# Init the agent
agent = initialize_agent(
    tools, model, agent=AgentType.CHAT_ZERO_SHOT_REACT_DESCRIPTION, verbose=True
)

# Start the agent
while True:
    user_prompt = input("You: ")
    agent(user_prompt, callbacks=callbacks)```
```

### Expected behavior

It should use the Joke tool and return a joke, but instead it gives me this error:
```
Found model file at  .cache/llama-2-7b-chat.ggmlv3.q4_0.bin
llama.cpp: loading model from .cache/llama-2-7b-chat.ggmlv3.q4_0.bin
llama_model_load_internal: format     = ggjt v3 (latest)
llama_model_load_internal: n_vocab    = 32000
llama_model_load_internal: n_ctx      = 2048
llama_model_load_internal: n_embd     = 4096
llama_model_load_internal: n_mult     = 256
llama_model_load_internal: n_head     = 32
llama_model_load_internal: n_head_kv  = 32
llama_model_load_internal: n_layer    = 32
llama_model_load_internal: n_rot      = 128
llama_model_load_internal: n_gqa      = 1
llama_model_load_internal: rnorm_eps  = 5.0e-06
llama_model_load_internal: n_ff       = 11008
llama_model_load_internal: freq_base  = 10000.0
llama_model_load_internal: freq_scale = 1
llama_model_load_internal: ftype      = 2 (mostly Q4_0)
llama_model_load_internal: model size = 7B
llama_model_load_internal: ggml ctx size = 3615.73 MB
llama_model_load_internal: mem required  = 4013.73 MB (+ 1024.00 MB per state)
llama_new_context_with_model: kv self size  = 1024.00 MB
You: Tell me a joke
Failed to load default session, using empty session: HTTPConnectionPool(host='localhost', port=8000): Max retries exceeded with url: /sessions?name=default (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff8fc87d290>: Failed to establish a new connection: [Errno 111] Connection refused'))
[chain/start] [1:chain:AgentExecutor] Entering Chain run with input:
{
  "input": "Tell me a joke"
}
[chain/start] [1:chain:AgentExecutor > 2:chain:LLMChain] Entering Chain run with input:
{
  "input": "Tell me a joke",
  "agent_scratchpad": "",
  "stop": [
    "Observation:"
  ]
}
[llm/start] [1:chain:AgentExecutor > 2:chain:LLMChain > 3:llm:GPT4All] Entering LLM run with input:
{
  "prompts": [
    "System: Answer the following questions as best you can. You have access to the following tools:\n\nJoke: useful for when you need to tell the user a joke\n\nThe way you use the tools is by specifying a json blob.\nSpecifically, this json should have a `action` key (with the name of the tool to use) and a `action_input` key (with the input to the tool going here).\n\nThe only values that should be in the \"action\" field are: Joke\n\nThe $JSON_BLOB should only contain a SINGLE action, do NOT return a list of multiple actions. Here is an example of a valid $JSON_BLOB:\n\n```\n{\n  \"action\": $TOOL_NAME,\n  \"action_input\": $INPUT\n}\n```\n\nALWAYS use the following format:\n\nQuestion: the input question you must answer\nThought: you should always think about what to do\nAction:\n```\n$JSON_BLOB\n```\nObservation: the result of the action\n... (this Thought/Action/Observation can repeat N times)\nThought: I now know the final answer\nFinal Answer: the final answer to the original input question\n\nBegin! Reminder to always use the exact characters `Final Answer` when responding.\nHuman: Tell me a joke"
  ]
}
$JSON_BLOB[llm/end] [1:chain:AgentExecutor > 2:chain:LLMChain > 3:llm:GPT4All] [18.05s] Exiting LLM run with output:
{
  "generations": [
    [
      {
        "text": "$JSON_BLOB",
        "generation_info": null
      }
    ]
  ],
  "llm_output": null,
  "run": null
}
[chain/end] [1:chain:AgentExecutor > 2:chain:LLMChain] [18.05s] Exiting Chain run with output:
{
  "text": "$JSON_BLOB"
}
[chain/error] [1:chain:AgentExecutor] [18.05s] Chain run errored with error:
"OutputParserException('Could not parse LLM output: $JSON_BLOB')"
Failed to persist run: HTTPConnectionPool(host='localhost', port=8000): Max retries exceeded with url: /chain-runs (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7ff8fc24f9d0>: Failed to establish a new connection: [Errno 111] Connection refused'))
Traceback (most recent call last):
  File "/mnt/0A956B927E2FFFE8/Code/jarvis/.venv/lib/python3.11/site-packages/langchain/agents/chat/output_parser.py", line 27, in parse
    raise ValueError("action not found")
ValueError: action not found

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/mnt/0A956B927E2FFFE8/Code/jarvis/langchain_try.py", line 41, in <module>
    agent(user_prompt, callbacks=callbacks)
  File "/mnt/0A956B927E2FFFE8/Code/jarvis/.venv/lib/python3.11/site-packages/langchain/chains/base.py", line 292, in __call__
    raise e
  File "/mnt/0A956B927E2FFFE8/Code/jarvis/.venv/lib/python3.11/site-packages/langchain/chains/base.py", line 286, in __call__
    self._call(inputs, run_manager=run_manager)
  File "/mnt/0A956B927E2FFFE8/Code/jarvis/.venv/lib/python3.11/site-packages/langchain/agents/agent.py", line 1122, in _call
    next_step_output = self._take_next_step(
                       ^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/0A956B927E2FFFE8/Code/jarvis/.venv/lib/python3.11/site-packages/langchain/agents/agent.py", line 930, in _take_next_step
    raise e
  File "/mnt/0A956B927E2FFFE8/Code/jarvis/.venv/lib/python3.11/site-packages/langchain/agents/agent.py", line 919, in _take_next_step
    output = self.agent.plan(
             ^^^^^^^^^^^^^^^^
  File "/mnt/0A956B927E2FFFE8/Code/jarvis/.venv/lib/python3.11/site-packages/langchain/agents/agent.py", line 532, in plan
    return self.output_parser.parse(full_output)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/0A956B927E2FFFE8/Code/jarvis/.venv/lib/python3.11/site-packages/langchain/agents/chat/output_parser.py", line 42, in parse
    raise OutputParserException(f"Could not parse LLM output: {text}")
langchain.schema.output_parser.OutputParserException: Could not parse LLM output: $JSON_BLOB
```