agent executor not using custom openai_api_base
Labels: name: area: agent, name: auto:bug, name: area: models

### System Info

OS: Using docker image amd64/python:3.10-slim
Python Version: 3.10.13
Langchain Version: 0.0.336
OpenAI Version: 0.27.7
Tenacity Version: 4.65.0

### Who can help?

@agola11 @hwchase17 

### Information

- [X] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [X] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

When I try to use an llm with a custom openai_api_base argument within an agent, the agent appears to be attempting to access the **OpenAI** API endpoint instead of the custom one I have specified.

Running:

        llm = ChatOpenAI(default_headers={"api-key":"**REDACTED**",
                         openai_api_base="**REDACTED**",
                         openai_api_key="none").bind(stop=["\nObservation"])

        tools =  []
        
        tools.append(Tool.from_function(func=self.get_scores,
                                     name="get_scores",
                                     description="function to get scores"))

        prompt = PromptTemplate.from_template("""Answer the following questions as best you can. You have access to the following tools:
                                                                  {tools}
                                                    
                                                                Use the following format:

                                                                Question: the input question you must answer
                                                                Thought: you should always think about what to do
                                                                Action: the action to take, should be one of [{tool_names}]
                                                                Action Input: the input to the action
                                                                Observation: the result of the action
                                                                ... (this Thought/Action/Action Input/Observation can repeat N times)
                                                                Thought: I now know the final answer
                                                                Final Answer: the final answer to the original input question

                                                                Begin!

                                                                Question: {input}
                                                                Thought:{agent_scratchpad}""")

        prompt = prompt.partial(tools=render_text_description(tools),
                                tool_names=", ".join([t.name for t in tools]),
                                )

        agent = (
        {
        "input": lambda x: x["input"],
        "agent_scratchpad": lambda x: format_log_to_str(x["intermediate_steps"]),
        }
        | prompt
        | llm
        | ReActSingleInputOutputParser()
        )

        agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True)

        output = agent_executor.invoke({"input":"foo"})

yields:

`File "/usr/local/lib/python3.10/site-packages/langchain/chains/base.py", line 87, in invoke
    return self(
  File "/usr/local/lib/python3.10/site-packages/langchain/chains/base.py", line 310, in __call__
    raise e
  File "/usr/local/lib/python3.10/site-packages/langchain/chains/base.py", line 304, in __call__
    self._call(inputs, run_manager=run_manager)
  File "/usr/local/lib/python3.10/site-packages/langchain/agents/agent.py", line 1245, in _call
    next_step_output = self._take_next_step(
  File "/usr/local/lib/python3.10/site-packages/langchain/agents/agent.py", line 1032, in _take_next_step
    output = self.agent.plan(
  File "/usr/local/lib/python3.10/site-packages/langchain/agents/agent.py", line 385, in plan
    output = self.runnable.invoke(inputs, config={"callbacks": callbacks})
  File "/usr/local/lib/python3.10/site-packages/langchain/schema/runnable/base.py", line 1427, in invoke
    input = step.invoke(
  File "/usr/local/lib/python3.10/site-packages/langchain/schema/runnable/base.py", line 2787, in invoke
    return self.bound.invoke(
  File "/usr/local/lib/python3.10/site-packages/langchain/chat_models/base.py", line 142, in invoke
    self.generate_prompt(
  File "/usr/local/lib/python3.10/site-packages/langchain/chat_models/base.py", line 459, in generate_prompt
    return self.generate(prompt_messages, stop=stop, callbacks=callbacks, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/langchain/chat_models/base.py", line 349, in generate
    raise e
  File "/usr/local/lib/python3.10/site-packages/langchain/chat_models/base.py", line 339, in generate
    self._generate_with_cache(
  File "/usr/local/lib/python3.10/site-packages/langchain/chat_models/base.py", line 492, in _generate_with_cache
    return self._generate(
  File "/usr/local/lib/python3.10/site-packages/langchain/chat_models/openai.py", line 422, in _generate
    response = self.completion_with_retry(
  File "/usr/local/lib/python3.10/site-packages/langchain/chat_models/openai.py", line 352, in completion_with_retry
    return _completion_with_retry(**kwargs)
  File "/usr/local/lib/python3.10/site-packages/tenacity/__init__.py", line 289, in wrapped_f
    return self(f, *args, **kw)
  File "/usr/local/lib/python3.10/site-packages/tenacity/__init__.py", line 379, in __call__
    do = self.iter(retry_state=retry_state)
  File "/usr/local/lib/python3.10/site-packages/tenacity/__init__.py", line 325, in iter
    raise retry_exc.reraise()
  File "/usr/local/lib/python3.10/site-packages/tenacity/__init__.py", line 158, in reraise
    raise self.last_attempt.result()
  File "/usr/local/lib/python3.10/concurrent/futures/_base.py", line 451, in result
    return self.__get_result()
  File "/usr/local/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
    raise self._exception
  File "/usr/local/lib/python3.10/site-packages/tenacity/__init__.py", line 382, in __call__
    result = fn(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/langchain/chat_models/openai.py", line 350, in _completion_with_retry
    return self.client.create(**kwargs)
  File "/usr/local/lib/python3.10/site-packages/openai/api_resources/chat_completion.py", line 25, in create
    return super().create(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/openai/api_resources/abstract/engine_api_resource.py", line 153, in create
    response, _, api_key = requestor.request(
  File "/usr/local/lib/python3.10/site-packages/openai/api_requestor.py", line 230, in request
    resp, got_stream = self._interpret_response(result, stream)
  File "/usr/local/lib/python3.10/site-packages/openai/api_requestor.py", line 624, in _interpret_response
    self._interpret_response_line(
  File "/usr/local/lib/python3.10/site-packages/openai/api_requestor.py", line 687, in _interpret_response_line
    raise self.handle_error_response(
  File "/usr/local/lib/python3.10/site-packages/openai/api_requestor.py", line 337, in handle_error_response
    raise error.APIError(
openai.error.APIError: Invalid response object from API: '{\n  "detail": "No authorization token provided",\n  "status": 401,\n  "title": "Unauthorized",\n  "type": "about:blank"\n}\n' (HTTP response code was 401)`

When I change the openai_api_base to something nonsensical, the same error is returned, making me think that it is using OpenAI's API base and not the custom one specified.

### Expected behavior

I would expect the agent to work as shown here: https://python.langchain.com/docs/modules/agents/agent_types/react