delayedkarma/langchain-issues huggingface dataset:
AmadeusClosestAirport tool should accept any LLM
Labels: name: auto:improvement, name: area: models

### Checked other resources

- [X] I added a very descriptive title to this issue.
- [X] I searched the LangChain documentation with the integrated search.
- [X] I used the GitHub search to find a similar question and didn't find it.

### Description

`AmadeusClosestAirport` contains a "hardcoded" call to `ChatOpenAI` (see [here](https://github.com/langchain-ai/langchain/blob/a06db53c37344b5a9906fbf656173c3421109398/libs/community/langchain_community/tools/amadeus/closest_airport.py#L50)), while it would make sense to use the same `llm` passed to the chain/agent when initialized.

In addition, this implies that `AmadeusToolkit` implicitly depends on `openai`, which should not be the case.

Example (source code from the [docs](https://python.langchain.com/docs/integrations/toolkits/amadeus))
```py
from langchain_community.agent_toolkits.amadeus.toolkit import AmadeusToolkit
# Set environmental variables here
import os

os.environ["AMADEUS_CLIENT_ID"] = "CLIENT_ID"
os.environ["AMADEUS_CLIENT_SECRET"] = "CLIENT_SECRET"
os.environ["OPENAI_API_KEY"] = "API_KEY"
# os.environ["AMADEUS_HOSTNAME"] = "production" or "test"

toolkit = AmadeusToolkit() 
tools = toolkit.get_tools()

llm = OpenAI(temperature=0) # this can be any `BaseLLM`
agent = initialize_agent(
    tools=tools,
    llm=llm,
    verbose=False,
    agent=AgentType.STRUCTURED_CHAT_ZERO_SHOT_REACT_DESCRIPTION,
)

#  ==> agent calls `ChatOpenAI` regardless of `llm` <===
agent.run("What is the name of the airport in Cali, Colombia?")
```

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [X] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async
DOC: Missing dependency when going through the Quickstart section
Labels: name: auto:documentation

### Issue with current documentation:

When running the `WebBaseLoader` it requires `bs4` installed which is not mentioned in the docs.
https://github.com/langchain-ai/langchain/blob/21a153894917e530cbe82a778be6f9cf10c9ae5f/docs/docs/get_started/quickstart.mdx#L185C1-L194C1

### Idea or request for content:

I think it should be mentioned just like `faiss` a few lines below.
Rschwabco/pinecone client support
Labels: name: area: vector store, name: auto:improvement, name: integration: pinecone, name: size:M

---
title: "community: Fix use_pod_based parameter in Pinecone class"
---

- Description: This PR adds an `use_pods_based` parameter which is optional and `True` by default. This is to support upcoming changes to the Pinecone client.
DOC: Page Not Found when clicking on different agent types in table
Labels: name: area: agent, name: auto:documentation

### Issue with current documentation:

Clicking on any of the `agent_types` [here](https://python.langchain.com/docs/modules/agents/agent_types) leads to faulty links with the following message:

Example from this link: https://python.langchain.com/docs/modules/agents/openai_tools

<img width="873" alt="image" src="https://github.com/langchain-ai/langchain/assets/8833114/d6d5268c-397a-4eda-9a07-e5ec4b4b2d13">
 

### Idea or request for content:

Update table to point to correct hyperlink i.e. https://python.langchain.com/docs/modules/agents/agent_types/openai_tools for the example above.
Cannot combine an output parser and a conversation buffer memory
Labels: name: area: memory, name: auto:bug

### Checked other resources

- [X] I added a very descriptive title to this issue.
- [X] I searched the LangChain documentation, with the integrated search.
- [X] I used the GitHub search to find a similar question and didn't find it.

### Example Code

```
from langchain.output_parsers import PydanticOutputParser
from pydantic import BaseModel, Field
from langchain.memory import ConversationBufferMemory

from langchain.prompts.chat import ChatPromptTemplate, SystemMessagePromptTemplate, HumanMessagePromptTemplate, MessagesPlaceholder
from langchain.chains import LLMChain


class TennisPlayer(BaseModel):
    age: int = Field(description="Age of the player")
    nb_victories: int = Field(description="Nb of victories in major tournaments")

llm = None # Instantiate the LLM here

parser = PydanticOutputParser(pydantic_object=TennisPlayer)

prompt = "You'll be asked information about a tennis player.\n" \
         "You'll answer with the following format:\n" \
         "{format_instructions}"

memory = ConversationBufferMemory(memory_key="chat_history", input_key="query", return_messages=True)
chat_prompt = ChatPromptTemplate.from_messages([SystemMessagePromptTemplate.from_template(prompt),
                                                MessagesPlaceholder(variable_name="chat_history"),
                                                HumanMessagePromptTemplate.from_template("{query}")])

chain = LLMChain(llm=llm, prompt=chat_prompt, memory=memory, output_parser=parser)

chain.invoke(input={"query": "Rafael Nadal", "format_instructions": parser.get_format_instructions()})
```

### Description

The previous code triggers an error while converting the output from the LLM to an AIMessage to place in the ConversationBufferMemory object. The problem is that it passes the constructed object (the output of PydanticOutputParser.parse) instead of the output message as a plain string.

### System Info

langchain 0.1.0
python 3.10.13
Windows 10

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [X] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [X] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async
Intercepting the output message in a callback handler before it is sent to the output parser
Labels: name: auto:bug, name: area: models

### System Info

langchain v0.1.0

### Who can help?

@agola11

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [X] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [X] Callbacks/Tracing
- [ ] Async

### Reproduction

```
from langchain.output_parsers import PydanticOutputParser
from pydantic import BaseModel, Field
from typing import Any, Dict
from langchain_core.outputs.llm_result import LLMResult

from langchain.prompts.chat import ChatPromptTemplate, HumanMessagePromptTemplate
from langchain.chains import LLMChain
from langchain.callbacks.base import BaseCallbackHandler


class CustomCallBack(BaseCallbackHandler):
    def on_llm_end(self, response: LLMResult, **kwargs: Any) -> Any:
         print(f"on_llm_end => {response}")

    def on_chain_end(self, outputs: Dict[str, Any], **kwargs: Any) -> Any:
        print(f"on_chain_end => {outputs}")

    def on_text(self, text: str, **kwargs: Any) -> Any:
        print(f"on_text => {text}")

class TennisPlayer(BaseModel):
    age: int = Field(description="Age of the player")
    nb_victories: int = Field(description="Nb of victories in major tournaments")

# Instantiate the LLM here
llm = None

parser = PydanticOutputParser(pydantic_object=TennisPlayer)

prompt = "Give me some information about Rafael Nadal.\n" \
         "You'll answer with the following format:\n" \
         "{format_instructions}"

chat_prompt = ChatPromptTemplate.from_messages([HumanMessagePromptTemplate.from_template(prompt)])

chain = LLMChain(llm=llm, prompt=chat_prompt, callbacks=[CustomCallBack()], output_parser=parser)

chain.invoke(input={"format_instructions": parser.get_format_instructions()})
```

### Expected behavior

The custom callback handler makes it possible to intercept the prompt sent to the LLM (through _on_text_) and the output in _on_chain_end_. The problem is that when an output parser is involved, the _outputs_ dictionary of _on_chain_end_ associates the "text" key with the final constructed object and not the output message (containing the JSON data that has been marshalled to an object). And for an unknown reason, the _on_llm_end_ callback function isn't invoked...

When something goes wrong in the marshalling process, the analysis of the LLM's output message is mandatory. Well, it doesn't seem abnormal to get the final object produced by the chain in the _on_chain_end_ callback, but in that case I would expect the _on_llm_end_ callback to be called just before with the output message in parameter. But it is not. So, at this stage, it's not possible to intercept the raw LLM's output message for debugging purposes.
community: Handle HuggingFaceInferenceAPI failure response of POST request
Labels: name: area: embeddings, name: auto:improvement, name: size:XS

  - **Description:** 
 Currently the embed_documents() function of HuggingFaceInferenceAPIEmbeddings class posts a request to HuggingFaceInference API to generate text embeddings. The POST request may not always return successful response due to various technical glitches. This PR handles the event when the status code of the response returned by HuggingFaceInference API is not 200 and raises an exception for this event.

  - **Dependencies:** None
  - **Twitter handle:** None
Enable customizing the output parser of `OpenAIFunctionsAgent`
Labels: name: area: agent, name: lgtm, name: auto:improvement, name: size:S

  - **Description:** This PR defines the output parser of OpenAIFunctionsAgent as an attribute, enabling customization and subclassing of the parser logic. 
  - **Issue:** Subclassing is currently impossible as the `OpenAIFunctionsAgentOutputParser` class is hard coded into the `plan` and `aplan` methods
  - **Dependencies:** None

<!-- Thank you for contributing to LangChain!

Please title your PR "<package>: <description>", where <package> is whichever of langchain, community, core, experimental, etc. is being modified.

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes if applicable,
  - **Dependencies:** any dependencies required for this change,
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` from the root of the package you've modified to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://python.langchain.com/docs/contributing/

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/docs/integrations` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
community: Handle HuggingFaceInferenceAPI failure response of POST request
Labels: name: area: embeddings, name: auto:bug, name: size:XS

- **Description:**
 Currently the _embed_documents()_ function of _HuggingFaceInferenceAPIEmbeddings_ class posts a request to HuggingFaceInference API to generate text embeddings. The POST request may not always return successful response due to various technical glitches. This PR handles the event when the status code of the response returned by HuggingFaceInference API is not 200 and raises an exception for this event.

  - **Dependencies:** None
  - **Twitter handle:** None
langchain, community, core: Async Cache support introduced
Labels: name: auto:improvement, name: area: models, name: integration: redis, name: size:L

# Description
We're using `FastAPI` in our current project with async functions from LangChain. Recently we needed to add `Redis`-based cache. However we quickly learned that regardless of the LangChain way of usage - sync or async - the cache lookup/update/clear will happen always in synchronous manner. Naturally, this means blocking the event loop, especially with slower than Redis cache implementations.

So I took a similar approach to `agenerate` and `ainvoke` for LLMs, and created an `AsyncBaseCache` mixin interface with corresponding `alookup/aupdate/aclear` async variants:
```python
class AsyncBaseCache(ABC):
    """Base interface for async cache."""

    @abstractmethod
    async def alookup(self, prompt: str, llm_string: str) -> Optional[RETURN_VAL_TYPE]:
        """Look up based on prompt and llm_string."""

    @abstractmethod
    async def aupdate(
        self, prompt: str, llm_string: str, return_val: RETURN_VAL_TYPE
    ) -> None:
        """Update cache based on prompt and llm_string."""

    @abstractmethod
    async def aclear(self, **kwargs: Any) -> None:
        """Clear cache that can take additional keyword arguments."""
```

The reference implementation is done for `InMemoryCache` and `RedisCache`. 

The `InMemoryCache` does not really need async variant, since there is no real IO, however having that this is the most convenient way to unit test changes.

The change spans throughout `core`, `community` and `langchain`. Linter, formatters, tests are passing.

# Issue
Fixes inability to use IO-bound Redis cache w/o blocking event loop. Opens opportunity for other cache implementations to have async variants.

# Dependencies
None

# Twitter
[@Mind_Clash](https://twitter.com/Mind_Clash)
document_loaders to support BytesIO or an interface for in-memory objects
Labels: name: area: doc loader, name: auto:improvement

### Feature request

The current document_loaders accept file path to process. But most of the time, especially if application deployed to somewhere, file is uploaded by user and not exist on file system.

Writing that in-memory bytes to disk and re-read is a unnecessary step.

It would be good to take BytesIO or some abstraction to process in-memory files.

### Motivation

It will eliminate writing in-memory files to disk and re-reading them from disk while using document_loaders.

### Your contribution

I can create a PR for this.
AzureChatOpenAI: `Configuration key azure_deployment not found in client`
Labels: name: auto:bug, name: area: models

### System Info

LC version: 0.1.0
Platform: MacOS
Python version: 3.12.1

### Who can help?

@hwchase17 @agola11 

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

```
Python 3.12.1 (main, Jan  9 2024, 18:02:09) [Clang 15.0.0 (clang-1500.0.40.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from langchain_openai import AzureChatOpenAI
>>> from langchain_core.runnables import ConfigurableField
>>> ConfigurableAzureChatOpenAI = AzureChatOpenAI(
...     openai_api_key = "asdfg",
...     openai_api_version = "asdg",
...     deployment_name='asdg',
...     azure_endpoint="asdg",
...     temperature=0.9
...     ).configurable_fields(
...       azure_endpoint=ConfigurableField(id="azure_endpoint"),
...       openai_api_key=ConfigurableField(id="openai_api_key"),
...       azure_deployment=ConfigurableField(id="deployment_name"),
...       openai_api_version=ConfigurableField(id="openai_api_version"),
... )
Traceback (most recent call last):
  File "<stdin>", line 7, in <module>
  File "/Users/pramodh/.pyenv/versions/3.12.1/lib/python3.12/site-packages/langchain_core/runnables/base.py", line 1368, in configurable_fields
    raise ValueError(
ValueError: Configuration key azure_deployment not found in client=<openai.resources.chat.completions.Completions object at 0x1079f4350> async_client=<openai.resources.chat.completions.AsyncCompletions object at 0x1079f58e0> temperature=0.9 openai_api_key='asdfg' openai_proxy='' azure_endpoint='asdg' deployment_name='asdg' openai_api_version='asdg' openai_api_type='azure': available keys are {self.__fields__.keys()}
```

### Expected behavior

`azure_deployment` is an alias for `deployment_name` defined inside `AzureChatOpenAI`, but it cannot be set as a `ConfigurableField` - we instead have to set `deployment_name` as a ConfigurableField.

I would expect the above code to not throw an error, as they are just aliases.
Luzhan docs update
Labels: name: auto:documentation, name: size:L

<!-- Thank you for contributing to LangChain!

Please title your PR "<package>: <description>", where <package> is whichever of langchain, community, core, experimental, etc. is being modified.

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes if applicable,
  - **Dependencies:** any dependencies required for this change,
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` from the root of the package you've modified to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://python.langchain.com/docs/contributing/

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/docs/integrations` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
DOC: <https://python.langchain.com/docs/integrations/chat/ollama_functions 'DOC: ' prefix>ollamafunctions not working at all
Labels: name: auto:bug, name: area: models

### Issue with current documentation:

i have tried with several tests, even with the most basic e.g in the doc, nothing. Dissapointed because it got me superexcited at first:
from langchain_experimental.llms.ollama_functions import OllamaFunctions
from langchain.schema import HumanMessage

model = OllamaFunctions(model="dolphinmodel",)


model = model.bind(
    functions=[
        {
            "name": "get_current_weather",
            "description": "Get the current weather in a given location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "The city and state, " "e.g. San Francisco, CA",
                    },
                    "unit": {
                        "type": "string",
                        "enum": ["celsius", "fahrenheit"],
                    },
                },
                "required": ["location"],
            },
        }
    ],
    function_call={"name": "get_current_weather"},
)


model.invoke("what is the weather in Boston?")

### Idea or request for content:

_No response_
Docs: Fix a typo in elasticsearch vectorstore notebook
Labels: name: lgtm, name: area: vector store, name: auto:documentation, name: integration: elasticsearch, name: size:XS

<!-- Thank you for contributing to LangChain!

Please title your PR "<package>: <description>", where <package> is whichever of langchain, community, core, experimental, etc. is being modified.

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes if applicable,
  - **Dependencies:** any dependencies required for this change,
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` from the root of the package you've modified to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://python.langchain.com/docs/contributing/

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/docs/integrations` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Why can't I store and retrieve vectors? Please help me fix it.
Labels: name: area: vector store, name: auto:bug

### Issue you'd like to raise.

import os

from langchain.prompts.prompt import PromptTemplate
from langchain.llms import OpenAI
from langchain.chains.question_answering import load_qa_chain
from langchain.chains import (
    ConversationalRetrievalChain,
    LLMChain
)
from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler
from langchain_community.chat_models import ChatOpenAI
from langchain_community.embeddings import HuggingFaceEmbeddings
from langchain_core.callbacks import CallbackManager
from qdrant_client import QdrantClient
from langchain.vectorstores import Qdrant

os.environ['OPENAI_API_KEY'] = "mykey"
embeddings = HuggingFaceEmbeddings(
    model_name="all-MiniLM-L6-v2"
)
llm_name = "gpt-3.5-turbo"
llm = ChatOpenAI(model_name=llm_name, temperature=0)


streaming_llm = OpenAI(
    streaming=True,
    callback_manager=CallbackManager([
        StreamingStdOutCallbackHandler()
    ]),
    verbose=True,
    max_tokens=150,
    temperature=0.2
)

condense_question_prompt = PromptTemplate.from_template(
    "在幹嘛"
)
qa_prompt = PromptTemplate.from_template("測")


question_generator = LLMChain(
    llm=llm,
    prompt=condense_question_prompt
)
doc_chain = load_qa_chain(
    llm=streaming_llm,
    chain_type="stuff",
    prompt=qa_prompt
)
client = QdrantClient(host="192.168.0.31", port=6333)
collection_name = "test"
vectorstore = Qdrant(client, collection_name,
                     embedding_function=embeddings.embed_query)

chatbot = ConversationalRetrievalChain(
    retriever=vectorstore.as_retriever(),
    combine_docs_chain=doc_chain,
    question_generator=question_generator
)

chat_history = []
question = input("Hi! What are you looking for today?")
while True:
    result = chatbot(
        {"question": question, "chat_history": chat_history}
    )
    print("\n")
    chat_history.append((result["question"], result["answer"]))
    question = input()
![image](https://github.com/langchain-ai/langchain/assets/103471919/7e0da9ed-56d1-412d-91b3-8b7acac1ddce)


### Suggestion:

Why can't I store and retrieve vectors? Please help me fix it.