delayedkarma/langchain-issues huggingface dataset:
Start cookbook and move stuff from use cases
Labels: name: auto:documentation

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Add Support for Azure Cosmos DB MongoDB vCore Vector Store #11627
Labels: name: lgtm, name: area: vector store, name: auto:enhancement

This PR adds support for the Azure Cosmos DB MongoDB vCore Vector Store

https://learn.microsoft.com/en-us/azure/cosmos-db/mongodb/vcore/
https://learn.microsoft.com/en-us/azure/cosmos-db/mongodb/vcore/vector-search

Summary:
  - **Description:** added vector store integration for  Azure Cosmos DB MongoDB vCore Vector Store, 
  - **Issue:** the issue # it fixes #11627,
  - **Dependencies:** pymongo dependency,
  - **Tag maintainer:** @hwchase17,
  - **Twitter handle:** @izzyacademy
Issue as a result of Chroma.from_documents, OpenAI and AWS API Gateway
Labels: name: area: embeddings, name: area: vector store, name: auto:bug, name: integration: chroma, name: integration: aws

### System Info

langchain version: 0.0.268
python version: 3.11
MacOS: Ventura 13.6


### Who can help?

@agola11 

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [X] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Using a Chroma vector store's from_documents method through an AWS API Gateway. The response size appears to be greater than 6MB. Is there a way to send a request whos response becomes smaller than 6MB?



Retrying langchain.embeddings.openai.embed_with_retry.<locals>._embed_with_retry in 10.0 seconds as it raised APIError: HTTP code 501 from API (Response size larger than Lambda 6MB Limit).
Traceback (most recent call last):
  File "/Users/john_ramos/mckinsey-git/policy-y/.venv/lib/python3.11/site-packages/openai/api_requestor.py", line 753, in _interpret_response_line
    data = json.loads(rbody)
           ^^^^^^^^^^^^^^^^^
  File "/usr/local/Cellar/python@3.11/3.11.4_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/Cellar/python@3.11/3.11.4_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/Cellar/python@3.11/3.11.4_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

### Expected behavior

Embeddings are stored in the data store after the response comes back through API gateway.
Fixed the assignment of custom_llm_provider argument
Labels: name: lgtm, name: auto:bug, name: area: models

  - **Description:** Assigning the custom_llm_provider to the default params function so that it will be passed to the  litellm
  - **Issue:** Even though the custom_llm_provider argument is being defined it's not being assigned anywhere in the code and hence its not being passed to litellm, therefore any litellm call which uses the custom_llm_provider as required parameter is being failed. This parameter is mainly used by litellm when we are doing inference via Custom API server. https://docs.litellm.ai/docs/providers/custom_openai_proxy
  - **Dependencies:** No dependencies are required

@krrishdholakia , @baskaryan
Add AzureCosmosDBVectorSearch VectorStore
Labels: name: area: vector store, name: auto:enhancement

### Feature request

### Feature request

Azure Cosmos DB for MongoDB vCore enables users to efficiently store, index, and query high dimensional vector data stored directly in Azure Cosmos DB for MongoDB vCore. It contains similarity measures such as COS (cosine distance), L2 (Euclidean distance) or IP (inner product) which measures the distance between the data vectors and your query vector. The data vectors that are closest to your query vector are the ones that are found to be most similar semantically and retrieved during query time. The accompanying PR would add support for Langchain Python users to store vectors from document embeddings generated from APIs such as Azure OpenAI Embeddings or Hugging Face on Azure.

[Azure Cosmos DB for MongoDB vCore](https://learn.microsoft.com/en-us/azure/cosmos-db/mongodb/vcore/vector-search)




### Motivation

This capability described in the feature request is currently not available for Langchain Python.

### Your contribution

I will be submitting a PR for this feature request.
similaritySearchWithScore and similaritySearch method signatures are identical in javascript
Labels: name: area: vector store, name: auto:bug

### System Info

The method signatures of similaritySearchWithScore and similaritySearch are identical in Javascript. I assume this is a mistake.

```
export declare abstract class VectorStore extends Serializable {
  ...
    similaritySearch(query: string, k?: number, filter?: this["FilterType"] | undefined, _callbacks?: Callbacks | undefined): Promise<Document[]>;
    similaritySearchWithScore(query: string, k?: number, filter?: this["FilterType"] | undefined, _callbacks?: Callbacks | undefined): Promise<[Document, number][]>;
```


### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Open the base.d.ts file and search for the abstract class VectorStore. It contains these methods.

### Expected behavior

I would assume the signature for similaritySearchWithScore should contain some kind of Score parameter that we could pass to it to only return documents with a certain score.
Release 3.0.4 breaks compatibility with ddtrace.
Labels: name: area: embeddings, name: auto:bug

### System Info

https://github.com/langchain-ai/langchain/releases/tag/v0.0.304

Reverting to 3.0.3 or under fixes it. The error looks something like (truncated confidential parts of the trace)

```
Traceback (most recent call last):
  ...
  File "/Users/sid.gupta/.pyenv/versions/3.11.3/lib/python3.11/site-packages/ddtrace/_monkey.py", line 205, in patch_all
    patch(raise_errors=False, **modules)
  File "/Users/sid.gupta/.pyenv/versions/3.11.3/lib/python3.11/site-packages/ddtrace/_monkey.py", line 245, in patch
    when_imported(module)(
  File "/Users/sid.gupta/.pyenv/versions/3.11.3/lib/python3.11/site-packages/ddtrace/vendor/wrapt/importer.py", line 284, in register
    register_post_import_hook(hook, name)
  File "/Users/sid.gupta/.pyenv/versions/3.11.3/lib/python3.11/site-packages/ddtrace/vendor/wrapt/decorators.py", line 470, in _synchronized
    return wrapped(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/sid.gupta/.pyenv/versions/3.11.3/lib/python3.11/site-packages/ddtrace/vendor/wrapt/importer.py", line 82, in register_post_import_hook
    hook(module)
  File "/Users/sid.gupta/.pyenv/versions/3.11.3/lib/python3.11/site-packages/ddtrace/_monkey.py", line 168, in on_import
    imported_module.patch()
  File "/Users/sid.gupta/.pyenv/versions/3.11.3/lib/python3.11/site-packages/ddtrace/contrib/langchain/patch.py", line 763, in patch
    if hasattr(langchain.embeddings, text_embedding_model):
               ^^^^^^^^^^^^^^^^^^^^
  File "/Users/sid.gupta/.pyenv/versions/3.11.3/lib/python3.11/site-packages/langchain/__init__.py", line 332, in __getattr__
    raise AttributeError(f"Could not find: {name}")
AttributeError: Could not find: embeddings
```



### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Be on `ddtrace 1.17.6` and a langchain version >= 3.0.4. Running an webapp on hypercorn and fastAPI.

### Expected behavior

Don't throw this trace and block the application.
AWS Sagemaker "code":424,
Labels: name: auto:bug, name: area: models, name: integration: aws

### System Info

from langchain.docstore.document import Document
from typing import Dict
from langchain import PromptTemplate, SagemakerEndpoint
from langchain.llms.sagemaker_endpoint import LLMContentHandler
from langchain.chains.question_answering import load_qa_chain
import json

example_doc_1 = """
Peter and Elizabeth took a taxi to attend the night party in the city. While in the party, Elizabeth collapsed and was rushed to the hospital.
Since she was diagnosed with a brain injury, the doctor told Peter to stay besides her until she gets well.
Therefore, Peter stayed with her at the hospital for 3 days without leaving.
"""

docs = [
    Document(
        page_content=example_doc_1,
    )
]

query = """How long was Elizabeth hospitalized?
"""

prompt_template = """Use the following pieces of context to answer the question at the end.

{context}

Question: {question}
Answer:"""
PROMPT = PromptTemplate(
    template=prompt_template, input_variables=["context", "question"]
)

class ContentHandler(LLMContentHandler):
    content_type = "application/json"
    accepts = "application/json"

    def transform_input(self, prompt: str, model_kwargs: Dict) -> bytes:
        input_dict = {"inputs": prompt, "parameters": model_kwargs}
        return json.dumps(input_dict).encode('utf-8')

    def transform_output(self, output: bytes) -> str:
        response_json = json.loads(output.read().decode("utf-8"))
        return response_json[0]["generated_text"]

content_handler = ContentHandler()

chain = load_qa_chain(
    llm=SagemakerEndpoint(
        endpoint_name="endpoint",
        region_name="ap-southeast-2",
        credentials_profile_name="dev",
        model_kwargs={"temperature": 1e-10},
        endpoint_kwargs={"CustomAttributes": 'accept_eula=true'},
        content_handler=ContentHandler(),
        
    ),
    prompt=PROMPT,
)

chain({"input_documents": docs, "question": query}, return_only_outputs=True)

Stacktrace :

ValueError: Error raised by inference endpoint: An error occurred (ModelError) when calling the InvokeEndpoint operation: Received client error (424) from primary with message "{
  "code":424,
  "message":"prediction failure",
  "error":"string indices must be integers"

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

run the same python code to reproduce. 

### Expected behavior

I am expecting a response generated by the chain.
Make metadata from the url_selenium loader match that of the web_base loader
Labels: name: area: doc loader, name: auto:improvement


**Description:** I noticed the metadata returned by the url_selenium loader was missing several values included by the web_base loader. (The former returned `{source: ...}`, the latter returned `{source: ..., title: ..., description: ..., language: ...}`.) This change fixes it so both loaders return all 4 key value pairs.

Files have been properly formatted and all tests are passing. Note, however, that I am not much of a python expert, so that whole "Adding the imports inside the code so that tests pass" thing seems weird to me. Please LMK if I did anything wrong.
Self-querying retriever for MongoDBAtlasVectorSearch
Labels: name: area: vector store, name: auto:enhancement

### Feature request

Add self-querying capability for MongoDBAtlasVectorSearch so that any natural language query could be converted to a MongoDB query and then be used to search the MongoDB vector store

### Motivation

I was testing self-quering using MongoDBAtlasVectorSearch when I got an error saying that self-querying is not supported. I think it will be useful to have that functionality since MongoDB vector store is well used.

### Your contribution

We will work on this feature and submit a pull request in the coming 1-2 months.
Add yandex llm and chat model
Labels: name: auto:enhancement, name: area: models

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Fix: invalid link to chat model in openai platform docs
Labels: name: auto:documentation

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->

There is some invalid link in open ai platform [docs](https://python.langchain.com/docs/integrations/platforms/openai).
So i fixed it to valid links.
- `/docs/integrations/chat_models/openai` -> `/docs/integrations/chat/openai`
- `/docs/integrations/chat_models/azure_openai` -> `/docs/integrations/chat/azure_chat_openai`

Thanks! ☺️
Add support for ChatOpenAI models in Infino callback handler
Labels: name: auto:improvement, name: area: models

**Description:** This PR adds support for ChatOpenAI models in the Infino callback handler. In particular, this PR implements `on_chat_model_start` callback, so that ChatOpenAI models are supported. With this change, Infino callback handler can be used to track latency, errors, and prompt tokens for ChatOpenAI models too (in addition to the support for OpenAI and other non-chat models it has today). The existing example notebook is updated to show how to use this integration as well. cc/ @naman-modi @savannahar68

**Issue:** https://github.com/langchain-ai/langchain/issues/11607 

**Dependencies:** None

**Tag maintainer:** @hwchase17 

**Twitter handle:** [@vkakade](https://twitter.com/vkakade)
Support ChatOpenAI models in Infino callback manager
Labels: name: auto:enhancement, name: area: models

### Feature request

[Infino callback handler](https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/callbacks/infino_callback.py) as of this writing does not support ChatOpenAI models, as it does not support `on_chat_model_start` callback.

Adding this callback will enable users to track latency, errors and token usage for ChatOpenAI models (in addition to existing support for OpenAI and other non-chat models).

### Motivation

Infino customers have requested for this integration, as this increases Infino callback handler's coverage to OpenAI chat models.

Customer request GitHub issue for Infino - https://github.com/infinohq/infino/issues/93

### Your contribution

I have a working code change for this issue, and will submit a PR shortly.
langchain is not working on AWS , It is always giving "ImportError: libmariadb.so.3: cannot open shared object file: No such file or directory" , no where i am using mariadb
Labels: name: auto:bug, name: integration: aws

### System Info

openai==0.27.6
urllib3==1.26.15
pandas==2.0.1
slack-sdk==3.21.3
pydantic==2.4.2
langchain==0.0.311
SQLAlchemy==2.0.11
mysqlclient==2.2.0
pymysql==1.1.0

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

i am using AWS lambda

### Expected behavior

it should run fine without MariaDB
Fix mutation bugs in callback manager configure
Labels: name: auto:bug

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Add configurable fields with options
Labels: name: auto:improvement

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Add id in the metadata of the Documents returned in Similarity Search
Labels: name: area: vector store, name: auto:improvement, name: integration: chroma

The documents returned in any similarity search function doesnt returns the id of the document. This id can later be use to update the page_content or the metadata of the stored document in the vector store.

@hwchase17 

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->