delayedkarma/langchain-issues huggingface dataset:
Issue: How Can I use Prompt Template?
Labels: name: auto:question, name: area: models

### Issue you'd like to raise.

#prompt template
How Can I use Prompt Template in my code below?
def chat_langchain(new_project_qa, query, not_uuid):
    check = query.lower()
    user_experience_inst = UserExperience.objects.get(not_uuid=not_uuid)
    # query="If the context is related to hi/hello with or without incorrect spelling then reply Hi! How can I assist you today?,else Search the entire context and provide formal and accurate answer for this query - {}. Explain the relevant information with important points, if necessary.If you don't find answer then give relavant answer else reply with text 'Sorry, I can't find the related information' ".format(check)
    if check not in ['hi','hello','hey','hui','hiiii','hii','hiii','heyyy'] and not user_experience_inst.custom_prompt:
        query = "Search the entire context and provide formal and accurate answer for this query - {}. Explain the relevant information with important points, if necessary.If you don't find answer then reply with text 'Sorry, I can't find the related information' otherwise give relavant answer".format(check)
    elif check not in ['hi','hello','hey','hui','hiiii','hii','hiii','heyyy'] and user_experience_inst.custom_prompt:
        query = f"{user_experience_inst.custom_prompt} {check}.If you don't find answer then reply with text 'Sorry, I can't find the related information'"
    else:
        query="Search the entire context and provide formal and accurate answer for this query - {}. Explain the relevant information with important points, if necessary.".format(check)
    result = new_project_qa(query)
    relevant_document = result['source_documents']
    if relevant_document:
        source = relevant_document[0].metadata.get('source', '')
        # Check if the file extension is ".pdf"
        file_extension = os.path.splitext(source)[1]
        if file_extension.lower() == ".pdf":
            source = os.path.basename(source)

        # Retrieve the UserExperience instance using the provided not_uuid
        user_experience_inst = UserExperience.objects.get(not_uuid=not_uuid)
        bot_ending = user_experience_inst.bot_ending_msg if user_experience_inst.bot_ending_msg is not None else ""

        # Create the list_json dictionary
        if bot_ending != '':
            list_json = {
                'bot_message': result['result'] + '\n\n' + str(bot_ending),
                "citation": source
            }
        else:
            list_json = {
                'bot_message': result['result'] + str(bot_ending),
                "citation": source
            }
    else:
        # Handle the case when relevant_document is empty
        list_json = {
            'bot_message': result['result'],
            'citation': ''
        }

    # Return the list_json dictionary
    return list_json


### Suggestion:

_No response_
custom_table_info along with sql_db_schema while using create_sql_agent
Labels: name: auto:improvement, name: area: models

### Feature request

When utilizing custom_table_info in the sqldatabase instance while employing the create_sql_agent function, it appears that there is an issue where it disregards the sql_db_schema. Currently, it only utilizes either the custom_table_info or the sql_db_schema. This poses a challenge, especially when crucial information, such as identifying which ID column corresponds to other tables, cannot be specified in the custom table info. There is a need for an option to use both the custom table information and the sql_db_schema 


table_info={'invoice':" the customer_id in invoice table is referenced to customers table's company_id",}

db=SQLDatabase(engine=dbengine,include_tables=["invoice","customer"],custom_table_info=table_info)

Invoking: `sql_db_schema` with `customer, invoice`



CREATE TABLE customer (
        id SERIAL NOT NULL, 
        key VARCHAR NOT NULL, 
        company_id VARCHAR NOT NULL, 
        company_name VARCHAR NOT NULL, 
        
)

/*
1 rows from customer table:
id      key     company_id      company_name   
670     CUST-0ab15       17      Aim Inc
*/

 the customer_id in invoice table is referenced to customers table's company_id,
 
 It doesnt have the schema of invoice table so generating wrong sql queries

### Motivation

I need both custom_table_info and sql_db_schema to work, as some kinda metadata is needed to specify that is specific to my use cases.

### Your contribution
NO
DOC: `SupabaseVectorStore` support for similarity `score_threshold` filtering in `as_retriever`
Labels: name: area: vector store, name: auto:improvement, name: auto:documentation

### Issue with current documentation:

The supabase vectorstore does not support setting the `score_threshold` in `as_retriever` despite being showcased as an option in the vectorestore superclass docstring example. 

https://github.com/langchain-ai/langchain/blob/a05230a4ba4dee591d3810440ce65e16860956ae/libs/langchain/langchain/vectorstores/supabase.py#L218
https://github.com/langchain-ai/langchain/blob/a05230a4ba4dee591d3810440ce65e16860956ae/libs/core/langchain_core/vectorstores.py#L596

### Idea or request for content:

The VectoreStore superclass of SupabaseVectoreStore contains logic in `similarity_search_by_vector_with_relevance_scores` that could be used in the SupabaseVectorStore subclass to support the `score_threshold` parameter.
import error in elasticsearch memory module: Module cannot be used as a type
Labels: name: area: memory, name: auto:bug, name: integration: elasticsearch

### System Info

langchain version: 0.0.348

for the type hint for es_connection variable in class ElasticsearchChatMessageHistory, module is used as a type

@hwchase17  @eyurtsev 

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [X] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

import json
import logging
from time import time
from typing import TYPE_CHECKING, Any, Dict, List, Optional

from langchain_core.chat_history import BaseChatMessageHistory
from langchain_core.messages import (
    BaseMessage,
    message_to_dict,
    messages_from_dict,
)

if TYPE_CHECKING:
    from elasticsearch import Elasticsearch

logger = logging.getLogger(__name__)


class ElasticsearchChatMessageHistory(BaseChatMessageHistory):
    """Chat message history that stores history in Elasticsearch.

    Args:
        es_url: URL of the Elasticsearch instance to connect to.
        es_cloud_id: Cloud ID of the Elasticsearch instance to connect to.
        es_user: Username to use when connecting to Elasticsearch.
        es_password: Password to use when connecting to Elasticsearch.
        es_api_key: API key to use when connecting to Elasticsearch.
        es_connection: Optional pre-existing Elasticsearch connection.
        index: Name of the index to use.
        session_id:Arbitrary key that is used to store the messages
            of a single chat session. 
    """

    def __init__(
        self,
        index: str,
        session_id: str,
        *,
        es_connection: Optional["**Elasticsearch**"] = None,
        es_url: Optional[str] = None,
        es_cloud_id: Optional[str] = None,
        es_user: Optional[str] = None,
        es_api_key: Optional[str] = None,
        es_password: Optional[str] = None,
    ):
        self.index: str = index
        self.session_id: str = session_id

### Expected behavior

import of module elasticsearch is not done properly
Improvement: Allow passing parameters to the underlying es_client. Closes: #14403
Labels: name: lgtm, name: area: vector store, name: auto:improvement, name: integration: elasticsearch, name: size:XS

### Description

In https://github.com/langchain-ai/langchain/issues/14403, the user mentioned that he hopes not to verify ssl and needs to pass more parameters

I found that the `Elasticsearch` class [has very many parameters](https://github.com/elastic/elasticsearch-py/blob/98f2af2134dfc3f2b2054b87a4056e2508b9b8db/elasticsearch/_sync/client/__init__.py#L131-L191
):

<img width="1097" alt="Screenshot 2023-12-08 at 4 24 39 PM" src="https://github.com/langchain-ai/langchain/assets/10000925/f2201554-b41a-4388-a8e8-c14a2d0466d4">

In order to adapt to more situations, I want to add the kwargs parameter so that users can enter more `Elasticsearch` parameters. Like [redis](https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/vectorstores/redis/base.py#L253), [tair](https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/vectorstores/tair.py#L32), [myscale](https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/vectorstores/myscale.py#L112) and so on.
Add option to preserve headers in MarkdownHeaderTextSplitter
Labels: name: lgtm, name: area: doc loader, name: auto:improvement, name: size:L

  - **Description:** `MarkdownHeaderTextSplitter` currently strips header lines from chunked content. Many applications require these header lines are preserved. This adds an optional parameter to preserve those headers in the chunked content.
  - **Issue:** #2836 (relevant)
  - **Dependencies:** -
  - **Tag maintainer:** @baskaryan
  - **Twitter handle:** @finnless

Unit tests and new examples in notebook included.

cc @rlancemartin
Improvement: Correct type annotation for azure_ad_token_provider Closed: #14402
Labels: name: lgtm, name: area: embeddings, name: auto:bug, name: size:XS

Description
Fix https://github.com/langchain-ai/langchain/issues/14402, Similar changes: https://github.com/langchain-ai/langchain/pull/14166

Twitter handle
[lin_bob57617](https://twitter.com/lin_bob57617)
Bug: ImportError for ElasticsearchStore
Labels: name: area: vector store, name: auto:bug, name: integration: elasticsearch

### System Info

Langchain Version: 0.0.346
Python: 3.11.4

### Who can help?

@eyurtsev 

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

```python
from langchain.vectorstores import ElasticsearchStore
```
Results in 
```
  File "C:\Users\XXX\Desktop\Projects\XXX\api\controllers\Vector.py", line 5, in <module>
    from langchain.vectorstores import ElasticsearchStore
ImportError: cannot import name 'ElasticsearchStore' from 'langchain.vectorstores'
```

or
```python
from langchain.vectorstores.elasticsearch import ElasticsearchStore
```
Results in 
```
  File "C:\Users\XXX\Desktop\Projects\XXX\api\controllers\Vector.py", line 5, in <module>
    from langchain.vectorstores.elasticsearch import ElasticsearchStore
ModuleNotFoundError: No module named 'langchain.vectorstores.elasticsearch'
```

### Expected behavior

I am upgrading from `langchain==0.0.279` to `langchain==0.0.346` and this is the issue that arised.

Expected behavior would be import successfully, new langchain version does not seems to be backward compatible for `ElasticsearchStore`
Retrieval Inability to Filter Based on Metadata in Milvius Database
Labels: name: area: agent, name: area: vector store, name: auto:bug

### System Info

Python ==3.11.3
pymilvus== 2.3.1
langchain==0.0.327
openai==0.28.1

### Who can help?

_No response_

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [X] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Description:
The issue encountered revolves around the inability of the retrieval process to filter data based on metadata, specifically focusing on the 'file_name' object in the metadata. The context pertains to utilizing Milvius as the database for the vector store.

Details:

Metadata Structure: The metadata comprises a list of dictionaries, where each dictionary holds key-value pairs containing metadata information. The 'file_name' attribute is utilized for filtration purposes.

Example Metadata List:
[{'page_number': '4', 'file_name': 'Apple_history.pdf', 'source_path': '.../Apple_history.pdf'}, ...]
Applied Options and Observations:

Attempted a approach by directly using a 'filter_query' with the 'file_name'.

filter_query = {"filter": {"file_name": 'samsung.pdf'}, "k": self.top}
retriever = vectorstore.as_retriever(search_kwargs=filter_query) # Retrieval of top k results
result = retriever.get_relevant_documents(agent_query)
Similar observations were made where the retrieval fetched data unrelated to the specified 'samsung.pdf' filename.

### Expected behavior

The anticipated functionality was to filter the retrieval process based on the 'file_name' metadata attribute. In scenarios where there are no chunks associated with the specified 'file_name', the retrieval should ideally return no data.
GITLAB_URL should take default https://gitlab.com instead of error
Labels: name: auto:bug, name: area: models, name: size:XS

The fix #14221 has broken default gitlab url which is forcing the users to specify GITLAB_URL  for default one.  With this fix if GITLAB_URL  is not set, the default gitlab url  will be taken.

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:**  Add a Default URL with https://gitlab.com instead of None
  - **Issue:** The fix #14221 has broken default gitlab url which is forcing the users to specify GITLAB_URL
  - **Dependencies:** None
  - **Tag maintainer:** @hwchase17
  - **Twitter handle:**  @manjunath_shiva

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
feat: patch chroma.similarity_search_with_score to return vector id
Labels: name: area: vector store, name: auto:improvement, name: integration: chroma, name: size:S

**Description:** In chroma, return the vector_ids for associated vectors when using the `similarity_search_with_score` and `similarity_search_by_vector_with_relevance_scores` so that turning around to manage the vector (update/delete) is instantly available. Right now, there is no easy way to get similarity and get the associated vector id for the results. Given the id on creation can be created in-library or _optionally_ set by the user we should greedily return the id so the user can always manage their vectors easily.
**Issue:** #11592,
**Dependencies:** N/A
**Tag maintainer:** N/A,
**Twitter handle:** @mintplexlabs
feat: masking api key for litellm 
Labels: name: auto:improvement, name: area: models, name: size:L

Description: Masking API key for LiteLLM Chat Model to protect user secrets.
Issue: https://github.com/langchain-ai/langchain/issues/12165
Dependencies: Litellm and LangChain 
Tag maintainer: @eyurtsev 


`make test` skips current unit_test and we are unable to install the dependency `litellm` due to a conflict with `tiktoken` See #14419
Dependency Conflict between litellm and tiktoken
Labels: name: area: doc loader, name: auto:bug, name: area: models

### System Info

Langchain Version: 0.0.348
Output from `poetry env info`
**Virtualenv**
Python:         3.9.12
Implementation: CPython
Path:           /Users/peternf/Desktop/langchain/libs/langchain/.venv
Executable:     /Users/peternf/Desktop/langchain/libs/langchain/.venv/bin/python
Valid:          True

**System**
Platform:   darwin
OS:         posix
Python:     3.9.12
Path:       /Users/peternf/opt/miniconda3
Executable: /Users/peternf/opt/miniconda3/bin/python3.9

### Who can help?

@eyurtsev 

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [X] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

### Reason to Install litellm
@xavier-xia-99 and I are collaborating on finishing the tasks of #12165 for _libs/langchain/langchain/chat_models/litellm.py_. We have finished the implementation and is adding unit tests to verify our logic when we discovered that we need to include litellm for the unit tests, or `make test` will just skip our newly added unit tests. Yet as we try to add the optional dependency litellm according to the instructions provided in langchain/.github/CONTRIBUTING.md, it shows the following dependency conflict. Hence, we will appreciate any help in addressing this conflict or running the CI. Thank you!

### Steps to Reproduce the Behavior

1. Run `poetry add --optional litellm`
2. Error Message: <img width="720" alt="Screenshot 2023-12-07 at 5 50 43 PM" src="https://github.com/langchain-ai/langchain/assets/98713019/3e0d5730-2134-4e11-b901-fbda927bc796">  


### Expected behavior

Output of litellm downloads in progress and successful installation of relevant packages.
Update prompts.py
Labels: name: auto:nit, name: area: models, name: size:XXL

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Issue: <Zilliz and Milvus metadata field and memory seperation>
Labels: name: area: memory, name: area: vector store, name: auto:question

### Issue you'd like to raise.


im trying to create a metadata to separate user id fields vectors,  I searched a lot and couldn't find anything , im using zilliz vector store , I tried using the userid as memory key but it's not possible , any ideas ? Im using the docs is automatically saved and retrieved from the saved collection on zilliz , so I can't edit the docs too , im just trying to add meta fields when user chats and when retrieved for context.
  ```  
vectordb = Milvus.from_documents(
        {} ,
            embeddings,

    

             connection_args={
        "uri": ZILLIZ_CLOUD_URI,
    
        "token": ZILLIZ_CLOUD_API_KEY,  # API key, for serverless clusters which can be used as replacements for user and password
        "secure": True,

             },
   
)
        # vectordb.
        retriever = Milvus.as_retriever(vectordb,search_kwargs= {"k":15 , 'user_id': "a3"}  ) # here we use userid with "a" for retreiving memory
        # print(retriever)
        memory= VectorStoreRetrieverMemory(retriever=retriever , memory_key="history" ,  metadata={"user_id": "a3"} )
        chain = ConversationChain(llm=self.llm, memory=memory, verbose=True,prompt = PROMPT , metadata={"user_id": "a3"})
   
        res = chain.predict(input=input_text)
        # with chain_recorder as recording:
        #     llm_response = chain(input_text)
   

        return    res
```



### Suggestion:

_No response_
'on_llm_new_token' was never awaited: resolved
Labels: name: auto:bug, name: area: models, name: size:XS

Issue #14399

  - **Description:** Added async to `on_llm_new_token` function in `CallbackManagerForLLMRun` class in `manager.py` file present in `libs/core/langchain_core`, 
  - **Issue:** It fixed the RuntimeWarning: coroutine 'AsyncCallbackManagerForLLMRun.on_llm_new_token' was never awaited ChatGPT',
  - **Twitter handle:** https://twitter.com/Triyambaka58284
  
It is passing the `make format`, `make lint` and `make test` to check this locally.

Kindly reviews my PR @baskaryan, @eyurtsev, @hwchase17.
FAISS db.delete() says `ids` is required even when it is Optional
Labels: name: area: vector store, name: auto:bug

### System Info

LangChain version : Latest

```python
    def delete(self, ids: Optional[List[str]] = None, **kwargs: Any) -> Optional[bool]:
```

[This check](https://github.com/langchain-ai/langchain/blob/54040b00a4a05e81964a1a7f7edbf0b830d4395c/libs/langchain/langchain/vectorstores/faiss.py#L798) causes the issue.

### Who can help?

@eyurtsev

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

```python
db = FAISS.from_documents(text_pages, embeddings)
db.delete()
```

Error

```
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[51], line 1
----> 1 db.delete()

File /opt/conda/lib/python3.10/site-packages/langchain/vectorstores/faiss.py:799, in FAISS.delete(self, ids, **kwargs)
    789 """Delete by ID. These are the IDs in the vectorstore.
    790 
    791 Args:
   (...)
    796     False otherwise, None if not implemented.
    797 """
    798 if ids is None:
--> 799     raise ValueError("No ids provided to delete.")
    800 missing_ids = set(ids).difference(self.index_to_docstore_id.values())
    801 if missing_ids:

ValueError: No ids provided to delete.
```

### Expected behavior

The index should be deleted without needing to pass an index `id`.
Vectara summarization
Labels: name: area: vector store, name: auto:enhancement, name: size:XXL

  - **Description:** Adding Summarization to Vectara, to reflect it provides not only vector-store type functionality but also can return a summary. 
Also added:
* MMR capability (in the Vectara platform side)
* Updated templates
* Updated documentation and IPYNB examples

  - **Tag maintainer:** @baskaryan
  - **Twitter handle:** @ofermend