delayedkarma/langchain-issues huggingface dataset:
Add RAG template for Timescale Vector
Labels: name: area: vector store, name: auto:enhancement

**Description:**  Add RAG template for Timescale Vector

Also fix UTM links to ensure 90-day trial.

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Feat auto hftextgen
Labels: name: auto:enhancement, name: area: models

**Description**

I have developed a class `HuggingFaceTextGenInferenceAuto` to automate the process of managing Docker inference server containers for the `HuggingFaceTextGenerationInference` LLM. It essentially acts as a factory to create an instance of `HuggingFaceTextGenInference` via the `from_docker` classmethod. This class can create, reuse, or recreate containers as-needed based on parameter changes.

The objective is to simplify the management of HuggingFace TextGen inference servers, and allow users to spin up Docker containers for model inference with ease. The code also contains methods to handle the health checks, wait for model downloads, and ensure container and API readiness before proceeding.

**Issue**
See feature request issue #12564 

**Dependencies**

Updated toml and lockfile per contrib guidelines.
```
docker = "^6.1.3"
pytest-skip-slow = "^0.0.5"
text-generation = "^0.6.1"
```

**Twitter Handle**: [@daniel_c_barker](https://twitter.com/daniel_c_barker)

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Install and use `ruff format` instead of black for code formatting.
Labels: name: auto:refactor

Best to review one commit at a time, since two of the commits are 100% autogenerated changes from running `ruff format`:
- Install and use `ruff format` instead of black for code formatting.
- Output of `ruff format .` in the `langchain` package.
- Use `ruff format` in experimental package.
- Format changes in experimental package by `ruff format`.
- Manual formatting fixes to make `ruff .` pass.
fixed _load_prompt_from_file file check
Labels: name: area: doc loader, name: auto:bug

**Description:** Fixed security issue in `_load_prompt_from_file` function(s). By passing instance of `class NotStrLike(os.PathLike)`, users are able to execute arbitrary code. `file` parameter should be either `str` or `Path` instance, and nothing else.

PoC: https://github.com/tangledgroup/langchain-prompt-exploit/blob/01795ccde3bf176e148e688079af6f6c93a730c3/1_langchain_prompt_wikipedia/main.py#L67
Test-publish to test PyPI and separate jobs to limit permissions.
Labels: name: auto:improvement

Before making a new `langchain` release, we want to test that everything works as expected. This PR lets us publish `langchain` to test PyPI, then install it from there and run checks to ensure everything works normally before publishing it "for real".

It also takes the opportunity to refactor the build process, splitting up the build, release-creation, and PyPI upload steps into separate jobs that do not share their elevated permissions with each other.
Issue: Encoding error with UnstructuredFileLoader
Labels: name: area: doc loader, name: auto:bug

Hi! Usually I process English documents, but I recently had to process files in Portuguese and UnstructuredFileLoader can't deal with it. I loose characters such as ç, é, í, ã, õ, etc. It doesn't matter if its PDF, HTML or DOCX. Is there a way to encode it properly?

ex:

Art. 110. Sem prejuzo da nova redao dada aos arts. 380 e 381 da Lei n 16.050, de 31 de julho de 2014, as outras disposies previstas nesta Lei sero aplicadas apenas aos processos de licenciamento de obras e edificaes protocolados aps a data de publicao desta Lei, exceto nos casos de manifestao formal do interessado, a qualquer tempo, optando pela anlise integral de acordo com as novas disposies. Art. 111. O 4 do art. 382 da Lei n 16.050, de 2014, passa a vigorar com a seguinte redao: Art. 382. ................................................................................... .................................................................................................... 4 Para as Operaes Urbanas Consorciadas preexistentes em vigor, as regras de finalizao devero ser definidas por lei especfica e as regras de transio para finalizao devero ser iniciadas apenas para lotes que no se enquadrem no tamanho mnimo estabelecido pela OUC, quando a Operao Urbana Consorciada atingir o limite de estoque de rea Adicional disponvel de 5% do total previsto em sua lei especfica. (NR) Art. 112. A reviso da Lei Municipal de Parcelamento, Uso e Ocupao LPUOS, Lei n 16.402, de 2016, poder rever os limites das reas de ZEU, ZEUa, ZEUP, ZEUPa, ZEM e ZEMP, com base em parmetros que considerem o disposto no art. 77 da Lei n 16.050, de 2014.

### Suggestion:

_No response_
[Bing Search Wrapper] KeyError Exception 'webPages' in LangChain Bing Search Tool: Recurring Bug Impacting Search Results
Labels: name: auto:bug, name: area: models

### System Info

Langchain version: v0.0.326
Python: 3.10

### Who can help?

@hw We have been encountering a recurring issue with the Bing Search tool (BingSearchAPIWrapper) in LangChain developed in Python. The tool occasionally fails to return results, and the backend function displays the message `webPages`. This issue is connected to the BingSearchAPIWrapper 

![image](https://github.com/langchain-ai/langchain/assets/15362783/98f0517b-6c12-4591-9017-a32fc8e952c4)


### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [X] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

#### Steps to Reproduce:
1. Query the Bing Search tool with a specific text : `{"keywords":["Everyday C%23", "advanced"]} site:https:learn.microsoft.com'.`
2. Observe that the tool fails to return results and instead displays the message `webPages`.

#### Actual Behavior:
The Bing API fails to return relevant results if there are no matches. In these situations, the API omits the `webPages` property from the response.

### Expected behavior

#### Expected Behavior:
The Bing Search tool should return relevant results for any given keyword or a message that the results can not be found.
Issue: How to convert ConversationSummaryBufferMemory messages to list of tuples of strings
Labels: name: area: memory, name: auto:question

### Issue you'd like to raise.

I have a ConversationSummaryBufferMemory object which stores the chat history between human and AI. One such history looks like 

```python
[HumanMessage(content='Hi', additional_kwargs={}, example=False),
 AIMessage(content='Hi, how can I help you?', additional_kwargs={}, example=False),
 HumanMessage(content='What is TPA', additional_kwargs={}, example=False),
 AIMessage(content='TPA stands for TPA', additional_kwargs={}, example=False)]
```

I am looking a list of tuples of messages like 

```[('Hi', 'Hi, how can I help you?'), ('What is TPA','TPA stands for TPA')]```

How can I do that?

### Suggestion:

_No response_
After doing merge_from with Faiss, vector store appears as None. 
Labels: name: area: vector store, name: auto:bug, name: auto:documentation

### Issue with current documentation:

I have a faiss vector store that I created before, when I print it it appears as <langchain.vectorstores.faiss.FAISS object at 0x0000022B3DFBBB50>. Then, I create a new faiss vector store and when I print it, it appears as <langchain.vectorstores.faiss.FAISS object at 0x0000022B7D0F4810>. I want to merge these two faiss objects, so I tried the merge_from method. However, when I try merge_from, vector store appears None. Here is my code:

    vector_store = FAISS.from_texts(text_chunks, embeddings,
                                        metadatas=[{"source": f"{created_pkl_name}:{i}"} for i in
                                                   range(len(text_chunks))])
    print("vector_store_new: ",vector_store) # <langchain.vectorstores.faiss.FAISS object at 0x0000022B7D0F4810>

    if os.path.exists(faiss_path_all):

        with open(faiss_path_all, 'rb') as faiss_file_all:
            all_data = pickle.load(faiss_file_all)
            print("all_data: ", all_data) # <langchain.vectorstores.faiss.FAISS object at 0x0000022B3DFBBB50>

        all_data = all_data.merge_from(vector_store)
        print(all_data) # None
How I can solve this problem? Is there another way for merge two faiss vector stores?

### Idea or request for content:

From the example (https://python.langchain.com/docs/integrations/vectorstores/faiss?highlight=FAISS#merging) I tried the same way, but still it doesn't work for me.
Feature: Addition of HuggingFaceTextGenInferenceAuto for Automated Container and Server Management
Labels: name: auto:enhancement, name: area: models

### Update: Pull request here: #12586 

Video Demo: [https://youtu.be/wspB6QbDzBc](https://youtu.be/wspB6QbDzBc)


### Feature request

Description:

I have developed a class `HuggingFaceTextGenInferenceAuto` to automate the process of managing Docker inference server containers for the `HuggingFaceTextGenerationInference` LLM. It essentially acts as a factory to create an instance of `HuggingFaceTextGenInference` via the `from_docker` classmethod. This class can create, reuse, or recreate containers as-needed based on parameter changes.

The objective is to simplify the management of HuggingFace TextGen inference servers, and allow users to spin up Docker containers for model inference with ease. The code also contains methods to handle the health checks, wait for model downloads, and ensure container and API readiness before proceeding.

Here is a breakdown of the main functionalities:

**Automated Docker Container Management:**

* Create, reuse, or recreate Docker containers for HuggingFace TextGen inference.
* Auto-increment host port if the specified port is unavailable.
* Provide a health check and wait for model download completion.

**Parameter Management:**

* Handle Docker parameters for textgen-inference-server and HuggingFaceTextGenInference parameters with Pydantic models.
* Provide methods to convert parameters to dictionaries for easier management.
Validate parameters and create new containers based on parameter diffs

**Utility Methods:**

* Methods to get the model name, image tag, container, etc., from various inputs.
* Utility methods for converting shared memory size, checking port availability, and finding available host ports.

The code is ready and I plan to submit a pull request for review and integration. I believe this feature will be a valuable addition to the Langchain library, allowing for easier rapid prototyping of LLM-powered applications.

### Motivation

1. **Reduce Manual Intervention:**
   - Minimize the steps a user has to take to spin up a HuggingFace TextGen Inference server, making the process faster and less error-prone.
   - By automated handling of health checks, waiting for model downloads, and ensuring the readiness of the container and API, we aim to expedite prototyping of LLM powered applications. (We expect this will be especially useful for things like, e.g. auto-evaluation / parameter searches for LLM apps.)
 
2. **Ensure Consistency:**
   - Provide a structured way to handle parameters and configurations for both container and Langchain LLM wrapper

### Your contribution

The `HuggingFaceTextGenInferenceAuto` which facilitates the above goals.
while importing LlamaIndexRetriever, an error occurs that
Labels: name: area: vector store, name: auto:bug

### System Info

from langchain.retrievers.llama_index import LlamaIndexRetriever
------

```shell
llama-index==0.8.55
langchain==0.0.325
```


> in Chain.run(self, callbacks, tags, metadata, *args, **kwargs)
    503     if len(args) != 1:
    504         raise ValueError("`run` supports only one positional argument.")
--> 505     return self(args[0], callbacks=callbacks, tags=tags, metadata=metadata)[
    506         _output_key
    507     ]
    509 if kwargs and not args:
    510     return self(kwargs, callbacks=callbacks, tags=tags, metadata=metadata)[
    511         _output_key
    512     ]
in Chain.__call__(self, inputs, return_only_outputs, callbacks, tags, metadata, run_name, include_run_info)
    308 except BaseException as e:
    309     run_manager.on_chain_error(e)
--> 310     raise e
    311 run_manager.on_chain_end(outputs)
    312 final_outputs: Dict[str, Any] = self.prep_outputs(
    313     inputs, outputs, return_only_outputs
...
---> 32 response = index.query(query, response_mode="no_text", **self.query_kwargs)
     33 response = cast(Response, response)
     34 # parse source nodes
AttributeError: 'VectorStoreIndex' object has no attribute 'query'

Attribute "query" is a deprecated method. Therefore, it needs to be updated. 

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

```python
from llama_index import VectorStoreIndex
from langchain.chains import RetrievalQA
from langchain.llms import OpenAI
from langchain.retrievers.llama_index import LlamaIndexRetriever
index = VectorStoreIndex.from_documents(**hoge)
retriever = LlamaIndexRetriever(index = index)
qa = RetrievalQA.from_chain_type(llm=OpenAI(), chain_type="stuff",retriever=LlamaIndexRetriever(index=index))
query = "What did the president say about Ketanji Brown Jackson"
qa.run(query)
```

### Expected behavior

Successfully read llama index retriever
Allow astream_log to be used inside atrace_as_chain_group
Labels: name: auto:improvement

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
AttributeError: 'Engine' object has no attribute '_instantiate_plugins', while trying to connect with MS SQL
Labels: name: auto:bug, name: area: models

### System Info

latest version of langchain
python=3.11
sqlalchemy==1.3.17

### Who can help?

from sqlalchemy import create_engine
import os, sys, openai
import constants, definitions 
from langchain.sql_database import SQLDatabase
from langchain.llms.openai import OpenAI
from langchain_experimental.sql import SQLDatabaseChain
from sqlalchemy import create_engine

from langchain.chat_models import ChatOpenAI



from urllib.parse import quote_plus
driver = 'ODBC Driver 17 for SQL Server'
host = '******'
user = '*****'
database = '*****'
password = '*******' 
encoded_password = quote_plus(password)  

# db = SQLDatabase.from_uri(f"mssql+pyodbc://{user}:{encoded_password}@{host}/{database}?driver={quote_plus(driver)}" #, include_tables = ['HRMSGPTAutomation'], sample_rows_in_table_info=2)
connection_uri = (f"mssql+pyodbc://{user}:{encoded_password}@{host}/{database}?driver={quote_plus(driver)}")
engine = create_engine(connection_uri)
model_name="gpt-3.5-turbo-16k"
db = SQLDatabase(engine, view_support=True)
llm = ChatOpenAI(temperature=0, verbose=False, model=model_name)
db_chain = SQLDatabaseChain.from_llm(llm, db, verbose=True)

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

from sqlalchemy import create_engine
import os, sys, openai
import constants, definitions 
from langchain.sql_database import SQLDatabase
from langchain.llms.openai import OpenAI
from langchain_experimental.sql import SQLDatabaseChain
from sqlalchemy import create_engine

from langchain.chat_models import ChatOpenAI



from urllib.parse import quote_plus
driver = 'ODBC Driver 17 for SQL Server'
host = '******'
user = '*****'
database = '*****'
password = '*******' 
encoded_password = quote_plus(password)  

# db = SQLDatabase.from_uri(f"mssql+pyodbc://{user}:{encoded_password}@{host}/{database}?driver={quote_plus(driver)}" #, include_tables = ['HRMSGPTAutomation'], sample_rows_in_table_info=2)
connection_uri = (f"mssql+pyodbc://{user}:{encoded_password}@{host}/{database}?driver={quote_plus(driver)}")
engine = create_engine(connection_uri)
model_name="gpt-3.5-turbo-16k"
db = SQLDatabase(engine, view_support=True)
llm = ChatOpenAI(temperature=0, verbose=False, model=model_name)
db_chain = SQLDatabaseChain.from_llm(llm, db, verbose=True)

### Expected behavior

how to solve the error
Fix sources in RetrievalQAWithSourcesChain
Labels: name: auto:bug


  - **Description:** Fix sources in RetrievalQAWithSourcesChain
  - **Issue:** #10575 #11387 #9774
  - **Twitter handle:** Syed_Mujeeb_H
  
  I was facing issues with sources like it was not coming in chain response
  
  Then I noticed this [PR](https://github.com/langchain-ai/langchain/pull/11032) and tested it worked fine for me
  
  So, I like to submit the PR
[Matching Engine] Update the Matching Engine to include the distance and filters
Labels: name: lgtm, name: area: vector store, name: auto:improvement

 Hello 👋,

This Pull Request adds more capability to the [MatchingEngine](https://api.python.langchain.com/en/latest/vectorstores/langchain.vectorstores.matching_engine.MatchingEngine.html) vectorstore of GCP. It includes the `similarity_search_by_vector_with_relevance_scores` function and also [filters](https://cloud.google.com/vertex-ai/docs/vector-search/filtering) to `filter` the namespaces when retrieving the results.

 - **Description:** Add [filter](https://cloud.google.com/python/docs/reference/aiplatform/latest/google.cloud.aiplatform.MatchingEngineIndexEndpoint#google_cloud_aiplatform_MatchingEngineIndexEndpoint_find_neighbors) in  `similarity_search` and add `similarity_search_by_vector_with_relevance_scores` method
  - **Dependencies:** None
  - **Tag maintainer:** Unknown

Thank you!
Issue: fix the issue #11648 init minimax llm 
Labels: name: lgtm, name: auto:bug, name: area: models

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->



Fix Issue https://github.com/langchain-ai/langchain/issues/11648  Minimax llm failed to initialize

The idea of this fix is https://github.com/langchain-ai/langchain/issues/10917#issuecomment-1765606725

do not use  underscore in python model class