delayedkarma/langchain-issues huggingface dataset:
Update writer.py
Labels: name: auto:nit, name: size:XS

fixed typos

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
fix: InfoSQLDatabaseTool on openai functions mode sometimes calling tables instead of table_names as function arg
Labels: name: auto:bug, name: area: models, name: size:XS

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->

**Description:** 
Improve the description for the InfoSQLDatabaseTool in order to ensure that the llm guesses the input properly on openai functions mode.
Nc/28nov/revert ci changes
Labels: name: auto:refactor, name: size:L

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Mask api key for Together LLM
Labels: name: auto:improvement, name: area: models, name: size:M

  - **Description:** Add unit tests and mask api key for Together LLM
  - **Issue:** the issue https://github.com/langchain-ai/langchain/issues/12165 ,
  - **Dependencies:** N/A
  - **Tag maintainer:** ?,
  - **Twitter handle:** N/A
Add retriever for Knowledge Bases for Amazon Bedrock
Labels: name: lgtm, name: auto:enhancement, name: area: models, name: integration: aws, name: size:L

  - **Description:** Adds a retriever implementation for [Knowledge Bases for Amazon Bedrock](https://aws.amazon.com/bedrock/knowledge-bases/), a new service announced at AWS re:Invent, shortly before this PR was opened. This depends on the `bedrock-agent-runtime` service, which will be included in a future version of `boto3` and of `botocore`. We will open a follow-up PR documenting the minimum required versions of `boto3` and `botocore` after that information is available.
  - **Issue:** N/A
  - **Dependencies:** `boto3>=1.33.2, botocore>=1.33.2`
  - **Tag maintainer:** @baskaryan
  - **Twitter handles:** `@pjain7` `@dead_letter_q`

This PR includes a documentation notebook under `docs/docs/integrations/retrievers`, which I (@dlqqq) have verified independently.

EDIT: `bedrock-agent-runtime` service is now included in `boto3>=1.33.2`:  https://github.com/boto/boto3/commit/5cf793f49369607f2557fd11741c357df866e4a5
Using langchain and LLaMA2 to QA with a large SQL DB
Labels: name: auto:question, name: area: models

I was working with a sqlite DB that I created for a large dataset (~150k rows).

Code snippets:
`db = SQLDatabase.from_uri("sqlite:///MLdata.sqlite")`
`SQLITE_PROMPT_TEXT = '''You are a SQLite expert. Given an input question, first create a 
syntactically correct SQLite query to run, then look at the results of the query and return 
the answer to the input question.
Unless the user specifies in the question a specific number of examples to obtain, query for 
at most {top_k} results using the LIMIT clause as per SQLite. You can order the results to 
 return the most informative data in the database.
 Never query for all columns from a table. You must query only the columns that are needed to 
 answer the question. Wrap each column name in double quotes (") to denote them as delimited 
  identifiers.
 Pay attention to use only the column names you can see in the tables below. Be careful to not 
 query for columns that do not exist. Also, pay attention to which column is in which table.

 Use the following format:

  Question: Question here
  SQLQuery: SQL Query to run
  SQLResult: Result of the SQLQuery
  Answer: Final answer here

 Only use the following tables:
 {table_info}

 Question: {input}'''`

`SQLITE_PROMPT = PromptTemplate(input_variables=['input', 'table_info', 'top_k'], template=SQLITE_PROMPT_TEXT)
sql_chain = SQLDatabaseChain(llm=local_llm, database=db, prompt=SQLITE_PROMPT, return_direct=False, return_intermediate_steps=False, verbose=False)

res=sql_chain("How many rows is in this db?")`

Response: 'There are 142321 rows in the input_table of this db.'

Second query

`res=sql_chain("Count rows with 'Abdominal pain', VAX_TYPE='COVID19', SEX= 'F' and HOSPITAL= 'Y' is in the input_table of this db")`

Response: 'There are 115 rows in the input_table where Abdominal pain is present, VAX_TYPE is COVID19, Sex is Female, and Hospital is Yes.'

Third query I was trying to find the patient ID only instead of the count. But I am not able to get the patient ID.

`res=sql_chain("What is the VAERS_ID with 'Abdominal pain', VAX_TYPE='COVID19', SEX= 'F' and HOSPITAL= 'Y' in this db. ")`

But the output generated is same as my second query. 

Seems like the counting is working fine but nothing more. 

Can anyone help me in displaying table like output from sqlDbchain via langchain and llama2?
Nvidia Triton + TRT-LLM Connector
Labels: name: needs documentation, name: auto:enhancement, name: area: models, name: size:XL

**Description**:
This PR includes a connector for using the [Nvidia TensorRT LLM](https://developer.nvidia.com/tensorrt) which allows for accelerated LLM optimization and performance.

**Issue**: This closes #13975 

**Dependencies**:
triton-client library was added to pyproject.toml
Nvidia TensorRT LLM Connector
Labels: name: auto:enhancement, name: area: models

### Feature request

NVIDIA TensorRT is an open-source SDK for high-performance deep learning inference, includes a deep learning inference optimizer and runtime that delivers low latency and high throughput for inference applications. I propose that a connector be added to langchain allowing users to use TensorRT with minimal configuration.

### Motivation

There are several implementations of this connector floating around the internet. Lots of folks seem to want this. Rather than have everyone need to add that connector manually it seems to make sense to natively include it in Langchain.

### Your contribution

I am happy to open the PR for this and will do so shortly.
Singel object extraction
Labels: name: auto:enhancement, name: area: models, name: extraction

### Feature request

Currently, the extraction chain only supports the extraction of an Array of object.

For example

```python
from typing import Optional

from langchain.chains import create_extraction_chain_pydantic
from langchain.pydantic_v1 import BaseModel


# Pydantic data class
class Properties(BaseModel):
    person_name: str
    person_height: int
    person_hair_color: str
    dog_breed: Optional[str]
    dog_name: Optional[str]


# Extraction
chain = create_extraction_chain_pydantic(pydantic_schema=Properties, llm=llm)

# Run
inp = """Alex is 5 feet tall. Claudia is 1 feet taller Alex and jumps higher than him. Claudia is a brunette and Alex is blonde."""

chain.run(inp)
# Results in
#
#  [Properties(person_name='Alex', person_height=5, person_hair_color='blonde', dog_breed=None, dog_name=None),
#  Properties(person_name='Claudia', person_height=6, person_hair_color='brunette', dog_breed=None, dog_name=None)]
#
````

There is currently no option available to just get one `Properties` object.

It would be nice if you could define at the beginning if you are interested in one object or an array of objects

For example

```` python
chain = create_extraction_chain_pydantic(pydantic_schema=List[Properties], llm=llm) -> Array of Properties
chain = create_extraction_chain_pydantic(pydantic_schema=Properties] llm=llm) -> Just one Propertie
`````

### Motivation

It would just make life easier when you knew that you were only dealing with one object.

It might also improve the response and prevent wrong or incomplete responses. 

### Your contribution

I would create and submit a PR that contains this feature.
Issue: Passing context(previous question and answer) to the create_pandas_dataframe_agent function. 
Labels: name: area: agent, name: auto:question

### Issue you'd like to raise.

Hello Team,
We are finding a way to pass the context, previous question and answer to **create_pandas_dataframe_agent**. Can u please help me understand how i can pass the context(previous question and answer) to the **create_pandas_dataframe_agent**. it will be helpful if you have any such example implementation

Thanks,
Akash

### Suggestion:

_No response_
Add doc for the development of core and experimental sections
Labels: name: lgtm, name: auto:documentation, name: size:S

### **Description**

Hi, I just started learning the source code of `langchain` and hope to contribute code. However, according to the instructions in the [CONTRIBUTING.md](https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md) document, I could not run the test command `make test` to run normally. I found that many modules did not exist after [splitting `langchain_core`](https://github.com/langchain-ai/langchain/discussions/13823), so I updated the document. 

### **Twitter handle** 

lin_bob57617
Why does FAISS similarity search not fetch data with respect to alphanumeric keys like ORA-14300?
Labels: name: area: vector store, name: auto:bug, name: auto:question, name: integration: redis

### Issue you'd like to raise.

**Summary:**
The FAISS similarity search in LangChain is encountering difficulties when processing alpha-numeric queries that involve numeric integers. While the search performs well for queries like "What are resolutions to problems related to SAF," it exhibits suboptimal behavior when processing queries such as "Give me complete details of L2-resolution against ORA-14300," which involve alpha-numeric combinations.
Note that we have already successfully embedded and indexed the above documents that includes alpha numeric key as well 
such as "ORA-14300".

**Expected Behavior:**
The FAISS similarity search should accurately and effectively retrieve relevant information for alpha-numeric queries, providing precise results even when numeric integers are included in the query.

**Current Behavior:**
The search is not functioning correctly when processing alpha-numeric queries with numeric integers. It fails to accurately identify and retrieve relevant documents, leading to a suboptimal user experience.

![correct](https://github.com/langchain-ai/langchain/assets/134971688/362451a6-981f-4edc-b7e0-790fc50b93de)
![wrong](https://github.com/langchain-ai/langchain/assets/134971688/9f169624-4079-489e-9303-e2b3c840a2ca)

**Steps to Reproduce:**
Index CSV data containing both text and numerical values, and subsequently execute a query that includes an alphanumeric question.

**Additional Information:**
Environment: Langchain version (0.0.284)

**Impact:**
This issue affects the accuracy and reliability of the FAISS similarity search, particularly when handling alpha-numeric queries that include numeric integers. Users relying on LangChain for information retrieval may experience challenges when seeking relevant documents related to such queries.

**Priority:**
High

Are FAISS and Redis similarity searches capable of providing such high similarity search over the index? If not, please guide me on where I should turn to achieve better and more accurate results

Thank you for your attention to this matter. Feel free to request additional information if needed.


### Suggestion:

_No response_
Issue: Chroma.from_documents does not save metadata properly
Labels: name: area: vector store, name: auto:bug, name: integration: chroma

### Issue you'd like to raise.

I create my vector db using the following code:
```
db = Chroma.from_documents(
            chunked_documents,
            embeddings,
            persist_directory=db_path + '/' + db_type,
            client_settings=chroma_settings,)
```
`chunked_documents` is a list of elements of type Document.
I have added metadata which is a simple numerical id: `{'id': 1}`

```
embeddings = HuggingFaceInstructEmbeddings(
        model_name=args.embedding_model,
        model_kwargs={"device": args.device},
    )
```

```
CHROMA_SETTINGS = Settings(
    anonymized_telemetry=False,
    is_persistent=True,
)
```

What happens is that I run `db.similarity_search(query, k=3)` and for part of the answers, the metadata dict is empty. Has anyone encountered such an issue? 

Just to point out, when I create the db using the `from_texts()` method where I add raw texts and metadata separately I do not encounter the issue and when running `db.similarity_search()` the returned answer, contains the respective metadata.


### Suggestion:

_No response_
Remove `python_repl` from _BASE_TOOLS
Labels: name: area: agent, name: lgtm, name: auto:bug, name: size:XS

### **Description:**

Previously `python_repl` was a built-in tool, but now it has been moved to `langchain_experimental`.

When I use `load_tools` I get an error:

```python
In [1]: from langchain.agents import load_tools

In [2]: load_tools(["python_repl"])
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
Cell In[2], line 1
----> 1 load_tools(["python_repl"])

File ~/workspace/langchain/libs/langchain/langchain/agents/load_tools.py:530, in load_tools(tool_names, llm, callbacks, **kwargs)
    528     tool_names.extend(requests_method_tools)
    529 elif name in _BASE_TOOLS:
--> 530     tools.append(_BASE_TOOLS[name]())
    531 elif name in _LLM_TOOLS:
    532     if llm is None:

File ~/workspace/langchain/libs/langchain/langchain/agents/load_tools.py:84, in _get_python_repl()
     83 def _get_python_repl() -> BaseTool:
---> 84     raise ImportError(
     85         "This tool has been moved to langchain experiment. "
     86         "This tool has access to a python REPL. "
     87         "For best practices make sure to sandbox this tool. "
     88         "Read https://github.com/langchain-ai/langchain/blob/master/SECURITY.md "
     89         "To keep using this code as is, install langchain experimental and "
     90         "update relevant imports replacing 'langchain' with 'langchain_experimental'"
     91     )

ImportError: This tool has been moved to langchain experiment. This tool has access to a python REPL. For best practices make sure to sandbox this tool. Read https://github.com/langchain-ai/langchain/blob/master/SECURITY.md To keep using this code as is, install langchain experimental and update relevant imports replacing 'langchain' with 'langchain_experimental'
```

In this case, it will be very confusing. I think it is no longer a built-in tool now, so it can be removed from `_BASE_TOOLS`

### **Issue:** 

https://github.com/langchain-ai/langchain/issues/13858, https://github.com/langchain-ai/langchain/issues/13859, https://github.com/langchain-ai/langchain/issues/13856
### **Twitter handle:** 

[lin_bob57617](https://twitter.com/lin_bob57617)
Enable Limit False in Self Query Retriever doesn't have the expected behavior
Labels: name: area: embeddings, name: area: vector store, name: auto:bug, name: area: models

### System Info

I am using the SelfQueryRetriever class to make questions about a set of documents. It seems that there is an issue with the use of the `enable_limit` argument

Versions:
Langchain Version: 0.0.340
openai 1.3.5


### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [X] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

 enable_limit = False:
```python
 retriever = SelfQueryRetriever.from_llm(
    llm,
    vectorstore,
    document_content_description,
    metadata_field_info,
    enable_limit=False
)

query = "I want indentify thousand repairs with xxxx"

result = retriever.invoke(query)

len(result)

#Result: 4
 ```

If change enable_limit = True:
```python

retriever = SelfQueryRetriever.from_llm(
    llm,
    vectorstore,
    document_content_description,
    metadata_field_info,
    enable_limit=True
)

query = "I want indentify thousand repairs with xxxx"

result = retriever.invoke(query)

len(result)

#Result: 1000
 ```

If  enable_limit=True and change in query for "All":
```python
retriever = SelfQueryRetriever.from_llm(
    llm,
    vectorstore,
    document_content_description,
    metadata_field_info,
    enable_limit=True
)

query = "I want indentify All repairs with xxxx"

result = retriever.invoke(query)

len(result)

#Result: 4
```

### Expected behavior

The expected behaviour with  "enable_limit = False" was to show more than 1000 documents.
As there is no defined limit, all documents were expected as a result.
Fixes for opengpts release
Labels: name: lgtm, name: auto:bug, name: area: models, name: size:L

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
ImportError: cannot import name 'agenerate_from_stream' from 'langchain_core.language_models.chat_models'
Labels: name: auto:bug, name: area: models

### System Info

Current LangChain master commit: 
https://github.com/langchain-ai/langchain/commits/391f200 

### Who can help?

@hwchase17 @baskaryan

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Just run 
```
from langchain.chat_models import BedrockChat
```

Stacktrace: 
```
tests/e2e-tests/test_compatibility_matrix.py:10: in <module>
    from langchain.chat_models import ChatOpenAI, AzureChatOpenAI, ChatVertexAI, BedrockChat
/tmp/venv/lib/python3.11/site-packages/langchain/chat_models/__init__.py:20: in <module>
    from langchain.chat_models.anthropic import ChatAnthropic
/tmp/venv/lib/python3.11/site-packages/langchain/chat_models/anthropic.py:18: in <module>
    from langchain.chat_models.base import (
/tmp/venv/lib/python3.11/site-packages/langchain/chat_models/base.py:1: in <module>
    from langchain_core.language_models.chat_models import (
E   ImportError: cannot import name 'agenerate_from_stream' from 'langchain_core.language_models.chat_models' (/tmp/venv/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py)
```

### Expected behavior

To being able to import the required modules
Use langchain core in-tree as a dev dependency
Labels: name: lgtm, name: auto:improvement, name: size:L

Using the published version means master is broken for contributors whenever we make changes in one lib that depend on the other.

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->