delayedkarma/langchain-issues huggingface dataset:
Fixes to SelfQueryRetriever and LLMRouterChain deprecation warning
Labels: name: auto:bug, name: area: models

  - **Description:** Changed the `_get_relevant_documents` function inside `SelfQueryRetriever` and `_call` function inside `LLMRouterChain` to avoid using the deprecated `predict_and_parse` function inside `LLMChain`, and not getting the (annoying) warning message.
  - **Issue:** #10606 and #10462 for `SelfQueryRetriever`, and #6819 for `LLMRouterChain` 
  - **Dependencies:** None,
  - **Tag maintainer:** @baskaryan,
  - **Twitter handle:** @schnell_dj

**PS :** This is my first PR ever, and I am far from being a pro Dev. So your feedback, comments and guidance are more than welcome!
ArcGISLoader: Limit number of results in query
Labels: name: area: doc loader, name: auto:bug

Description: this PR changes the `ArcGISLoader` to set `return_all_records` to `False` when `result_record_count` is provided as a keyword argument. Previously, `return_all_records` was `True` by default and this made the API ignore `result_record_count`.

Issue: `ArcGISLoader` would ignore `result_record_count` unless user also passed `return_all_records=False`.
Let Notion document loader support utf-8 and make it default.
Labels: name: lgtm, name: area: doc loader, name: auto:improvement

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
- **Description**: Let Notion document loader support utf-8 and make it default. Otherwise, it gets an error when reading non-English documents.
- **Issue:** Not applicable.
- **Dependencies:** None.
- **Tag maintainer:** 
- **Twitter handle:** richarddwang
Thought Instruction (Alternative to CoT)
Labels: name: auto:enhancement, name: area: models

### Feature request

It would be great to see **thought instruction** be implemented as an alternative to chain of thought (CoT) prompting.

**Thought instruction** is proposed as an alternative to chain of thought (CoT) prompting for a more nuanced approach to software development. It involves explicitly addressing specific problem-solving thoughts in instructions, akin to solving subtasks in a sequential manner. The method includes role swapping to inquire about unimplemented methods or explain feedback messages caused by bugs. This process fosters a clearer understanding of the existing code and identifies specific gaps that need addressing. By doing so, **thought instruction** aims to mitigate code hallucinations and enable a more accurate, context-aware approach to code completion, resulting in more reliable and comprehensive code outputs.

### Motivation

See ChatDev ([source code](https://github.com/OpenBMB/ChatDev/tree/main) and [paper](https://arxiv.org/pdf/2307.07924)) for inspiration.

### Your contribution

Idea
mv
Labels: name: auto:documentation

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Issue: I cannot seem to find how to make a System role message in my chain.
Labels: name: auto:question, name: area: models, name: integration: pinecone

### Issue you'd like to raise.

Hi ,

I'm trying to see how I can put a system message for my chain to tell him that for example "Your name is XX".

I've tried a lot of things and saw a lot of issues resolved and documentations but they never worked... Any help will be appreciated. here is the code of my chain.ts:
`
import {OpenAI} from "langchain/llms/openai";
import {pinecone} from "@/utils/pinecone-client";
import {PineconeStore} from "langchain/vectorstores/pinecone";
import {OpenAIEmbeddings} from "langchain/embeddings/openai";
import {ConversationalRetrievalQAChain} from "langchain/chains";
import { PromptTemplate } from "langchain/prompts";
import { ChatOpenAI } from "langchain/chat_models/openai";


async function initChain() {
    const model = new ChatOpenAI({
        modelName: "gpt-3.5-turbo",
        temperature: 0,
      });    
    const pineconeIndex = pinecone.Index('canada');
    const vectorStore = await PineconeStore.fromExistingIndex(
        new OpenAIEmbeddings({}),
        {
            pineconeIndex: pineconeIndex,
            textKey: 'text',
        },
    );
    return ConversationalRetrievalQAChain.fromLLM(
        model,
        vectorStore.asRetriever(),
        {returnSourceDocuments: true}
    );
    return ConversationalRetrievalQAChain.fromLLM(
        model,
        vectorStore.asRetriever(),
        {returnSourceDocuments: true},
    );
}

export const chain = await initChain()`

### Suggestion:

_No response_
Issue: Help fixing "predict_and_parse" deprecation from SelfQueryRetriever
Labels: name: area: vector store, name: auto:question, name: auto:improvement, name: area: models

### Issue you'd like to raise.

As noted here: #10462 and #6819

I've realized I'm thousands of miles away from having the skills to fix this and make a PR (I'm not a Pro Dev) in my attempt to update the `SelfQueryRetriever`. However, I think this will be a great learning opportunity, with help from someone who knows what they're doing (@agola11).

After taking a close look at the `SelfQueryRetriever` [source](https://api.python.langchain.com/en/latest/_modules/langchain/retrievers/self_query/base.html#SelfQueryRetriever), I noticed that what needs to be updated is this part from the `_get_relevant_documents` function:
```
        structured_query = cast(
            StructuredQuery,
            self.llm_chain.predict_and_parse(
                callbacks=run_manager.get_child(), **inputs
            ),
        )
```
I even ran `SelfQueryRetriever` (in my ignorance) with just `self.llm_chain.predict` to see what it did, but I got the JSON as the output and the vectorstore complaining it was expecting a tuple:
```
in RedisTranslator.visit_structured_query(self, structured_query)
     91 def visit_structured_query(
     92     self, structured_query: StructuredQuery
     93 ) -> Tuple[str, dict]:
---> 94     if structured_query.filter is None:
     95         kwargs = {}
     96     else:

AttributeError: 'str' object has no attribute 'filter'
```

I also took a look at the `predict_and_parse` method in the `LLMChain` [source](https://api.python.langchain.com/en/latest/_modules/langchain/chains/llm.html#LLMChain.predict_and_parse). And here's where I knew I was biting way more than I could (ever) chew.

### Suggestion:

Can someone please guide me to replace and update the  `_get_relevant_documents` function?

I think I need to find a way to convert the JSON to the required tuple, but I can't figure out how. Am I on the right track?
Adding Meroka Transformer
Labels: name: auto:enhancement, name: area: models

Meroka Managed Security: Enjoy all the benefits that AI and LLM based platforms have to offer, while maintaining a high standard for your privacy and compliance needs

Meroka provides software vendors that use LangChain, the ability for their users to have fine grain control over how their sensitive data (PII/PHI/etc) are being handled by the vendor. They will also gain visibility and insight, around how to improve their data hygiene 

Twitter: @RayzerCA
Improve tools
Labels: name: auto:enhancement

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Bagatur/replicate kwargs
Labels: name: lgtm, name: auto:improvement, name: area: models

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Refactor Fireworks and add ChatFireworks (#3)
Labels: name: auto:refactor, name: area: models

Description 
* Refactor Fireworks within Langchain LLMs.
* Remove FireworksChat within Langchain LLMs.
* Add ChatFireworks (which uses chat completion api) to Langchain chat models.
* Users have to install `fireworks-ai` and register an api key to use the api.

Issue - Not applicable
Dependencies - None
Tag maintainer - @rlancemartin @baskaryan
vertex default model_name changed to the stable version
Labels: name: auto:nit, name: area: models



<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:**  the default model_name for VertexAI llm is "text-bison", this model points to text-bison@latest, which is NOT the stable version of text-bison., 
  - **Issue:** llm results when no model_name are not stable, as it is pointing to the latest/experimental model and not to the stable version ,
  - **Dependencies:** no,
  - **Tag maintainer:** 
  - **Twitter handle:**

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->
Fix intermediate steps example in docs : replaced json.dumps with Langchain's dumps()
Labels: name: lgtm, name: area: doc loader, name: auto:documentation

The intermediate steps example in docs has an example on how to retrieve and display the intermediate steps.
But the intermediate steps object is of type AgentAction which cannot be passed to json.dumps (it raises an error).
I replaced it with Langchain's dumps function (from langchain.load.dump import dumps) which is the preferred way to do so.
ArcGIS Summarizer
Labels: name: area: doc loader, name: auto:enhancement

This PR enhances the utility of the `ArcGISLoader` by introducing a document transformer and chains for converting query results into natural language summaries. The following features have been added:

* `langchain.chains.arcgis.row.base.ArcGISRowSummaryChain`
* `langchain.document_transformers.ArcGISRowSummaryTransformer`
* `langchain.chains.arcgis.layer.base.ArcGISLayerSummaryInnerChain`
* `langchain.chains.arcgis.layer.base.ArcGISLayerStuffSummaryChain`
* Notebooks demonstrating the functionalities of these new classes
Added template_format to from_template_file
Labels: name: auto:improvement, name: area: models

Issue: #10519

There is no clear reason for `PromptTemplate.from_template` and `PromptTemplate.from_file` to be implemented so differently. I added the ability to `PromptTemplate.from_file` to infer `input_variables` from the template when they are not given by using the new `template_format` parameter. Also made sure to maintain backward compatibility. Added a few tests for the new functionality.
Llama - n_ctx defaults to 512 even if overide passed during invocation
Labels: name: auto:bug, name: area: models

### System Info

Langain 0.288
Windows 11
Python 3.11

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

called using the below code, where model_n_ctx is set to 1024

llm = LlamaCpp(model_path=model_path, max_tokens=model_n_ctx, n_batch=model_n_batch, callbacks=callbacks, verbose=model_verbose, echo=True)

when executing inference getting error inputs token exceed 512

### Expected behavior

called using the below code, where model_n_ctx is set to 1024

llm = LlamaCpp(model_path=model_path, max_tokens=model_n_ctx, n_batch=model_n_batch, callbacks=callbacks, verbose=model_verbose, echo=True)

when executing inference getting error inputs token exceed 512
Add Google Cloud Document AI integration
Labels: name: area: doc loader, name: auto:enhancement, name: area: models

### Feature request

Add integration for [Document AI](https://cloud.google.com/document-ai/docs/overview) from Google Cloud for intelligent document processing.


### Motivation

This product offers Optical Character Recognition, specialized processors from specific document types, and built in Generative AI processing for Document Summarization and entity extraction.

### Your contribution

I can implement this myself, I mostly want to understand where and how this could fit into the library.
Should it be a document transformer? An LLM? An output parser? A Retriever? Document AI does all of these in some capacity.

Document AI is designed as a platform that non-ML engineers can use to extract information from documents, and I could see several features being useful to Langchain (Like Document OCR to extract text and fields before sending it to an LLM) or using the Document AI Processors with Generative AI directly for the summarization/q&a output.
bump 289
Labels: name: auto:nit

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
PythonAstREPLTool won't execute code with functions/lambdas
Labels: name: area: agent, name: auto:bug

### System Info

langchain==0.0.288
python==3.10

This bug is reproducible on older langchain version (0.0.240) and different os (Windows, Debian).

### Who can help?

@hwchase17

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [X] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

```python
from langchain.tools.python.tool import PythonAstREPLTool

query = """
import pandas as pd
import random
import string

def generate_random_text():
    return ''.join(random.choices(string.ascii_letters + string.digits, k=128))

df = pd.DataFrame({
    'Column1': [generate_random_text() for _ in range(1000)],
    'Column2': [generate_random_text() for _ in range(1000)],
    'Column3': [generate_random_text() for _ in range(1000)]
})
df
"""
ast_repl = PythonAstREPLTool()
ast_repl(query)
>>> "NameError: name 'generate_random_text' is not defined"
```


### Expected behavior

I expect it to return a df.
Document' object has no attribute '_lc_kwargs
Labels: name: area: vector store, name: auto:bug, name: stale, name: area: models

### System Info

langchain version 0.0.285
langsmith version 0.0.28
Python version 3.11.2

### Who can help?

@hwchase17 

### Information

- [X ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

I followed tutorial in RAG cookbook with "With Memory and returning source documents"

Steps for changing behaviour 

1. use FAISS.from documents and save to files
    embeddings = OpenAIEmbeddings()    
    vectorstore  = FAISS.from_documents(documents, embeddings)
    vectorstore.save_local("data/faiss_index")

2. load from file
    embeddings = OpenAIEmbeddings()
    vectorstore = FAISS.load_local("data/faiss_index", embeddings)
    retriever = vectorstore.as_retriever()
 
3. error raised "Document' object has no attribute '_lc_kwargs" in step 5 
    final_inputs = {
         "context": lambda x: _combine_documents(x["docs"]),
         "question": itemgetter("question")
    }

here is the screen shot when using langsmith

<img width="543" alt="RAG" src="https://github.com/langchain-ai/langchain/assets/105797032/038b1b98-4dde-47a1-a79b-4061014c05a2">


### Expected behavior

Expected behaviour is LLM give result without error
Fix typing of RecursiveUrlLoader's exclude_dirs
Labels: name: lgtm, name: area: doc loader, name: auto:bug

  - **Description:** This fixes an issue in `RecursiveUrlLoader` where a typing appears to be incorrect. The `exclude_dirs` parameter is iterated over, yet is defined as `Optional[str]` - this PR changes that to `Optional[Iterable[str]]`.
  - **Issue:** None - this is a find-and-fix.
  - **Dependencies:** None
  - **Tag maintainer:** Maybe @rlancemartin?
  - **Twitter handle:** @benc_uk
fix 'Thought:' prefix for structured chat agent
Labels: name: area: agent, name: auto:bug

The current implementation of the `StructuredChatAgent` includes a system prompt that is less than ideal for the following reasons:

- The first "Thought:" prefix is added to the system prompt before the human message.
- Subsequent "Thought:" prefixes are added everytime the `agent_scratchpad` is created.

As a result, the input format presented to the LLMs in the first round appears as follows:

```
You are an agent ...
...
Begin!
Thought:

Human: {input}

```

In the subsequent rounds, the input format is as follows:

```
You are an agent ...
...
Begin!
Thought:

Human: {input}

This was your previous work (but I haven\'t seen any of it! I only see what you return as final answer):
Action: ...
Observation: ...
Thought:
```

The issue here is that the first "Thought:" prefix precedes the human input, which can be misleading to the LLM, causing it to sometimes skip generating any thoughts in the first round.

This misplaced "Thought:" also disrupts the intended format instruction, which is designed to guide the LLM to follow the routing sequence of Question -> Thought -> Action -> Observation -> ... -> Final Answer.

This PR fixes this issue by removing the "Thought:" prefix in the system prompt, and add it back in the `agent_scratchpad`
agent got "No I need to use a tool? No" response from llmm,which CANNOT be parsed
Labels: name: auto:bug, name: area: models

### System Info

    def parse(self, text: str) -> Union[AgentAction, AgentFinish]:
        if f"{self.ai_prefix}:" in text:
            return AgentFinish(
                {"output": text.split(f"{self.ai_prefix}:")[-1].strip()}, text
            )
        regex = r"Action: (.*?)[\n]*Action Input: (.*)"
        match = re.search(regex, text)
        if not match:
            raise OutputParserException(f"Could not parse LLM output: `{text}`")
        action = match.group(1)
        action_input = match.group(2)
        return AgentAction(action.strip(), action_input.strip(" ").strip('"'), text)

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

1. init an agent
2. ask the agent a simple question which it can solve without using any tools

### Expected behavior

DONT RAISE ERROR