delayedkarma/langchain-issues huggingface dataset:
GrobidParser: process_xml chunk_bboxes[0][0]["page"], IndexError: list index out of range
Labels: name: area: doc loader, name: auto:bug

Using the GrobidParser with `segment_sentences=False` results in `langchain/document_loaders/parsers/grobid.py", line 87, in process_xml
    chunk_bboxes[0][0]["page"],
IndexError: list index out of range` for some PDFs.

From my own testing with the offending PDFs I suspect the following if-block in the GrobidParser is indented incorrectly, it needs one more level of indentation. Adding this indentation solved the issue on my end.
https://github.com/langchain-ai/langchain/blob/20c6ade2fc24c458a080eaa468add7c61dc8dbc3/langchain/document_loaders/parsers/grobid.py#L80-L93

If you need a reproduction to verify, let me know.
gradient.ai LLM intregration
Labels: name: lgtm, name: auto:enhancement, name: area: models

DRAFT

- **Description:** This PR implements a new LLM API to https://gradient.ai 
- **Issue:** Feature request for LLM #10745 
- **Dependencies**: No additional dependencies are introduced. 
- **Tag maintainer:** I am opening this PR for visibility, once ready for review I'll tag.

- ```make format && make lint && make test``` is running.
- added a `integration` and `mock unit` test.


<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Query: Is It Possible to Extract Content Using All HTML Tags in ReadTheDocsLoader?
Labels: name: area: doc loader, name: auto:question

### Issue you'd like to raise.

According to the [ReadTheDocsLoader documentation](https://api.python.langchain.com/en/latest/document_loaders/langchain.document_loaders.readthedocs.ReadTheDocsLoader.html), this loader iterates through all the files located under a specified path and extracts the actual content of the files by retrieving `main` html tags.

```python
from langchain.document_loaders import ReadTheDocsLoader

loader = ReadTheDocsLoader("docs", encoding="utf-8", features="html.parser")
docs = loader.load()
len(docs)
```

Question: In my HTML files, I don't have main HTML tags. Is it possible to configure the ReadTheDocsLoader to extract content from files by capturing all HTML tags instead of just the main HTML tags? Alternatively, are there alternative loaders available that can traverse all files within a specified path and extract content by capturing all HTML tags?
LLMRails
Labels: name: lgtm, name: area: vector store, name: auto:enhancement

### LLMRails Integration
This PR provides integration with LLMRails. Implemented here are:

langchain/vectorstore/llm_rails.py
tests/integration_tests/vectorstores/test_llm_rails.py
docs/extras/integrations/vectorstores/llm-rails.ipynb
Batching for hf_pipeline
Labels: name: auto:improvement, name: area: models

The huggingface pipeline in langchain (used for locally hosted models) does not support batching. If you send in a batch of prompts, it just processes them serially using the base implementation of _generate: https://github.com/docugami/langchain/blob/master/libs/langchain/langchain/llms/base.py#L1004C2-L1004C29

This PR adds support for batching in this pipeline, so that GPUs can be fully saturated. I updated the accompanying notebook to show GPU batch inference.
Prompt was changed to different meaning
Labels: name: area: memory, name: area: doc loader, name: area: vector store, name: auto:bug, name: area: models

### System Info

I'm building a chatbot to introduce matching job openings from files to users. When the user input "I want to find a job", it is modified to "What kind of job are you looking for?" which ask the model what job he is looking for, but of course, it won't get an answer.

Human input: I want to find a job
Human input in Prompt after formatting: What kind of job are you looking for?
Chatbot: As an AI, I don't have personal preferences or the ability to look for a job.

Here is my prompt and conversation:
```
template = """ 
You are an AI assistant specifically tasked with finding matching
job opportunities in our job data based on user requests.

###
Context from data: {context}
###
{chat_history}

Human: {question}
Chatbot:"""

prompt = PromptTemplate(
input_variables=["question", "chat_history", "context"],
template=template
)
qa = ConversationalRetrievalChain.from_llm(
    llm=llm,
    retriever=db.as_retriever(),
    combine_docs_chain_kwargs={"prompt": prompt},
    memory=memory,
    verbose=True
)
```

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [X] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [X] Document Loaders
- [X] Vector Stores / Retrievers
- [X] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

template = """ 
You are an AI assistant specifically tasked with finding matching
job opportunities in our job data based on user requests.

###
Context from data: {context}
###
{chat_history}

Human: {question}
Chatbot:"""

prompt = PromptTemplate(
input_variables=["question", "chat_history", "context"],
template=template
)
qa = ConversationalRetrievalChain.from_llm(
    llm=llm,
    retriever=db.as_retriever(),
    combine_docs_chain_kwargs={"prompt": prompt},
    memory=memory,
    verbose=True
)

### Expected behavior

the prompt should be what user have input
initializing and using a Supabase vector store using SupabaseVectorStore
Labels: name: area: vector store, name: auto:bug

### System Info

`Python 3.11.4`
`langchain==0.0.271`
`lark==1.1.7`
`openai==0.27.8`

### Who can help?
Getting the following error:
```sh
 retriever = SelfQueryRetriever.from_llm(
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/langchain/retrievers/self_query/base.py", line 135, in from_llm
    structured_query_translator = _get_builtin_translator(vectorstore)
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/langchain/retrievers/self_query/base.py", line 45, in _get_builtin_translator
    raise ValueError(
ValueError: Self query retriever with Vector Store type <class 'langchain.vectorstores.supabase.SupabaseVectorStore'> not supported.
```
_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

-  initialize a SupabaseVectorStore. I want to use existing index, embeddings already created and stored in table `documents`

table definition:
```sql
create table
  public.documents (
    id bigserial,
    content text null,
    metadata jsonb null,
    embedding public.vector null,
    constraint documents_pkey primary key (id)
  ) tablespace pg_default;

```

```python
document_content_description = "Brief summary of a movie"

embeddings = OpenAIEmbeddings()
vectorstore = SupabaseVectorStore(
    client=supabase,
    embedding=embeddings,
    table_name="documents",
    query_name="match_documents",
)

llm = OpenAI(temperature=0)
retriever = SelfQueryRetriever.from_llm(
    llm, vectorstore, document_content_description, metadata_field_info, verbose=True
)

retriever.get_relevant_documents("What are some movies about dinosaurs")
```

### Expected behavior

return relevant documents
Python: Add iterator support to chat history base class
Labels: name: area: memory, name: auto:improvement

  - **Description:** Add Python iterator support for chat messages in `BaseChatMessageHistory`.  This makes any chat history class iterable on its message list.  Here is my use-case that triggered this change.  
  
  I have a `ConversationBufferMemory` that is backed by a `FileChatMessageHistory`:
```python
vectorstore = get_vectorstore()
chat_history = FileChatMessageHistory("chat_memory_path/session.json")
memory = ConversationBufferMemory(memory_key="chat_history", chat_memory=chat_history, return_messages=True)
qa = ConversationalRetrievalChain.from_llm(
    get_chat_model(),
    vectorstore.as_retriever(),
    return_source_documents=True,
    memory=memory,
)
```
where `get_chat_model()` returns a LLM and `vectorstore` is your favorite Vector store.  

A problem occurs when `BaseConversationalRetrievalChain._get_chat_history()` is called.  In my setup, `chat_history` is an instance of `FileChatMessageHistory` and `_get_chat_history()` tries to create an iterator from it (see this [line](https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/chains/conversational_retrieval/base.py#L37)).  I could supply my own callable version when building the chain, but it kind of made sense to me that a chat history would be iterable.

I'm looking for input on how to make this pass in `mypy` since it complains with the following:
```
langchain/memory/chat_message_histories/in_memory.py:10: error: Definition of "__iter__" in base class "BaseChatMessageHistory" is incompatible with definition in base class "BaseModel"  [misc]
```
Vector store-backed retriever -> allow using search_kwargs in get_relevant_documents function (PGVector)
Labels: name: area: vector store, name: auto:improvement

### Feature request

This is the current implementation. You get your Vectorstore and turn it into a retriever. You pass search kwargs into the as_retriever method.

```
retriever = db.as_retriever(search_type="similarity_score_threshold", search_kwargs={"score_threshold": .5})
docs = retriever.get_relevant_documents("what did he say about ketanji brown jackson")
```

Is this as intended? I think it´s not the best way. We use PGVector and use it in a API where we pass metadata to do filtered search in the vectorstore. 

How I think it should look like:

```
docs = retriever.get_relevant_documents("what did he say about ketanji brown jackson", search_kwargs={"score_threshold": .5})
```
So instead of using it in the constructor, I would rather prefer it in `get_relevant_documents` method. 



### Motivation

I don´t want to create a new retriever instance every request and rather have `get_relevant_documents` more dynamic.

### Your contribution

As far as I can see this should be doable by passing some kwargs around. I could probably implement this if I am allowed to.
Add option to use SharePoint folder ID instead of folder path
Labels: name: area: doc loader, name: auto:improvement

* Description: Provides user with the option to use folder ID to get the SharePoint folder. Updated notebook to show use of this optional feature. Also fixed a typo in the url to get Subsite ID in the notebook (url was missing 'sites/' before ),
* Issue: GraphAPI will often return "No Resource Found" error when using the folder path. This is as far as I can tell a bug on the GraphAPI and not user related. This can be circumvented by using the folder ID and get_item() method instead of the folder path and get_item_by_path() method,
* Dependencies: None,
* Tag maintainer: [@baskaryan](https://github.com/baskaryan),
* Twitter handle: @WRhetoric
Make tools async by default unless user overrides `_arun`
Labels: name: auto:improvement, name: area: models

### Feature request

Most of the tools are currently sync tools, so if an async agent executor calls them tools raise `NotImplementedError`

My suggestion is to change `_arun` method of the `langchain.tools.base.BaseTool` class with a bolierplate code asyncer. An [example is given in the gh issues](https://github.com/langchain-ai/langchain/issues/5011#issuecomment-1725356727). 

Asyncer is a small lib developped by @tiangolo (FastApi creator) that can convert a sync code to async (runs in worker threads) 

This way by default we would have an async version of any tool. Tool implementors can always override `_arun` method in their implementation providing more native async support. But otherwise we would have a default async implementation. 

### Motivation

In production systems we need to scale the requests to our langchain app, thus this means servin cconcurrently to many users. In python for IO Bound operations this is best done with async libs and frameworks. However all parts of the code has to be async, including LLMs + tools ..etc 

Since many tools lack support for `_arun` method, this enhancement will provide default async support for all tools. 

### Your contribution

Yes, I'll create a PR for this.
Using StructuredTool with agent
Labels: name: area: agent, name: auto:bug

### System Info

Using this Code I am getting this error:
TypeError: Chain.__call__() got an unexpected keyword argument 'question'

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [X] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

  from pydantic import BaseModel, Field
  
  from langchain.chat_models import ChatOpenAI
  from langchain.agents import Tool
  from langchain.embeddings.openai import OpenAIEmbeddings
  from langchain.text_splitter import CharacterTextSplitter
  from langchain.vectorstores import FAISS
  from langchain.document_loaders import PyPDFLoader
  from langchain.chains import RetrievalQA
  from langchain.agents import initialize_agent
  from langchain.agents import AgentType
  from langchain.tools import StructuredTool
  from dotenv import load_dotenv
  class DocumentInput(BaseModel):
      question: str = Field()
  
  load_dotenv()
  
  llm = ChatOpenAI(temperature=0, model="gpt-3.5-turbo-0613")
  
  tools = []
  files = [
      # https://abc.xyz/investor/static/pdf/2023Q1_alphabet_earnings_release.pdf
      {
          "name": "alphabet-earnings",
          "path": "/Users/harrisonchase/Downloads/2023Q1_alphabet_earnings_release.pdf",
      },
      # https://digitalassets.tesla.com/tesla-contents/image/upload/IR/TSLA-Q1-2023-Update
      {
          "name": "tesla-earnings",
          "path": "/Users/harrisonchase/Downloads/TSLA-Q1-2023-Update.pdf",
      },
  ]
  
  for file in files:
      loader = PyPDFLoader(file["path"])
      pages = loader.load_and_split()
      text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)
      docs = text_splitter.split_documents(pages)
      embeddings = OpenAIEmbeddings()
      retriever = FAISS.from_documents(docs, embeddings).as_retriever()
  
      # Wrap retrievers in a Tool
      tools.append(
          StructuredTool.from_function(
              args_schema=DocumentInput,
              name=file["name"],
              description=f"useful when you want to answer questions about {file['name']}",
              func=RetrievalQA.from_chain_type(llm=llm, retriever=retriever),
          )
      )
  
  llm = ChatOpenAI(
      temperature=0,
      model="gpt-3.5-turbo-0613",
  )
  
  agent = initialize_agent(
      agent=AgentType.STRUCTURED_CHAT_ZERO_SHOT_REACT_DESCRIPTION,
      tools=tools,
      llm=llm,
      verbose=True,
  )
  
  agent({"input": "did alphabet or tesla have more revenue?"})

### Expected behavior

Compare the two documents!
Provide option to use folder_id instead of folder_path
Labels: name: area: doc loader, name: auto:improvement

  - **Description:** Provides user with the option to use folder ID to get the SharePoint folder. Updated notebook to show use of this optional feature. Also fixed a typo in the url to get Subsite ID in the notebook (url was missing 'sites/' before <site-name>), 
  - **Issue:** GraphAPI will often return "No Resource Found" error when using the folder path. This is as far as I can tell a bug on the GraphAPI and not user related. This can be circumvented by using the folder ID and get_item() method instead of the folder path and get_item_by_path() method,
  - **Dependencies:** None,
  - **Tag maintainer:** @baskaryan,
  - **Twitter handle:** @WRhetoric
chore: allow using dataclasses_json dependency v0.6.0
Labels: name: lgtm, name: auto:nit

**Description:** upgrade the `dataclasses_json` dependency to its latest version ([no real breaking change](https://github.com/lidatong/dataclasses-json/releases/tag/v0.6.0) if used correctly), while allowing previous version to not break other users' setup
**Issue:** I need to use the latest version of that dependency in my project, but `langchain` prevents it.

Note: it looks like running `poetry lock --no-update` did some changes to the lockfiles as it was the first time it was with the `macosx_11_0_arm64` architecture 🤷
make auto-setting the encodings optional, alow explicitly setting it
Labels: name: lgtm, name: area: doc loader, name: auto:improvement

I was trying to use web loaders on some spanish documentation (e.g. [this site](https://www.fromdoppler.com/es/mailing-tendencias/), but the auto-encoding introduced in https://github.com/langchain-ai/langchain/pull/3602 was detected as "MacRoman" instead of the (correct)  "UTF-8".

To address this, I've added the ability to disable the auto-encoding, as well as the ability to explicitly tell the loader what encoding to use.

  - **Description:** Makes auto-setting the encoding optional in `WebBaseLoader`, and introduces an `encoding` option to explicitly set it.
  - **Dependencies:** N/A
  - **Tag maintainer:** @hwchase17 
  - **Twitter handle:** @czue
In xcode, use swift to call Python code and introduce Ollama error
Labels: name: auto:bug, name: area: models

### System Info

I am developing a macOS application that uses Swift to call Python. To achieve this, I have integrated PythonKit, which also works properly. When I followed the integration of Ollama guided by Langchain, I encountered a problem and reported an error in 'from langchain.llms import Ollama'. The subsequent code cannot execute properly, but I ran it on the terminal without any issues. By setting environment variables, I managed to run both the Python environment and the terminal as 3.11.5, And I can successfully find olama.py under langchain/llms under site packages. I have tried installing TextLoader, which is directly installed under site packages and can import TextLoader normally. I guess it may not be possible to find deeper level packages. Can anyone encounter similar problems and help me solve them or provide some ideas? Thank you

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

from langchain.llms import Ollama
print('hello')

### Expected behavior

print('hello')
add clustered vearch in langchain
Labels: name: lgtm, name: area: vector store, name: auto:enhancement

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Remembrall Integration
Labels: name: area: doc loader, name: auto:documentation

  - **Description:** Added integration instructions for Remembrall. 
  - **Tag maintainer:** @hwchase17 
  - **Twitter handle:** @raunakdoesdev

Fun fact, this project originated at the Modal Hackathon in NYC where it won the Best LLM App prize sponsored by Langchain. Thanks for your support 🦜
Issue with BaseChatMessageHistory Attribute Scope
Labels: name: area: memory, name: auto:bug

### System Info

LangChain v0.0.294

### Who can help?

@hwchase17 

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [X] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

I've recently encountered an issue that took a bit of time to isolate and identify. I have a service that utilizes a chain for conversations, where multiple instances are created of the same chain to manage responses to all users. However, I observed an issue where messages started to get mixed up.

Initially, I suspected the issue might be related to the websocket, but upon further investigation, I realized that the root cause seems to be the way `BaseChatMessageHistory` is declared. It appears that the messages list is implemented as a class attribute rather than an instance attribute. This seems to be causing memory to be shared across all instances, which doesn't align with the expected behavior and is causing the message mix-up that I observed.

I wanted to reach out to understand if there is a specific reason for implementing the messages list as a class attribute? In my perspective, modifying it to be an instance attribute might prevent such issues from occurring, and I believe it could potentially be considered a bug in its current state. I'm open to collaborating on a fix and would be happy to submit a PR to address this.

### Expected behavior

Just make it an instance attribute.