delayedkarma/langchain-issues huggingface dataset:
OnlinePDFLoader crashes with import error
Labels: name: area: doc loader, name: auto:bug

### System Info

langchain = "^0.0.304"
Python 3.11.5
MacBook Pro, Apple M2 chip, 8GB memory


### Who can help?

@eyurtsev

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [X] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

```python
from langchain.document_loaders import OnlinePDFLoader

OnlinePDFLoader("https://www.africau.edu/images/default/sample.pdf").load()
```

I saw the following runtime import error:

```
ImportError: cannot import name 'PDFResourceManager' from 'pdfminer.converter' (/Users/{user}/Library/Caches/pypoetry/virtualenvs/replai-19tDtF2f-py3.11/lib/python3.11/site-packages/pdfminer/converter.py)
```

### Expected behavior

I expected `OnlinePDFLoader` to load the file.
Fix a Typo in Documentation
Labels: name: auto:documentation

- **Description:** This commit corrects a minor typo in the documentation. It changes "frum" to "from" in the sentence: "The results from search are passed back to the LLM for synthesis into an answer" in the file `docs/extras/use_cases/more/agents/agents.ipynb`. This typo fix enhances the clarity and accuracy of the documentation.
- **Tag maintainer:** @baskaryan
adding support to modify documents in azure search vectorstore
Labels: name: area: vector store, name: auto:improvement

**Description** 
Adding support to replace documents in Azure Cognitive Search. Right now, we can only upload documents but azure [azure-search-documents](https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/search/azure-search-documents/azure/search/documents/_search_client.py#L623) provides the option to modify the documents. 


**References**
- https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/search/azure-search-documents/azure/search/documents/_search_client.py#L623
- https://learn.microsoft.com/en-us/rest/api/searchservice/addupdate-or-delete-documents


**Issue** 
None

**Dependencies** 
None

**Tag maintainer** 
@eyurtsev 

**Example**
[azure_search_modify_documents.ipynb](https://github.com/bhanu-pappala/langchain/blob/azure_modify_documents/docs/extras/modules/vectorstore/azure_search_modify_documents.ipynb)
Skip `OpenAIWhisperParser` extremely small audio chunks to avoid api error
Labels: name: area: doc loader, name: auto:bug

**Description**
This PR addresses a rare issue in `OpenAIWhisperParser` that causes it to crash when processing an audio file with a duration very close to the class's chunk size threshold of 20 minutes.

**Issue**
#11449

**Dependencies**
None

**Tag maintainer**
@agola11 @eyurtsev 

**Twitter handle**
leonardodiegues
`OpenAIWhisperParser` raises error if audio has duration too close to the chunk limit
Labels: name: area: doc loader, name: auto:bug

### System Info

LangChain: 0.0.250
Python: 3.11.4
OS: Pop 22.04

### Who can help?

@agola11
@eyurtsev 

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [X] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

**Input**

```py
from pathlib import Path

from langchain.document_loaders.blob_loaders.schema import Blob
from langchain.document_loaders.parsers.audio import OpenAIWhisperParser


OPENAI_API_KEY = "<MY-KEY>"

audio_file = Path("~/Downloads/audio.mp3").expanduser()
audio_total_seconds = AudioSegment.from_file(audio_file).duration_seconds

print("Audio length (seconds):", audio_total_seconds)

blob = Blob.from_path(audio_file, mime_type="audio/mp3")
parser = OpenAIWhisperParser(api_key=OPENAI_API_KEY)
parser.parse(blob)
```


**Output**

```
Audio length (seconds): 1200.064
Transcribing part 1!
Transcribing part 2!
Attempt 1 failed. Exception: Audio file is too short. Minimum audio length is 0.1 seconds.
Attempt 2 failed. Exception: Invalid file format. Supported formats: ['flac', 'm4a', 'mp3', 'mp4', 'mpeg', 'mpga', 'oga', 'ogg', 'wav', 'webm']
Attempt 3 failed. Exception: Invalid file format. Supported formats: ['flac', 'm4a', 'mp3', 'mp4', 'mpeg', 'mpga', 'oga', 'ogg', 'wav', 'webm']
Failed to transcribe after 3 attempts.

```

### Expected behavior

Due to its internal rule of processing audio in [20-minute chunks](https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/document_loaders/parsers/audio.py#L47), `OpenAIWhisperParser` is prone to crashing when transcribing audio with durations that are dangerously close to this limit. Given that OpenAI already has a very low audio length threshold of 0.1 seconds, a simple bypass could effectively resolve this issue.

```python
# https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/document_loaders/parsers/audio.py#L47

chunk_duration = 20
chunk_duration_ms = chunk_duration * 60 * 1000
chunk_length_threshold = 0.1

# Split the audio into chunk_duration_ms chunks
for split_number, i in enumerate(range(0, len(audio), chunk_duration_ms)):
    # Audio chunk
    chunk = audio[i : i + chunk_duration_ms]
    if chunk.duration_seconds < chunk_length_threshold:
        continue
```
Error while importing LLMChain 
Labels: name: auto:bug, name: area: models

### Issue you'd like to raise.

When I'm running the code "from langchain.chains import LLMChain", I'm getting the following error: 
'RuntimeError: no validator found for <class 're.Pattern'>, see `arbitrary_types_allowed` in Config'

<img width="887" alt="Screenshot 2023-10-06 at 12 01 49 AM" src="https://github.com/langchain-ai/langchain/assets/85587494/0b171391-bbc6-41f8-b237-17bc99b11b95">
<img width="871" alt="Screenshot 2023-10-06 at 12 01 58 AM" src="https://github.com/langchain-ai/langchain/assets/85587494/592c2688-be0f-4cf4-8e7d-a4a56d7d056c">


### Suggestion:

_No response_
Issue: Trying to be able to parse all parse of LLM outputs
Labels: name: auto:bug, name: auto:question, name: area: models

### Issue you'd like to raise.

I'm trying to make a general chat bot when ever I try to get it to write code I get [ERROR] OutputParserException: Could not parse LLM output:

is there a way to parse all parse of output?

Any help would be great.

### Suggestion:

_No response_
Remove str() from RunnableConfigurableAlternatives
Labels: name: auto:nit

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Update langchain_release.yml
Labels: name: auto:nit

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
fixed return documents
Labels: name: area: memory, name: auto:bug

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Chroma is adding an extra space (very consistently) in front of '≤ ’ characters and it has a real impact.
Labels: name: area: embeddings, name: auto:bug, name: integration: chroma

### System Info

**Specs:**
langchain 0.0.301
Python 3.11.4
embeddings =HuggingFace embeddings
llm = Claud 2

EXAMPLE:

1. Chunks object below in my code contains the following string: leflunomide **(LEF) (≤ 20 mg/day)**

`Chroma.from_documents(documents=chunks, embedding=embeddings, collection_name=collection_name, persist_directory=persist_db)  `

2. after saving and retrieving from my local file with:

'db = Chroma(persist_directory=persist_db, embedding_function=embeddings, collection_name=collection_name)'

. . . then extracting with . . .

'db.get(include=['documents'])'

3. That string is now: leflunomide **(LEF) ( ≤20 mg/day)**, with a single space newly inserted before the ≤

This matters because it messes up retrieval augmentation with queries like “what doses of leflunomide are appropriate?” using Claude-2 as the llm


### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

1. Chunks object below in my code contains the following string: leflunomide **(LEF) (≤ 20 mg/day)**

`Chroma.from_documents(documents=chunks, embedding=embeddings, collection_name=collection_name, persist_directory=persist_db)  `

2. after saving and retrieving from my local file with:

'db = Chroma(persist_directory=persist_db, embedding_function=embeddings, collection_name=collection_name)'

. . . then extracting with . . .

'db.get(include=['documents'])'

3. That string is now: leflunomide **(LEF) ( ≤20 mg/day)**, with a single space newly inserted before the ≤

This matters because it messes up retrieval augmentation with queries like “what doses of leflunomide are appropriate?” using Claude-2 as the llm

### Expected behavior

See description above
Bearer token support for self-deployed OpenLLM
Labels: name: auto:improvement, name: area: models

Hello there,

I have deployed an OpenLLM on a managed service that is protected by the use of an auth Bearer token:

```bash
curl -X 'POST' \
  'https://themodel.url/v1/generate' \
  -H "Authorization: Bearer Sh6Kh4[ . . . super long bearer token ]W18UiWuzsz+0r+U"
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "prompt": "What is the difference between a pigeon",
  "llm_config": {
    "use[ . . . and so on]
    }
  }'
```

Curl works like a charm. 

In LangChain, I try to create my new llm as such:

```python
llm = OpenLLM(server_url="https://themodel.url/v1/generate", temperature=0.2)
```

But I don't know how to include this Bearer token, I even suspect it is impossible... 
**Bot seems to confirm this is impossible as of 5 Oct 2023, as seen in https://github.com/langchain-ai/langchain/discussions/11437** ; this issue would then be a feature request.

Thanks! 
Cheers,
Colin

_Originally posted by @ColinLeverger in https://github.com/langchain-ai/langchain/discussions/11437_
Fix .dict() for agent/chain
Labels: name: area: agent, name: auto:improvement

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Add yandex stt parser
Labels: name: lgtm, name: area: doc loader, name: auto:enhancement

Description: Introducing an ability to load a transcription document of audio file using [Yandex SpeechKit](https://cloud.yandex.com/en-ru/services/speechkit)
Issue: None
Dependencies: yandex-speechkit
Tag maintainer: @rlancemartin, @eyurtsev
Fix PyMuPDFLoader kwargs
Labels: name: lgtm, name: area: doc loader, name: auto:bug

- **Description:**  Fix the `PyMuPDFLoader` to accept `loader_kwargs` from the document loader's `loader_kwargs` option. This provides more flexibility in formatting the output from documents.

- **Issue:**  The `loader_kwargs` is not passed into the `load` method from the document loader, which limits configuration options.

- **Dependencies:**  None
Fix PyMUPDFLoader kwargs
Labels: name: area: doc loader, name: auto:improvement

- **Description:**  Fix the `PyMuPDFLoader` to accept `loader_kwargs` from the document loader's `loader_kwargs` option. This provides more flexibility in formatting the output documents.

- **Issue:**  The `loader_kwargs` is not passed into the `load` method from the document loader, which limits configuration options.

- **Dependencies:**  None
Nc/5oct/runnable release
Labels: name: auto:improvement

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Parse string arguments correctly when using a structured tool
Labels: name: auto:bug

**Description:**
Previously when setting up a structured tool with multiple parameters e.g. `arg1, arg2`
we got a schema validation error by pydantic. The reason for this was simple: Although the required parameters for the tool were correctly identified, they were propagated as string e.g. `'{"arg1": "foo", "arg2": "bar"}'`

This PR fixes this issue by using the `parse_raw` method if the args are of type string

**Issue:** 
Didn't find a related issue.

**Dependencies:**
None

**Tag maintainer:**
None

**Twitter handle:**
None

/CC @Depaccu
pip installing 'langchain[all]' installs a newer (broken) version than pip install langchain does
Labels: name: area: doc loader, name: auto:bug, name: auto:question

### System Info

I am using Python 3.11.5 and I run into this issue with `ModuleNotFoundError: No module named 'langchain.document_loaders'` after running `pip install 'langchain[all]'`, which appears to be installing langchain-0.0.39

If I then run `pip uninstall langchain`, followed by `pip install langchain`, it proceeds to install langchain-0.0.308 and suddenly my document loaders work again.

 - Why does langchain 0.0.39 have document_loaders broken?
 - Why does langchain[all] install a different version of langchain?


### Reproduction

1. Install python 3.11.5 (using pyenv on OSX)
2. Run `pip install `langchain[all]'`
3. Observe langchain version that is installed
4. Run `ipython` and execute `from langchain.document_loaders import WebBaseLoader` to receive the module error.
5. Run `pip uninstall langchain`
6. Run `pip install langchain`
7. Note how an older version of langchain is installed
8. Run `ipython` and execute `from langchain.document_loaders import WebBaseLoader` to see how it's working.

### Expected behavior

`pip install langchain` should install the same underlying version of langchain as the version of langchain that includes all of the various modules.
DOC: QA using a Retriever, why do we need additional empty space?
Labels: name: area: doc loader, name: auto:question, name: auto:documentation

### Issue with current documentation:

In [QA using a Retriever docs](https://python.langchain.com/docs/use_cases/question_answering/how_to/vector_db_qa) you can observe that after each LLM reply we observe additional empty space. What is the purpose of having this space or is it just a bug in Prompt? 

Copied from the docs provided above
```python3
query = "What did the president say about Ketanji Brown Jackson"
qa.run(query)
```
Provides response:
`" The president said that she is one of the nation's top legal minds ..."`

### Idea or request for content:

_No response_
Issue: Custom Redirect URI as parameter in SharePointLoader
Labels: name: area: doc loader, name: auto:question

### Issue you'd like to raise.

I am trying to access documents from SharePoint using Langchain's [SharePointLoader](https://api.python.langchain.com/en/latest/document_loaders/langchain.document_loaders.sharepoint.SharePointLoader.html).

I changed the redirect URI to a local host on this [documentation](https://python.langchain.com/docs/integrations/document_loaders/microsoft_sharepoint). How can I pass the new redirect URI as a parameter to the [SharePointLoader](https://api.python.langchain.com/en/latest/document_loaders/langchain.document_loaders.sharepoint.SharePointLoader.html) function.

### Suggestion:

_No response_
Upgrade langchain dependencies.
Labels: name: auto:nit

I was hoping this would pick up numpy 1.26, which is required to support the new Python 3.12 release, but it didn't. It seems that some transitive dependency requirement on numpy is preventing that, and the highest we can currently go is 1.24.x.

But to find this out required a 15min `poetry lock`, so I figured we might as well upgrade the dependencies we can and hopefully make the next dependency upgrade a bit smaller.
Wfh/rtds
Labels: name: area: doc loader, name: auto:documentation

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->