delayedkarma/langchain-issues huggingface dataset:
Fixed wrong paper reference
Labels: name: area: agent, name: auto:documentation

The ReAct reference references to MRKL paper. Corrected so that it points to the actual ReAct paper  #8964.

<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the same people again.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->
Bugs in `GoogleDriveLoader.file_loader_cls`
Labels: name: area: doc loader, name: auto:bug, name: auto:improvement

### System Info

langchain version 0.0.259

### Who can help?

@hwchase17 @baskaryan @eyurtsev

## Issue with `GoogleDriveLoader.file_loader_cls` accepting only classes with constructors that receive a `file` parameter.

The current behavior of `GoogleDriveLoader.file_loader_cls` is that it only accepts a class with a constructor that 
takes a `file` parameter. For instance, it can accept the class `UnstructuredFileIOLoader`, and 
typically other classes like `NotebookLoader` as well. However, when attempting to use 
the `NotebookLoader` class with the following code:

```python
from langchain.document_loaders import NotebookLoader
file_id = "1Hrrf3b4cgjwuKEt1wQUgRtipxqyprKaU"
loader = GoogleDriveLoader(
    file_ids=[file_id],
    file_loader_cls=NotebookLoader,
    file_loader_kwargs={"mode": "elements"},
)
loader.load()
```

An exception is thrown: `TypeError: NotebookLoader.__init__() got an unexpected keyword argument 'file'`.

## Issue with `GoogleDriveLoader` and `import PyPDF2`
If the `file_loader_cls` is not set explicitly, the code attempts to execute `import PyPDF2`. 
However, the code is only designed to handle PDF files. Additionally, the dependencies 
of `PyPDF2` are not declared in the `pyproject.toml` file. Currently, only `pypdf` is 
declared as a dependency. To address this, it is recommended to update the code to utilize `pypdf` 
instead of `PyPDF2`. Otherwise, an exception will be raised.



### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [X] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

from langchain.document_loaders import NotebookLoader
file_id = "1Hrrf3b4cgjwuKEt1wQUgRtipxqyprKaU" # Link to colab file
loader = GoogleDriveLoader(
    file_ids=[file_id],
    file_loader_cls=NotebookLoader,
    file_loader_kwargs={"mode": "elements"},
)
loader.load()

### Expected behavior

The code should be modified to accept the provided code without any exceptions.
I propose a solution with [a pull request](https://github.com/langchain-ai/langchain/pull/5135),
with a reimplementation of this class.
Extend google drive loader
Labels: name: area: doc loader, name: auto:enhancement

### Feature request

- The google drive loader, can manage only GDoc and GSheet. 
- The code can load others type of files, but only with one other loader (`file_loader_cls`).
- It's not possible to request Google Drive to find some files with criteria.
- A Google Drive tools is not exist now.
- The can not manage the Google shortcut
- Can not use the description metadata of Google file
  - Load only the description for a snippet of document
- It's not possible to set the advanced parameters like 
  - corpora, driveId, fields, includeItemsFromAllDrives, includeLabels, includePermissionsForView, orderBy, pageSize, spaces, supportsAllDrives, 
  - Select if return the URL of the document if for view or download
- It's not possible to return
  - For GSheet: mode `single` or `elements` 
  - For GSlide: mode `single`, `elements` or  ̀slide`
- It's not possible to use a fine filter (to refuse some document during the load)
- It's not possible to lazy loading if case of import a long list of documents and save memory
- It's not possible to use a standardized environment variable to manage the authentification (like all others technologies)

[LINK](https://github.com/langchain-ai/langchain/pull/5135)

### Motivation

All my company's documents are on GDrive, with more or less complex organizations and formats 
(in the organization of CVs, customer documents, etc.).
The actual implementation of `GoogleDriveLoader` is very limited.

We tag all CVs with #cv. This allows us to quickly search for skills using Google Drive search. 
Since the directory structure is complex, this approach requires only a single query 
(instead of one query per subdirectory).

We utilize Google Drive searches for various purposes.


### Your contribution

@hwchase17 @baskaryan 

For the last 10 weeks, I've been offering a pull-request number 
[5135](https://github.com/langchain-ai/langchain/pull/5135) that nobody is taking up. 
I've had various commitments, but they've never been kept.

My [proposition](https://github.com/langchain-ai/langchain/pull/5135) resolves all these features, and maintains the compatibility with the current version 
(with many deprecated warning).
Issue: re-implement pivot files to facilitate the integration of new functions
Labels: name: auto:improvement, name: auto:refactor

### Issue you'd like to raise.

@hwchase17

The code utilizes certain pivot files responsible for the comprehensive integration of classes (__init__.py, load_tools.py, etc.).
Currently, maintaining an up-to-date pull request for these files poses challenges, as they undergo updates with each new version. This necessitates frequent rebasing until the code is approved.

In my opinion, it would be more advantageous to establish a mechanism that prevents alterations to these files (via naming conventions, abstract classes, etc.). This way, incorporating a new feature would involve solely the addition of new files without requiring modifications to existing ones. Such an approach could serve as the foundation for implementing plugins.

What are your thoughts on this idea?

### Suggestion:

_No response_
OpenAIEmbeddings expects client parameter
Labels: name: area: embeddings, name: auto:bug, name: auto:documentation, name: area: models

### System Info

langchain: 0.0.257
Python: 3.11.4

### Who can help?

@hwchase17 @agola11 

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [X] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

I am trying to create an object for `OpenAIEmbeddings` as described [here](https://python.langchain.com/docs/integrations/text_embedding/azureopenai), but the constructor expects a `client `parameter which is documented nowhere.
<img width="591" alt="image" src="https://github.com/langchain-ai/langchain/assets/33937977/6d78c87b-bdcb-4c84-82d8-2c171739d47d">


### Expected behavior

Initialize the object without passing a client
Enhance regex of structured_chat agents' output parser
Labels: name: area: agent, name: auto:improvement

Current regex only extracts agent's action between '` ``` ``` `', this commit will extract action between both '` ```json ``` `' and '` ``` ``` `'

This is very similar to #7511 

<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the same people again.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->
Wrong paper reference
Labels: name: area: agent, name: auto:documentation

In the docs it seems the ReAct reference references to the wrong paper.
The actual ReAct paper is [this](https://arxiv.org/pdf/2210.03629.pdf)

https://github.com/langchain-ai/langchain/blob/b8df15cd647ca645ef16b2d66be271dc1f5187c1/docs/docs_skeleton/docs/modules/agents/agent_types/index.mdx#L15
EdenAI LLM update. Add models name option
Labels: name: lgtm, name: auto:improvement, name: area: models

This PR follows the **Eden AI (LLM + embeddings) integration**. #8633 

We added an optional parameter to choose different AI models for providers (like 'text-bison' for provider 'google', 'text-davinci-003' for provider 'openai', etc.).

Usage:

```python
llm = EdenAI(
    feature="text",
    provider="google",
    params={
        "model": "text-bison",  # new
        "temperature": 0.2,
        "max_tokens": 250,
    },
)

```

You can also change the provider + model after initialization
```python
llm = EdenAI(
    feature="text",
    provider="google",
    params={
        "temperature": 0.2,
        "max_tokens": 250,
    },
)

prompt = """
hi 
"""

llm(prompt, providers='openai', model='text-davinci-003')  # change provider & model
```

The jupyter notebook as been updated with an example well.


Ping: @hwchase17, @baskaryan
How to improve my embedding quality?
Labels: name: area: memory, name: area: embeddings, name: auto:question, name: area: models

Now I am using Pinecone and langchain in my Project.
And I am using RecursiveCharacterTextSplitter when I embedding my bot based on my specific data.
And also I am using ConversationalRetrievalQAChain in chain.

These are my code.

```
const text_splitter = new RecursiveCharacterTextSplitter({
        chunkSize: 1000,
        chunkOverlap: 200,
      });
const docs = await text_splitter.splitDocuments(rowDocs);
```

```
const chain = ConversationalRetrievalQAChain.fromLLM(
      llm,
      vectorStore.asRetriever(),
      {
        memory: new BufferMemory({
          memoryKey: 'chat_history', // Must be set to "chat_history"
          inputKey: 'question',
          returnMessages: true,
        }),
      },
    );
```

But my embedding quality is not good.
Sometimes my Bot gives strange answers to my questions. 😢

For example
I have already trained ChatBot using one docx(about Abrantes)
I have asked like that
Me: Please tell me about Abrantes
AI: Abrantes is a Project Controls Specialist ....
Me: What Project he managed?
AI: Ed has managed some .....

Who is Ed in this answer? 😣

There is no Ed in this docx too.

So how can I fix it?
Agents on a local llm with custom data
Labels: name: area: agent, name: auto:question, name: area: models

I'm implementing vectorstore agent on my custom data. Can I implement this with  a local llm like gpt4all(GPT4All-J v1.3-groovy) 
Can agents give better and expected answer when we use agents, or should go with better model like(gpt4, llama2)?
Issue: FAISS taking long time to add to index for 30MB csv file
Labels: name: area: embeddings, name: area: vector store, name: auto:question, name: auto:improvement

### Issue you'd like to raise.

FAISS is taking around 12hrs to create embedding and add it to index for 100000 rows csv file. is there any bulk load strategy for CSV files embedding

### Suggestion:

_No response_
Embeddings are regenerated for entire vector db on updating collections
Labels: name: area: embeddings, name: area: vector store, name: auto:bug

### System Info

macOS Ventura Version `13.4.1`
Python `3.11.4`
```
langchain==0.0.251
chromadb==0.4.5
```

### Who can help?

@hwchase17 @eyurtsev 

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [X] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

1. Create a vector database and insert some items with relevant embeddings
2. Update vector database collection with some new docs

Python snippet to create/update vector db collection and it's embeddings:
```
documents = []
for row in get_data_rows():
    documents.append(
        Document(
            page_content=row['full_text'],
            metadata={
                'id': int(row['id']),
                'created_at': parser.parse(row['created_at']).timestamp(),
            }
        )
    )

Chroma.from_documents(
    documents,
    embedding=OpenAIEmbeddings(),
    collection_name='my_collection',
    persist_directory=f'embeddings/my_collection'
)
```

### Expected behavior

Running above snippet in both scenarios should only call embeddings function for newly added or changed docs instead of everything?
lang chain connection with mongo DB 
Labels: name: area: agent, name: auto:question

### Issue you'd like to raise.

Can you please help me with connecting my LangChain agent to a MongoDB database?
I have connected my relational database below like 

db = SQLDatabase.from_uri("mysql_db_url")

The above process is not working for Mongo Db because it is a NoSQL database 
anyone can help me to do this ???????
Thank you .....

### Suggestion:

_No response_
add filter by metadata to annoy search
Labels: name: area: vector store, name: auto:improvement

# What
- This is to add filter by metadata to annoy search with test.

<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: add filter by metadata to annoy search
  - Issue: None
  - Dependencies: None
  - Tag maintainer: @rlancemartin, @eyurtsev
  - Twitter handle: @MLOpsJ

Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the same people again.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->
refactor(document_loaders): abstract page evaluation logic in PlaywrightURLLoader
Labels: name: area: doc loader, name: auto:refactor

This PR brings structural updates to `PlaywrightURLLoader`, aiming at making the code more readable and extensible through the abstraction of page evaluation logic. These changes also align this implementation with a similar structure used in LangChain.js. 

The key enhancements include:

1. Introduction of 'PlaywrightEvaluator', an abstract base class for all evaluators. 
2. Creation of 'UnstructuredHtmlEvaluator', a concrete class implementing 'PlaywrightEvaluator', which uses `unstructured` library for processing page's HTML content.
3. Extension of 'PlaywrightURLLoader' constructor to optionally accept an evaluator of the type 'PlaywrightEvaluator'. It defaults to 'UnstructuredHtmlEvaluator' if no evaluator is provided.
4. Refactoring of 'load' and 'aload' methods to use the 'evaluate' and 'evaluate_async' methods of the provided 'PageEvaluator' for page content handling.

This update brings flexibility to 'PlaywrightURLLoader' as it can now utilize different evaluators for page processing depending on the requirement. The abstraction also improves code maintainability and readability.

Please find the reference for LangChain.js implementation [here](https://github.com/hwchase17/langchainjs/blob/29f3ffdcb34acce1f1db78078f4e7d184eaec6b5/langchain/src/document_loaders/web/playwright.ts#L59).

Maintainers for review: @rlancemartin, @eyurtsev

Twitter: @ywkim
schema consideration while generating query
Labels: name: auto:bug, name: area: models

### System Info

I am working with hugging face opensource models for SQL generation with langchain 
Our model works without a connection db, but when I connect the db, the chain generates inaccurate answers and takes unknown columns.
I am using below models 
wizard, alpaca,vicuna,falcon


### Who can help?

_No response_

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

db_chain = SQLDatabaseChain.from_llm(llm, db=db, verbose=True,prompt=PROMPT)
db_chain.run(input('enter the question:'))

### Expected behavior

I need to connect the database with my open-source model using Langchain
when I ask a question to my chain, the chain generates the correct SQL query with consideration of my database schema.
Hope to get in touch

### Feature request

Dear [langchain](https://github.com/langchain-ai/langchain) developer,
Greetings! I am Jimmy, a community developer and volunteer at InternLM. Your work has been immensely beneficial to me, and I believe it can be effectively utilized in InternLM as well. Welcome to add Discord https://discord.gg/gF9ezcmtM3 . I hope to get in touch with you.

Best regards,
Jimmy

### Motivation

Hope to get in touch

### Your contribution

Hope to get in touch
DOC: Tabular doc not found
Labels: name: area: doc loader, name: auto:documentation

### Issue with current documentation:

got 404 error while looking for tabular documentation:

From link: 
https://python.langchain.com/docs/use_cases/question_answering/

To link: 
https://python.langchain.com/docs/use_cases/tabular

got 404:

![image](https://github.com/langchain-ai/langchain/assets/4071796/d4d88d82-f132-4f0a-8cf2-7e498e182a3f)



### Idea or request for content:

_No response_
`Brave Search` and `DataDog Logs` retrievers
Labels: name: area: doc loader, name: auto:enhancement

- Created two new retrievers: `Brave Search` and `DataDog Logs` based on `BaseLoaderAsRetriever`
- Created `BaseLoaderAsRetriever` with `QueryMixin`. This class is used to treat DocumentLoaders as Retrievers where DocumentLoader has query semantics.
- added retriever examples
- used `Arxiv` as a testbed:
  - derived `ArxivLoader` from  `BaseLoaderAsRetriever`. This eliminates the necessity of a separate `ArxivRetriever`.
  - removed `ArxivRetriever` and tested `ArxivLoader` in place of `ArxivRetriever`
  - changed `Arxiv` examples to address the above changes.
  - 
 @rlancemartin, @eyurtsev
add instructions on integrating Log10
Labels: name: auto:documentation

  - Description: Instruction for integration with Log10: an [open source](https://github.com/log10-io/log10) proxiless LLM data management and application development platform that lets you log, debug and tag your Langchain calls
  - Tag maintainer: @baskaryan
  - Twitter handle: @log10io @coffeephoenix

Several examples showing the integration included [here](https://github.com/log10-io/log10/tree/main/examples/logging) and in the PR
📖 refactored `conversational_retrieval`
Labels: name: area: agent, name: auto:refactor

Fixed #9133 issue.

1. Refactored `conversational_retrieval` by removing unnecessary nesting.
Current nesting makes `API Reference` unreadable (see the `description` column that is 6-char width):
![image](https://github.com/langchain-ai/langchain/assets/2256422/ba09a924-51d2-4f69-bfe8-3d8de1a3939a)
It makes the full function name (with namespace) soo long, which makes the `description` column unreadable.
The resulting namespace+name is still too long.

2. renamed function `create_conversational_retrieval_agent` -> `create_agent`. Old name kept for backward compatibility

NOTE: The problem of too-long namespaces is also addressed in #8398 Please, review #8398 !!!!!

@baskaryan, @hinthorn
DOC: API Reference: bug in the Search
Labels: name: auto:bug, name: auto:documentation

### Issue with current documentation:

Try https://api.python.langchain.com/en/latest/search.html?q=PubMed (a search on the "PubMed" string)
It returns too many lines.
It seems it converts every found "PubMed" string into a link. That creates duplicate links in big quantities.

### Idea or request for content:

Fix a script that generates the "Search results" page.