delayedkarma/langchain-issues huggingface dataset:
DirectoryLoader for different file types
Labels: name: area: doc loader, name: auto:question

### Discussed in https://github.com/langchain-ai/langchain/discussions/9605

<div type='discussions-op-text'>

<sup>Originally posted by **nima-cp** August 22, 2023</sup>
Hello everyone, I wanna have a Q&A over some documents including pdf, xml and csv. I'm having some difficulty to write a DirectoryLoader for different types of files in a folder. I'm using Chroma and couldn't find a way to solve this. However, I found this in TS documentation:

```typescript
   const directoryLoader = new DirectoryLoader(filePath, {
      '.pdf': (path) => new PDFLoader(path, { splitPages: true }),
      '.docx': (path) => new DocxLoader(path),
      '.json': (path) => new JSONLoader(path, '/texts'),
      '.jsonl': (path) => new JSONLinesLoader(path, '/html'),
      '.txt': (path) => new TextLoader(path),
      '.csv': (path) => new CSVLoader(path, 'text'),
      '.htm': (path) => new UnstructuredLoader(path),
      '.html': (path) => new UnstructuredLoader(path),
      '.ppt': (path) => new UnstructuredLoader(path),
      '.pptx': (path) => new UnstructuredLoader(path),
    });
```
How can I write the same in Python?

```python
loader = DirectoryLoader(
    filePath,
    glob="./*.pdf",
    loader_cls=PyMuPDFLoader,
    show_progress=True,
)
```</div>
BambooHR Tool
Labels: name: auto:enhancement

### Feature request

I would like to have BambooHR Tool here(https://python.langchain.com/docs/integrations/tools/).
To make it possible to request information about employees within the company, their vacations, and so on.  

### Motivation

BambooHR is quite a popular service, so I believe this tool will be used a lot. 

### Your contribution

I am willing to contribute by coding a portion, but I am hesitant to code everything as it may be too much. It would be great if other enthusiasts could also join in.

I've already found the BambooHR OpenAPI file
[bamboo_openapi.json.zip](https://github.com/langchain-ai/langchain/files/12437406/bamboo_openapi.json.zip)
Issue: LLM Multiple access problem 
Labels: name: auto:bug, name: auto:question, name: area: models

Hi, I would like to build a chat bot to support multple users to access the chat bot.

Since the llm model size is very big, my VRAM can only load only one copy of LLM.
I would like to know if there is any way to load the model once and multiple access concurrently.

Here is what I just tried.
I tried to create two threads, and each thread run the llm model with different prompt.
Unfortunately, the responses are very strange. The r1 and r2 are gibberish code.
If I remove one of the thread, the response is good.

`
from langchain.llms import CTransformers
from langchain.llms import LlamaCpp
from langchain import PromptTemplate, LLMChain
from langchain.callbacks.manager import CallbackManager
from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler

import yaml
import os
import threading
import datetime
import time


def job1():
    print("job1: ", datetime.datetime.now())
    question1 = "Please introduce the history of China"
    r1 =  llm(question1)
    print("r1:", r1)


def job2():
    print("job2: ", datetime.datetime.now())
    question2 = "Please introduce the history of The United States"
    r2 = llm(question2)
    print("r2:", r2)

// load the model once
llm = LlamaCpp(
    model_path="/workspace/test/llama-2-7b-combined/ggml-model-q6_K.bin",
    n_gpu_layers=20,
    n_batch=128,
    n_ctx=2048,
    temperature=0.1,
    max_tokens=512,
)

t1 = threading.Thread(target=job1)
t2 = threading.Thread(target=job2)
t1.start()
t2.start()

t1.join()
t2.join()

print("Done.")

`



### Suggestion:

_No response_
Add `optional_variables` for templates and make `format_document` customizable
Labels: name: area: doc loader, name: auto:improvement

### Feature request

 1. Ideally, current `input_variables` should be separated into `required_variables` and `allowed_variables`
 2. `allowed_variables` should consist of `required_variables` + `optional_variables`
 3. Current implementation of `format_document` requires some overhaul, as suggested by #7239. Since `format_document` is part of the schema, it should either be a class or, at least, its formatting and validation parts should be separated.

```python
# from:
def format_document(doc: Document, prompt: BasePromptTemplate) -> str:
    base_info = {"page_content": doc.page_content, **doc.metadata}
    missing_metadata = set(prompt.input_variables).difference(base_info)
    if len(missing_metadata) > 0:
        required_metadata = [
            iv for iv in prompt.input_variables if iv != "page_content"
        ]
        raise ValueError(
            f"Document prompt requires documents to have metadata variables: "
            f"{required_metadata}. Received document with missing metadata: "
            f"{list(missing_metadata)}."
        )
    document_info = {k: base_info[k] for k in prompt.input_variables}
    return prompt.format(**document_info)

# into (assumes required_variables is input_variables - optional_variables, backward compatible, not ideal or elegant so far):
def _validate_document(doc: Document, prompt: BasePromptTemplate) -> None:
   base_info = {"page_content": doc.page_content, **doc.metadata}
   missing_metadata = set(prompt.required_variables).difference(base_info)
    if missing_metadata:
        raise ValueError(
            f"Document prompt requires documents to have metadata variables: "
            f"{prompt.required_variables}. Received document with missing metadata: "
            f"{list(missing_metadata)}."
        )

def _format_document(doc: Document, prompt: BasePromptTemplate) -> None:
    base_info = {"page_content": doc.page_content, **doc.metadata}
    document_info = {k: base_info[k] for k in prompt.input_variables} # or allowed_variables
    return prompt.format(**document_info)

def format_document(doc: Document, prompt: BasePromptTemplate, validation_function: Callable = _validate_document, formatting_function: Callable = _format_document, **kwargs) -> str:
    _validate_document(doc, prompt)
    return _format_document(doc, prompt, **kwargs) # format_kwargs?
```

### Motivation

Given that both `f-string` and `jinja2` support some control logic, it seems quite logical to allow optional variables, or to make `format_document` function more customizable.

### Your contribution

I'd like to work on it, but I believe there's a need for further discussion.
Fix: when only config custom evaluators, the keys(input_key,prediction_key,reference_key) config will be lost 
Labels: name: auto:bug

* Description:  When only use  custom evaluators, Will meet ` Could not map run {which} with multiple keys {source} \n Please manually specify a {which}_key`  ERROR from `ChainStringRunMapper`.   Because  the configuration of the keys(input_key,prediction_key,reference_key) are initialized only when `config.evaluators` are configured. So `ChainStringRunMapper` can't get the keys configuration.
* Issue: None
* Dependencies: None
* Maintainer:  @hinthornw @baskaryan 
* Twitter handle: @_LiuHu
Fix QuestionListOutputParser
Labels: name: auto:bug, name: area: models

This PR fixes `QuestionListOutputParser` text splitting.

`QuestionListOutputParser` incorrectly splits numbered list text into lines. If text doesn't end with `\n` , the regex doesn't capture the last item. So it always returns `n - 1` items, and `WebResearchRetriever.llm_chain` generates less queries than requested in the search prompt.

How to reproduce:

```python
from langchain.retrievers.web_research import QuestionListOutputParser

parser = QuestionListOutputParser()

good = parser.parse(
    """1. This is line one.
    2. This is line two.
    """  # <-- !
)

bad = parser.parse(
    """1. This is line one.
    2. This is line two."""    # <-- No new line.
)

assert good.lines == ['1. This is line one.\n', '2. This is line two.\n'], good.lines
assert bad.lines == ['1. This is line one.\n', '2. This is line two.'], bad.lines
```

NOTE: Last item will not contain a line break but this seems ok because the items are stripped in the `WebResearchRetriever.clean_search_query()`.
Issue: Error in Similarity Search with MongoDB Vector Store in Langchain
Labels: name: area: vector store, name: auto:bug

### Issue you'd like to raise.

I've been using Langchain to connect with the MongoDB vector store. While the file upload functionality works seamlessly, I encounter an error when trying to use the similarity search feature. Here's the error message I receive:
![image](https://github.com/langchain-ai/langchain/assets/101790522/f73cb88d-ae8c-49d8-995d-2b8749397508)


### Suggestion:

_No response_
AWS Sagemaker
Labels: name: auto:bug, name: area: models

### System Info

Hi, I'm getting the following error with Langchain integration with AWS Sagemaker: 
`ValueError: Error raised by inference endpoint: An error occurred (ModelError) when calling the InvokeEndpoint operation: Received client error (424) from primary with message "{
  "code":424,
  "message":"prediction failure",
  "error":"Input payload must contain a 'inputs' key and optionally a 'parameters' key containing a dictionary of parameters."
}".`

I've tried adding a custom attribute to accept any relevant terms in order to run my model, but I'm still having issues. See below for my initialization of the model:
chain = load_qa_chain(
    llm=SagemakerEndpoint(
        endpoint_name="endpointname"
        credentials_profile_name="profilename",
        region_name="region",
        model_kwargs={"temperature": 1e-10},
        endpoint_kwargs={"CustomAttributes": "accept_eula=true"},
        content_handler=content_handler,
    ),
    prompt=PROMPT,
)


### Who can help?

_No response_

### Information

- [X] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

chain = load_qa_chain(
    llm=SagemakerEndpoint(
        endpoint_name="endpointname"
        credentials_profile_name="profilename",
        region_name="region",
        model_kwargs={"temperature": 1e-10},
        endpoint_kwargs={"CustomAttributes": "accept_eula=true"},
        content_handler=content_handler,
    ),
    prompt=PROMPT,
)


### Expected behavior

error described above
Langchain agent doesn't complete reasoning sequence stops halfway and can't use structured tools
Labels: name: area: agent, name: auto:bug

### System Info

langchain==0.0.271

### Who can help?

@hwchase17 @agola11 

### Information

- [X] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [X] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

from langchain import PromptTemplate
from langchain.agents import AgentType
from dotenv import load_dotenv
from langchain.chat_models import ChatOpenAI
from langchain.agents import initialize_agent
from api.utils.agent.tools import view_capabilities_tool
load_dotenv()
system_message = "You are required to use only the tools provided to answer a query.If no given tool can help,"\
                  " truthfully tell the user that you are unable to help them.Always end reply with see ya!."\
                  "Query: {query}"


prompt_template = PromptTemplate(
        template=system_message,
        input_variables=["query"],
    )

capabilities = view_capabilities_tool.CapabilitiesTool()

llm = ChatOpenAI(temperature=0)
agent_chain = initialize_agent(
    [capabilities],
    llm,
    agent=AgentType.STRUCTURED_CHAT_ZERO_SHOT_REACT_DESCRIPTION,
    verbose=True,
    agent_kwargs={
      "system_message": system_message
    }
)

response = agent_chain.run(input="What can you do")
print(response)



capabilities_tool:
from typing import Type
from langchain.tools import BaseTool
from pydantic import BaseModel, BaseSettings


class CapabilitiesToolSchema(BaseModel):
    pass


class CapabilitiesTool(BaseTool, BaseSettings):
    name = "capabilities_tool"
    description = """Tool that shows what you are capable of doing."""
    args_schema: Type[CapabilitiesToolSchema] = CapabilitiesToolSchema

    def _run(
        self,
    ) -> str:
        body = ("I can help you out with"
                "\nAdding a site\nRemoving a  site\nAdding an interest\nRemoving an interest\nViewing your details\n "
                "Opt out")

        return body



### Expected behavior

The model is meant to go through the whole reasoning process, select a tool and wait for the response for that tool called.
Instead the agent just stops at """Action_Input: ...""" every time. The model doesnt use any tool given, sometimes it would give this steps:

> Entering new AgentExecutor chain...
I can use the capabilities_tool to see what I am capable of doing. Let me check.

> Finished chain.
I can use the capabilities_tool to see what I am capable of doing. Let me check.
Enable grammars for JSON, list w/ LLaMA
Labels: name: auto:improvement, name: area: models

Takes feedback from [here](https://github.com/abetlen/llama-cpp-python/issues/637#issuecomment-1692459836).

And builds on [here](https://github.com/langchain-ai/langchain/pull/9712).

I needed to modify `json.gbnf` to get it to work properly (else, it does not stop, and repeats newlines).

I check it in temporarily, but it's not obvious it should actually live in this location.
Pgvector support to filter by List
Labels: name: area: vector store, name: auto:improvement

### Feature request

Hi there, 

Currently using PGVector you can pass a filter object. But this filter object only allows key value pairs. 

`Dict[str, str]. 
`
I am requesting the ability to also send a list of strings for easy filter across many pieces of data.

`Dict[str, list[str]]
`
### Motivation

I have a lot of Data with Metadata that contains content type. 

I would like to provide a list of content types to PGvector and allow it to filter and return content from multiple types.

### Your contribution

N/A
feat(llms): add missing params to huggingface text-generation
Labels: name: auto:improvement, name: area: models

This small PR aims at supporting the following missing parameters in the `HuggingfaceTextGen` LLM:
- `return_full_text` - sometimes useful for completion tasks
- `do_sample` - quite handy to control the randomness of the model.
- `watermark`

@hwchase17 @baskaryan
remove Value error "pyspark is not installed. Please install it with `pip i…
Labels: name: auto:bug

Description: You cannot execute spark_sql with versions prior to 3.4 due to the introduction of pyspark.errors in version 3.4.
And if you are below you get 3.4 "pyspark is not installed. Please install it with pip nstall pyspark" which is not helpful. Also if you not have pyspark installed you get already the error in init. I would return all errors. But if you have a different idea feel free to comment.

Issue: None
Dependencies: None
Maintainer:
rm google api core
Labels: name: auto:nit

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. These live is docs/extras directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17, @rlancemartin.
 -->
Xata as  a chat message memory store
Labels: name: area: memory, name: auto:enhancement

This adds Xata as a memory store also to the python version of LangChain, similar to the [one for LangChain.js](https://github.com/hwchase17/langchainjs/pull/2217).

I have added a Jupyter Notebook with a simple and a more complex example using an agent.

To run the integration test, you need to execute something like:

```
XATA_API_KEY='xau_...' XATA_DB_URL="https://demo-uni3q8.eu-west-1.xata.sh/db/langchain"  poetry run pytest tests/integration_tests/memory/test_xata.py
```

Where `langchain` is the database you create in Xata.

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. These live is docs/extras directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17, @rlancemartin.
 -->
rfc/wip Bagatur/new msg
Labels: name: auto:enhancement

![Screenshot 2023-08-24 at 12 58 48 PM](https://github.com/langchain-ai/langchain/assets/22008038/3b40de97-9920-4b42-9a72-dd3107f9c0a7)
![Screenshot 2023-08-24 at 12 58 51 PM](https://github.com/langchain-ai/langchain/assets/22008038/23444ed7-c858-432c-a000-56d97e34c648)
change error "pyspark is not installed. Please install it with `pip i…
Labels: name: auto:bug

Description: The issue is you can not run spark_sql with a version below 3.4, because pyspark.errors is added in 3.4.
And if you are below you get 3.4  "pyspark is not installed. Please install it with `pip nstall pyspark`" which is not helpful. Also if you not have pyspark installed you get already the error in __init__. I would return all errors. But if you have a different idea feel free to comment.

Issue: None
Dependencies: None
Maintainer:
Issue: pandas agent tries to create new variables but returns along the lines of "the top 10 are {top_10_df}"
Labels: name: area: agent, name: auto:bug, name: auto:question

### Issue you'd like to raise.

It looks like it's creating new dataframes to answer complex questions, but in the answer it provides it keeps referencing the variable name rather than the printed variable.

Does the current agent not have the ability to create and use new variables?

Note: I'm using the CSV agent as a tool within another agent

### Suggestion:

Will the CSV agent having the create file tool alleviate this?
feat: grammar-based sampling in llama-cpp
Labels: name: auto:enhancement, name: area: models

## Description 

The following PR enables the [grammar-based sampling](https://github.com/ggerganov/llama.cpp/tree/master/grammars) in llama-cpp LLM.

In short, loading file with formal grammar definition will constrain model outputs. For instance, one can force the model to generate valid JSON or generate only python lists.

In the follow-up PR we will add:
* docs with some description why it is cool and how it works
* maybe some code sample for some task such as in llama repo
Nc/runnables retry
Labels: name: auto:improvement

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. These live is docs/extras directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17, @rlancemartin.
 -->
Chat Loaders
Labels: name: area: doc loader, name: auto:enhancement

Still working out interface/notebooks + need discord data dump to test out things other than copy+paste

Update:
- Going to remove the 'user_id' arg in the loaders themselves and just standardize on putting the "sender" arg in the extra kwargs. Then can provide a utility function to map these to ai and human messages
- Going to move the discord one into just a notebook since I don't have a good dump to test on and copy+paste maybe isn't the greatest thing to support in v0
- Need to do more testing on slack since it seems the dump only includes channels and NOT 1 on 1 convos
-
ParentDocumentRetriever doesn't implement BaseRetriever._aget_relevant_documents
Labels: name: area: vector store, name: auto:bug

### System Info

LangChain Version: 0.0.271, Python 3.11

Like the title says, _aget_relevant_documents isn't implemented in ParentDocumentRetriever so async calls are not working.  It throws an NotImplementedError in BaseRetriever:
```
    async def _aget_relevant_documents(
        self, query: str, *, run_manager: AsyncCallbackManagerForRetrieverRun
    ) -> List[Document]:
        """Asynchronously get documents relevant to a query.
        Args:
            query: String to find relevant documents for
            run_manager: The callbacks handler to use
        Returns:
            List of relevant documents
        """
        raise NotImplementedError()
```
        

### Who can help?

@hwchase17 @agola11 

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Just do a chain.acall from a chain using documents, (like a ConverstionalRetrievalChain), and it should trigger this error.

### Expected behavior

Should be able to call the chain async when using ParentDocumentRetriever.