delayedkarma/langchain-issues huggingface dataset:
Add integration for MLflow AI Gateway
Labels: name: auto:enhancement

<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the same people again.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->


- Adds integration for MLflow AI Gateway (this will be shipped in MLflow 2.5 this week).


Manual testing:

```sh
# Move to mlflow repo
cd /path/to/mlflow

# install langchain
pip install git+https://github.com/harupy/langchain.git@gateway-integration

# launch gateway service
mlflow gateway start --config-path examples/gateway/openai/config.yaml

# Then, run the examples in this PR
```
STRUCTURED_CHAT_ZERO_SHOT_REACT_DESCRIPTION Custom Tools Failure
Labels: name: auto:bug

### System Info

I have been using STRUCTURED_CHAT_ZERO_SHOT_REACT_DESCRIPTION with a custom tool that takes 2 inputs, however have not been able to get the agent to produce the outputs we had before. Specifically, we're getting the intermediate, custom function input as output. 

Ie instead of getting a value associated with a query "What is the `purchase total` for customer 5432 on 07-03-2023?"
We are now getting
```
{
  "action": "database_tool",
  "action_input": {
    "customer_id": "5432",
    "local_date": "2023-07-03"
  }
}
```

This did not occur before this weekend. Here's more code snippet in the below example



### Who can help?

@hwcha

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [X] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [X] Agents / Agent Executors
- [X] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

I also tried simplifying the agent call, etc to no avail:
```
llm = ChatOpenAI(
    temperature=0,
    openai_api_key=openai.api_key
)
memory = ConversationBufferWindowMemory(
            memory_key='chat_history',
            k=3,
            return_messages=True,
            input_key='input', 
            output_key="output"
)

@tool(return_direct=False)
def database_tool(customer_id, local_date) -> str:
    """Useful when questions are asked about specific customer_id details, 
        particularly recent ones.
        
        If date is not provided, it will default to today's date in yyyy-mm-dd format.

        Format your input using the following template. 
        ```
        {{
        "action": "database_action",
        "action_input": {{"customer_id": "<customer id>", "local_date": "<date in yyyy-mm-dd format>"}}
        }}
        ```
    """
    db_query = """<QUERY THAT WORKS>
    """.format(customer_id=customer_id, local_date=local_date)
   
    formatted_d = get_data.query_database(query=db_query)

    return formatted_d

conversational_agent = initialize_agent(
agent=AgentType.STRUCTURED_CHAT_ZERO_SHOT_REACT_DESCRIPTION,
tools=[database_tool],
llm=llm,
verbose=True,
early_stopping_method='generate',
memory=memory,
SystemAgentPromptTemplate=prompt_template+"\n The only tool available is the database tool.",
return_intermediate_steps=True,
return_source_documents=True,
handle_parsing_errors='Check your output and make sure there is an equal number of "{" and "}"'
)

response = conversational_agent("What is the `purchase total` for customer 5432 on 07-03-2023?")
print(response['output'])
```

### Expected behavior

"The purchase total for customer 5432 is 59.55"

I should note the following also works; it's just the agent integration that's problematic:
n = {
    "customer_id": "5432",
    "local_date": "2023-07-03"
  }
database_tool(n)
small fix for chroma embedding function
Labels: name: auto:nit

<!-- Thank you for contributing to LangChain!

Replace this comment with:

- Description: super small fix that makes the Chroma vectorstore object more compatible with the chroma library. 

In chroma when you initialize a client / collection, if you specify an embedding function, it is of type chromadb.utils.embedding_functions.SentenceTransformerEmbeddingFunction and that uses the __call__ to embed stuff. This was not compatable with langchain Chroma object because of [this line](https://github.com/hwchase17/langchain/blob/98dbea6310828ec817dd1f89dfa26a7c1c2de579/langchain/vectorstores/chroma.py#L96C57-L96C72) (the SentenceTransformerEmbeddingFunction) doesn't have an `embed_documents` functions. This PR fixes that so you can initialize a Chroma lang chain object with the same embedding_function object as you could with the chroma library. 

  - Description: Above
  - Issue: none
  - Dependencies:none
  - Tag maintainer: idk
  - Twitter handle: @McClainThiel

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the same people again.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->
Asyncio config in toml file
Labels: name: auto:improvement

<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: Added asyncio in tomal file, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @dev2049
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @dev2049
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @vowelparrot
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the same people again.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->
Fix GPT4All bug w/ "n_ctx" param 
Labels: name: auto:bug, name: auto:improvement

Running `GPT4All` per the [docs](https://python.langchain.com/docs/modules/model_io/models/llms/integrations/gpt4all), I see:

```
$ from langchain.llms import GPT4All
$ model = GPT4All(model=local_path)
$ model("The capital of France is ", max_tokens=10)
TypeError: generate() got an unexpected keyword argument 'n_ctx'
```

It appears `n_ctx` is [no longer a supported param](https://docs.gpt4all.io/gpt4all_python.html#gpt4all.gpt4all.GPT4All.generate) in the GPT4All API from https://github.com/nomic-ai/gpt4all/pull/1090.

It now uses `max_tokens`, so I set this.

And I also set other defaults used in GPT4All client [here](https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-bindings/python/gpt4all/gpt4all.py).

Confirm it now works:
```
$ from langchain.llms import GPT4All
$ model = GPT4All(model=local_path)
$ model("The capital of France is ", max_tokens=10)
< Model logging > 
"....Paris."
```
DOC: enhancement with Llama-cpp document
Labels: name: auto:improvement, name: auto:documentation

### Issue with current documentation:

The current document didn't cover the Metal support of Llama-cpp, which need different parameter to build.

Also, the default parameter of initial LLM will crash error:

```
GGML_ASSERT: .../vendor/llama.cpp/ggml-metal.m:706: false && "not implemented"
```


### Idea or request for content:

I would like to contribute a pull request to polish the document with better Metal (Apple Silicon Chip) support of Llama-cpp.
Add events to tracer runs
Labels: name: auto:improvement

<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @dev2049
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @dev2049
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @vowelparrot
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the same people again.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->
Use EggTimer to simplify timeout logic
Labels: name: auto:improvement

## Description

[EggTimer](https://pypi.org/project/egg-timer/) is a simple Python library that makes handling timeouts easier. Instead of tracking time elapsed, start times, and performing calculations, using EggTimer is a simple as `timer = EggTimer(); timer.set(TIMEOUT)`. Timeout can be checked by calling `timer.is_expired() -> bool`.

This PR modifies the `Agent` and `BlockchainDocumentLoader` to use EggTimer.

## A note about dependencies

I don't think there's a way to make this dependency optional, since it's used by the Agent. LMK if there's something different I should do.

## Tag Maintainer
Agents: @vowelparrot 
DataLoaders: @rlancemartin @eyurtsev
Update chroma.py: Persist directory from client_settings if provided there
Labels: name: auto:bug

Change details:
  - Description: When calling db.persist(), a check prevents from it proceeding as the constructor only sets member `_persist_directory` from parameters. But the ChromaDB client settings also has this parameter, and if the client_settings parameter is used without passing the persist_directory (which is optional), the `persist` method raises `ValueError` for not setting `_persist_directory`. This change fixes it by setting the member `_persist_directory` variable from client_settings if it is set, else uses the constructor parameter.
  - Issue: I didn't find any github issue of this, but I discovered it after calling the persist method
  - Dependencies: None
  - Tag maintainer: vectorstore related change - @rlancemartin, @eyurtsev
  - Twitter handle: Don't have one :(

*Additional discussion*: We may need to discuss the way I implemented the fallback using `or`.
Remove `None` default value for FAISS relevance_score_fn
Labels: name: auto:improvement

## Description

The type hint for `FAISS.__init__()`'s `relevance_score_fn` parameter allowed the parameter to be set to `None`. However, a default function is provided by the constructor. This led to an unnecessary check in the code, as well as a test to verify this check.

**ASSUMPTION**: There's no reason to ever set `relevance_score_fn` to `None`.

This PR changes the type hint and removes the unnecessary code.

## Tag Maintainer
@rlancemartin @eyurtsev
ChromaDB relevance score fn
Labels: name: auto:improvement

## Description

This PR adds a `relevance_score_fn` to `Chroma.__init__()` that allows developers to pass in a custom function to map relevance scores to [0,1]. This code follows the pattern set by `FAISS`.

Other changes made:
    - `Chroma.from_{documents,texts}()` pass `kwargs` through to the constructor

## Tag maintainer
@rlancemartin @eyurtsev
Mark some output parsers as serializable (cross-checked w/ JS)
Labels: name: auto:improvement

<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @dev2049
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @dev2049
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @vowelparrot
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the same people again.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->
SQLDatabaseChain Double Query
Labels: name: auto:bug

### System Info

When trying to do an aggregation on a table (I've tried average, min, max), two duplicate queries are being generated, resulting in an error

```
syntax error line 2 at position 0 unexpected 'SELECT'.
[SQL: SELECT AVG(YEAR) FROM my_table
SELECT AVG(YEAR) FROM my_table]
```

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

1. db_chain = SQLDatabaseChain.from_llm(...)
2. db_chain.run("what is the average year?")

### Expected behavior

should return the average year
Serialization of OpenLLM Local Inference Models does not work.
Labels: name: auto:bug

### System Info

- Langchain Version 0.0.219
## Where does the problem arise:
When saving the OpenLLM model, there is a key called "llm_kwargs" which cannot be parsed when reloading the model. Instead the "llm_kwargs" should be directly inside the "llm" object without the key "llm_kwargs".
### Example
```json
"llm": {
            "server_url": null,
            "server_type": "http",
            "embedded": true,
            "llm_kwargs": {
                "temperature": 0.2,
                "format_outputs": false,
                "generation_config": {
                    "max_new_tokens": 1024,
                    "min_length": 0,
                    "early_stopping": false,
                    "num_beams": 1,
                    "num_beam_groups": 1,
                    "use_cache": true,
                    "temperature": 0.2,
                    "top_k": 15,
                    "top_p": 1.0,
                    "typical_p": 1.0,
                    "epsilon_cutoff": 0.0,
                    "eta_cutoff": 0.0,
                    "diversity_penalty": 0.0,
                    "repetition_penalty": 1.0,
                    "encoder_repetition_penalty": 1.0,
                    "length_penalty": 1.0,
                    "no_repeat_ngram_size": 0,
                    "renormalize_logits": false,
                    "remove_invalid_values": false,
                    "num_return_sequences": 1,
                    "output_attentions": false,
                    "output_hidden_states": false,
                    "output_scores": false,
                    "encoder_no_repeat_ngram_size": 0
                }
            },
            "model_name": "opt",
            "model_id": "facebook/opt-1.3b",
            "_type": "openllm"
        }
```
In contrast the serialization for a OpenAI LLM looks the following:
```json
"llm": {
            "model_name": "text-davinci-003",
            "temperature": 0.7,
            "max_tokens": 256,
            "top_p": 1,
            "frequency_penalty": 0,
            "presence_penalty": 0,
            "n": 1,
            "request_timeout": null,
            "logit_bias": {},
            "_type": "openai"
        }
```

### Who can help?

@hwchase17 @eyurtsev @agola11 

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Steps to reproduce:

1. Create a OpenLLM object
2. Call `save()` on that object
3. Try to reload the previously generated OpenLLM object with `langchain.chains.loading.load_chain_from_config()`

### Expected behavior

The serialized OpenLLM model should be reloadable.
Add serialized object to retriever start callback
Labels: name: auto:improvement

<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @dev2049
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @dev2049
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @vowelparrot
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the same people again.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->
Add tags to all callback handler methods
Labels: name: auto:improvement

<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @dev2049
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @dev2049
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @vowelparrot
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the same people again.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->
Update base.py
Labels: name: needs work, name: auto:improvement

Added Support for Multiple Dataframes

<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @dev2049
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @dev2049
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @vowelparrot
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the same people again.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->
Retryable exception for empty OpenAI embedding.
Labels: name: lgtm, name: auto:bug

Description:

The OpenAI "embeddings" API intermittently falls into a failure state where an embedding is returned as [ Nan ], rather than the expected 1536 floats.  This patch checks for that state (specifically, for an embedding of length 1) and if it occurs, throws an ApiError, which will cause the chunk to be retried.

Issue:

I have been unable to find an official langchain issue for this problem, but it is discussed (by another user) at https://stackoverflow.com/questions/76469415/getting-embeddings-of-length-1-from-langchain-openaiembeddings

Maintainer: @dev2049

Testing: 

Since this is an intermittent OpenAI issue, I have not provided a unit or integration test.  The provided code has, though, been run successfully over several million tokens.
Add concurrency to GitbookLoader
Labels: name: lgtm, name: auto:improvement

  - Description: Fetch all pages concurrently.
  - Dependencies:  `scrape_all` -> `fetch_all` -> `_fetch_with_rate_limit` -> `_fetch` (might be broken currently: https://github.com/hwchase17/langchain/pull/6519)
  - Tag maintainer: @rlancemartin, @eyurtsev