delayedkarma/langchain-issues huggingface dataset:
storing metadata with the VectorStoreRetrieverMemory memory module
Labels: name: area: memory, name: area: vector store, name: auto:improvement

### Feature request

If I want to use VectorStoreRetrieverMemory to store my users' chat memories, I need to search and store them using user_id and session_id. However, memory.save_context doesn't have a 'metadata' option.

### Motivation

I want to associate chat merory with single user

### Your contribution

I can't submit PR
max length attribute for spacy splitter for large docs
Labels: name: lgtm, name: auto:improvement, name: size:S

For large size  documents spacy splitter doesn't work it throws an error as shown in  below screenshot.
Reason its default max_length is 1000000 and there is no option to increase it. So i added it in this PR. 

![image](https://github.com/langchain-ai/langchain/assets/73680423/613625c3-0e21-4834-9aad-2a73cf56eecc)
Fix: (issue #13825) Getting an error with DallEAPIWrapper
Labels: name: lgtm, name: auto:bug, name: size:S

  - **Description:** As of OpenAI's Python package 1.0, the existing DallEAPIWrapper does not work correctly, so the example in the LangChain Documentation link below does not work either. 
https://python.langchain.com/docs/integrations/tools/dalle_image_generator 
Also, since OpenAI only supports DALL-E version 2 or version 3, I modified the DallEAPIWrapper to support it. 

  - **Issue:** #13825 

  - **Twitter handle:** ggeutzzang
docs: fix link to `local_retrieval_qa`
Labels: name: auto:documentation, name: size:XS

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
The original link in [this section](https://python.langchain.com/docs/use_cases/question_answering/#:~:text=locally%2Drunning%20models-,here,-.):
https://python.langchain.com/docs/modules/use_cases/question_answering/local_retrieval_qa

After fix:
https://python.langchain.com/docs/use_cases/question_answering/local_retrieval_qa
Fix: getting an error with DallEAPIWrapper #13825
Labels: name: auto:bug, name: area: models, name: size:S

  - **Description:** Existing DallEAPIWrapper code will throw errors in OpenAI 1.0 and later packages. 
Also, OpenAI now only supports DALL-E2 or DALL-E3.
I modified the existing DallEAPIWrapper to accommodate this.

  - **Issue:** #13825 

  - **Twitter handle:** ggeutzzang
KENDRA: issue with user_context parameter when using get_relevant_documents method (langchain.retrievers.kendra.AmazonKendraRetriever)
Labels: name: area: vector store, name: auto:bug, name: integration: aws

### System Info

Python version: 3.11.5
Langchain version: 0.0.316

### Who can help?

@3coins 
@hw

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

I am using Amazon Kendra as vector store to retrieve relevant documents as part of a Q&A application. As `UserContext` I am using User token:
```
def get_kendra_json_token(user_name: str, groups: List[str]):
    kendra_json_token = {
        'username': user_name,
        'groups': groups
    }
    return kendra_json_token
```
This output is subsequently converted to: `'user_context': {'Token': json.dumps(kendra_json_token)}`
Everything is fine when I build the Retriever:
```
def get_kendra_doc_retriever(inputs: KendraRequest) -> AmazonKendraRetriever:
    try:
        kendra_client = boto3.client("kendra", os.environ.get('AWS_REGION'))
        retriever = AmazonKendraRetriever(
            index_id=inputs.kendra_index_id,
            top_k=get_param(AIAssistantParam.NB_KENDRA_DOCS),
            client=kendra_client,
            attribute_filter=inputs.attribute_filter,
            user_context=inputs.user_context
        )
        logger.info(f'Kendra retriever successfully instantiated')
        return retriever
```
But then, when I call `get_relevant_documents`:
```
def ask_question(
        chain: Chain,
        retriever: AmazonKendraRetriever,
        question: str
) -> Response:
        
    try:
        context = retriever.get_relevant_documents(question)
```
I get this exception: `An error occurred (AccessDeniedException) when calling the Retrieve operation: The provided JSON token isn't valid. The username couldn't be parsed. Generate a new token with username as an array of strings and try your request again.` Of course, `username` should be an string.

If I change the code doing this (swapping the content of `user_name` and `groups` in the user token):
```
def get_kendra_json_token(user_name: List[str], groups: str):
    kendra_json_token = {
        'username': groups,
        'groups': user_name
    }
    return kendra_json_token
```
everything works fine. It is like `user_name` and `groups` inputs parameters are messed up somewhere.

 

### Expected behavior

No exception should be raised when creating the user token as explained in the description above.
Support for elastic index without metadata field
Labels: name: area: vector store, name: auto:improvement, name: integration: elasticsearch

### Feature request

Currently the `_search` function in `ElasticsearchStore` assumes that the `hit` object returned in the search has a `metadata` field under `_source`:
```python
                    hit["_source"]["metadata"][field] = hit["_source"][field]
```

However, this is not the case in the index I work with - it does not have a `metadata` field.  Due to that, an exception is raised.

Note that the following code does not help - 
```python
        if "metadata" not in fields:
            fields.append("metadata")
```
The index still does not return any `metadata`.

I assume that in indexes created by `ElasticsearchStore` the `metadata` field is forced, and therefor there is no such issue. However, when using indexes created by external tools, it is better not to assume that the field exists, and support the case where it doesn't. 




### Motivation

I'd prefer to re-use the existing `ElasticsearchStore` instead of my own implementation of it.

### Your contribution

I think I can contribute a PR handling this issue, if the admins confirm the feature request.
DOC: How langchain_experimental.rl_chain works
Labels: name: auto:documentation, name: area: models

### Issue with current documentation:

Hi, can we get more documentation on `langchain_experimental.rl_chain`? I'm having trouble wrapping my head around how it works, and the documentation is sparse. 

From the notebook intro, originally I thought it was going to tune the human written prompt template and then output a new and improved prompt template that it found was better. However it seems to be doing something else.

### Idea or request for content:

_No response_
TypeError using Baseten in a RAG
Labels: name: auto:bug, name: area: models

### System Info

Hello,

I am trying to use a Baseten base LLM in a RAG pipeline.

```
from operator import itemgetter
from langchain.llms import Baseten
from langchain.schema.runnable import RunnableMap

llm = Baseten(model="MODEL_ID", verbose=True)

rag_chain_from_docs = (
    {
        "context": lambda input: format_docs(input["documents"]),
        "question": itemgetter("question"),
    }
    | rag_prompt_custom
    | llm
    | StrOutputParser()
)
rag_chain_with_source = RunnableMap(
    {"documents": retriever, "question": RunnablePassthrough()}
) | {
    "documents": lambda input: [doc.metadata for doc in input["documents"]],
    "answer": rag_chain_from_docs,
}

rag_chain_with_source.invoke("What is Task Decomposition")
```

I am using a FAISS retriever and I am getting the following error on the `.invoke()` method:

```
File "/Users/usr/miniconda3/envs/langchain/lib/python3.11/site-packages/langchain/llms/baseten.py", line 69, in _call
    response = model.predict({"prompt": prompt, **kwargs})
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/usr/miniconda3/envs/langchain/lib/python3.11/site-packages/baseten/common/core.py", line 67, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/usr/miniconda3/envs/langchain/lib/python3.11/site-packages/baseten/baseten_deployed_model.py", line 124, in predict
    raise TypeError('predict can be called with either a list, a pandas DataFrame, or a numpy array.')
TypeError: predict can be called with either a list, a pandas DataFrame, or a numpy array.
```

It seems the `model.predict()` method is expecting a list. Does anyone already encountered this error?

Thank you in advance !

### Who can help?

@hwchase17 @agola11 

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

```
from operator import itemgetter
from langchain.llms import Baseten
from langchain.schema.runnable import RunnableMap

llm = Baseten(model="MODEL_ID", verbose=True)

rag_chain_from_docs = (
    {
        "context": lambda input: format_docs(input["documents"]),
        "question": itemgetter("question"),
    }
    | rag_prompt_custom
    | llm
    | StrOutputParser()
)
rag_chain_with_source = RunnableMap(
    {"documents": retriever, "question": RunnablePassthrough()}
) | {
    "documents": lambda input: [doc.metadata for doc in input["documents"]],
    "answer": rag_chain_from_docs,
}

rag_chain_with_source.invoke("What is Task Decomposition")
```

### Expected behavior

It seems the `model.predict()` method is expecting a list. Could you fix this issue?
Serpapi google jobs tool
Labels: name: area: agent, name: auto:enhancement, name: size:L

Implemented tool for searching google jobs

<!-- Thank you for contributing to LangChain!
Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Issue: <Please write a comprehensive title after the 'Issue: ' prefix>
Labels: name: auto:bug, name: auto:question, name: area: models

### Issue you'd like to raise.

```# Load Tools
    tools = load_tools(["serpapi","langchain_experimental_python_repl"], llm=llm)
```
error 
```
Exception has occurred: ImportError
This tool has been moved to langchain experiment. This tool has access to a python REPL. For best practices make sure to sandbox this tool. Read  To keep using this code as is, install langchain experimental and update relevant imports replacing 'langchain' with 'langchain_experimental'
  File "/home/isayahc/projects/buy-bot/react_agent.py", line 49, in create_agent_executor
    tools = load_tools(["serpapi","python_repl"], llm=llm)
  File "/home/isayahc/projects/buy-bot/react_agent.py", line 88, in <module>
    agent_executor = create_agent_executor()
ImportError: This tool has been moved to langchain experiment. This tool has access to a python REPL. For best practices make sure to sandbox this tool. Read h To keep using this code as is, install langchain experimental and update relevant imports replacing 'langchain' with 'langchain_experimental'
```
how do i fix this?


### Suggestion:

_No response_
Issue: Load tools from experimental langchain module
Labels: name: area: agent, name: auto:bug

### Issue you'd like to raise.

Exception has occurred: ImportError
This tool has been moved to langchain experiment. This tool has access to a python REPL. For best practices make sure to sandbox this tool. Read https://github.com/langchain-ai/langchain/blob/master/SECURITY.md To keep using this code as is, install langchain experimental and update relevant imports replacing 'langchain' with 'langchain_experimental'
File "/home/isayahc/projects/buy-bot/react_agent.py", line 43, in create_agent_executor
tools = load_tools(["serpapi","python_repl"], llm=llm)
File "/home/isayahc/projects/buy-bot/react_agent.py", line 81, in
agent_executor = create_agent_executor()
ImportError: This tool has been moved to langchain experiment. This tool has access to a python REPL. For best practices make sure to sandbox this tool. Read https://github.com/langchain-ai/langchain/blob/master/SECURITY.md To keep using this code as is, install langchain experimental and update relevant imports replacing 'langchain' with 'langchain_experimental'


I am trying to use the python repl, and load it to my tools

### Suggestion:

_No response_
Issue: what string works for experimental tool
Labels: name: area: agent, name: auto:bug

### Issue you'd like to raise.

Exception has occurred: ImportError
This tool has been moved to langchain experiment. This tool has access to a python REPL. For best practices make sure to sandbox this tool. Read https://github.com/langchain-ai/langchain/blob/master/SECURITY.md To keep using this code as is, install langchain experimental and update relevant imports replacing 'langchain' with 'langchain_experimental'
  File "/home/isayahc/projects/buy-bot/react_agent.py", line 43, in create_agent_executor
    tools = load_tools(["serpapi","python_repl"], llm=llm)
  File "/home/isayahc/projects/buy-bot/react_agent.py", line 81, in <module>
    agent_executor = create_agent_executor()
ImportError: This tool has been moved to langchain experiment. This tool has access to a python REPL. For best practices make sure to sandbox this tool. Read https://github.com/langchain-ai/langchain/blob/master/SECURITY.md To keep using this code as is, install langchain experimental and update relevant imports replacing 'langchain' with 'langchain_experimental'

### Suggestion:

_No response_
Migrating Langchain's openai-python usage to v1.0.0
Labels: name: auto:improvement, name: area: models, name: size:L

  - **Description:** 
    - Converting Langchain to support openai-python v1.0.0 as per the [Official OpenAI v1.0.0 Migration Guide](https://github.com/openai/openai-python/discussions/742) and the [OpenAI v1.0.0 Migration Guide by Microsoft](https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/migration?tabs=python%2Cdalle-fix)
  - **TODO:**
    - [x] Run `openai migrate `
    - [x] Audit changes
      - [x] `libs/langchain/chains/moderation.py`
      - [x] `libs/langchain/chat_models/jinachat.py`
      - [x] `libs/langchain/chat_models/openai.py`
      - [x] `libs/langchain/document_loaders/parsrs/audio.py`
      - [x] `libs/langchain/embeddings/localai.py`
      - [x] `libs/langchain/embeddings/openai.py`
      - [x] `libs/langchain/llms/gooseai.py`
      - [x] `libs/langchain/llms/openai.py`
      - [x] `libs/langchain/utilities/dalle_image_generator.py`
      - [x] `tests/integration_tests/adapters/test_openai.py`
      - [x] `tests/integration_tests/embeddings/test_azure_openai.py`
      - [x] `tests/integration_tests/embeddings/test_openai.py`
      - [x] `tests/integration_tests/unit_tests/llms/test_openai.py`

  - **Issue:** #11755 
  - **Dependencies:** [openai-python v1.0.0](https://github.com/openai/openai-python) 
  - **Tag maintainer:** @baskaryan @eyurtsev @hwchase17
  - **Twitter handle:** [@Joshua_Concon](https://twitter.com/joshua_concon)
bump 0.0.339rc3
Labels: name: auto:nit, name: size:S

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Reduce the number of tokens required to describe a Cypher/Neo4j schema
Labels: name: lgtm, name: auto:improvement, name: size:L

Instead of using JSON-like syntax to describe node and relationship properties we changed to a shorter and more concise schema description

Old:

```
        Node properties are the following:
        [{'properties': [{'property': 'name', 'type': 'STRING'}], 'labels': 'Movie'}, {'properties': [{'property': 'name', 'type': 'STRING'}], 'labels': 'Actor'}]
        Relationship properties are the following:
        []
        The relationships are the following:
        ['(:Actor)-[:ACTED_IN]->(:Movie)']
```

New:

```
Node properties are the following:
Movie {name: STRING},Actor {name: STRING}
Relationship properties are the following:

The relationships are the following:
(:Actor)-[:ACTED_IN]->(:Movie)
```
Allow users to pass local embeddings to Weaviate Hybrid Search Retriever #5539
Labels: name: lgtm, name: integration: weaviate, name: area: embeddings, name: auto:improvement, name: size:S

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
#5539
🎞️ add json format to OllamaCommon class
Labels: name: auto:improvement, name: area: models, name: size:XS

The Ollama Client now allows a pure [json mode](https://github.com/jmorganca/ollama/blob/main/docs/api.md#request-json-mode) which I just added to the OllamaCommon class.

This allows using it like in this example:
<img width="876" alt="Screenshot 2023-11-25 at 14 11 19" src="https://github.com/langchain-ai/langchain/assets/34897716/1e47584e-58e0-497d-806d-00ef8e3ea6a5">
create_extraction_chain does not work with other LLMs?i try with llama_api
Labels: name: auto:bug, name: area: models

### System Info

Python 3.9.18


### Who can help?

_No response_

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [x] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Hi i try this example, but doesnt work
```
from llamaapi import LlamaAPI
from langchain.chains import create_extraction_chain
llama = LlamaAPI("My Api KEy")

from langchain_experimental.llms import ChatLlamaAPI
model = ChatLlamaAPI(client=llama)
schema = {
    "properties": {
        "name": {"type": "string"},
        "height": {"type": "integer"},
        "hair_color": {"type": "string"},
    },
    "required": ["name", "height"],
}
inp = """
Alex is 5 feet tall. Claudia is 1 feet taller Alex and jumps higher than him. Claudia is a brunette and Alex is blonde.
        """

chain = create_extraction_chain(schema, model)
chain.run(inp)
```


  File "C:\mini\envs\py39\lib\site-packages\langchain\output_parsers\openai_func
tions.py", line 136, in parse_result
    return res.get(self.key_name) if partial else res[self.key_name]
TypeError: string indices must be integers

### Expected behavior

Extraction from text