delayedkarma/langchain-issues huggingface dataset:
AttributeError: 'LLModel' object has no attribute 'model_type' (gpt4all)

### System Info

run on docker image with python:3.11.3-bullseye in MAC m1



### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

My docker image 
```

FROM python:3.11.3-bullseye
WORKDIR /src
COPY src /src

RUN python -m pip install --upgrade pip
RUN apt-get update -y
RUN apt install cmake -y
RUN git clone --recurse-submodules https://github.com/nomic-ai/gpt4all 
RUN cd gpt4all/gpt4all-backend/ && mkdir build && cd build && cmake .. && cmake --build . --parallel
RUN cd gpt4all/gpt4all-bindings/python && pip3 install -e .
RUN pip install -r requirements.txt
RUN chmod +x app/start_app.sh
EXPOSE 8501
ENTRYPOINT ["/bin/bash"]
CMD ["app/start_app.sh"]
```


where star_app.sh is run python file that have this line

`llm = GPT4All(model=llm_path, backend='gptj', verbose=True, streaming=True, n_threads=os.cpu_count(),temp=temp)`

llm_path is path of gpt4all model

### Expected behavior

Got this error when try to use gpt4all

```
AttributeError: 'LLModel' object has no attribute 'model_type'
Traceback:
File "/src/app/utils.py", line 20, in get_chain
    llm = GPT4All(model=llm_path, backend='gptj', verbose=True, streaming=True, n_threads=os.cpu_count(),temp=temp)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/pydantic/main.py", line 339, in __init__
    values, fields_set, validation_error = validate_model(__pydantic_self__.__class__, data)
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/pydantic/main.py", line 1102, in validate_model
    values = validator(cls_, values)
             ^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/langchain/llms/gpt4all.py", line 156, in validate_environment
    values["backend"] = values["client"].model.model_type
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
```
This is a fix for #5475

<!--
Currently in PGVector, the from_documents method will add the embeddings and then return the instance of the store. We should have a method that just returns the store object without inserting embeddings, This is useful when I already have a loaded vector store and I just need the instance of the store. It will be like the below code without the store.add_texts

store = cls(
            connection_string=connection_string,
            collection_name=collection_name,
            embedding_function=embedding,
            distance_strategy=distance_strategy,
            pre_delete_collection=pre_delete_collection,
        )

        store.add_texts(texts=texts, metadatas=metadatas, ids=ids, **kwargs)
        return store

After you're done, someone will review your PR. They may suggest improvements. If no one reviews your PR within a few days, feel free to @-mention the same people again, as notifications can get lost.

Finally, we'd love to show appreciation for your contribution - if you'd like us to shout you out on Twitter, please also include your handle!
-->

<!-- Remove if not applicable -->

Fixes # 5475

#### Before submitting

<!-- If you're adding a new integration, please include:

1. a test for the integration - favor unit tests that does not rely on network access.
2. an example notebook showing its use


See contribution guidelines for more information on how to write tests, lint
etc:

https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
-->

#### Who can review?

@hwchase17
@dev2049

Tag maintainers/contributors who might be interested:

<!-- For a quicker response, figure out the right person to tag with @

  @hwchase17 - project lead

  Tracing / Callbacks
  - @agola11

  Async
  - @agola11

  DataLoaders
  - @eyurtsev

  Models
  - @hwchase17
  - @agola11

  Agents / Tools / Toolkits
  - @vowelparrot

  VectorStores / Retrievers / Memory
  - @dev2049

 -->
Bug in langchain.llms.Modal which raised ValueError("LangChain requires 'prompt' key in response.")

### System Info

LangChain: 0.0.189
Platform: Ubuntu 22.04.2 LTS
Python: 3.10.10


### Who can help?

@hwchase17
@agola11

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

You can re-produce the error using the notebook at https://python.langchain.com/en/latest/modules/models/llms/integrations/modal.html

The following is the response json from my Modal endpoint:
```javascript
{'prompt': 'Artificial Intelligence (AI) is the study of artificial intelligence. Artificial intelligence is the study of artificial intelligence. So, the final answer is AI.'}
```
It still raise the ValueError("LangChain requires 'prompt' key in response.").

After look into the code, I think there is a bug at https://github.com/hwchase17/langchain/blob/master/langchain/llms/modal.py#L90, the correct syntax should be:
```python
if "prompt" in response.json():
```

### Expected behavior

No value error should be raised if the response json format is correct.
fix: correct momento chat history notebook typo and title

This PR corrects a minor typo in the Momento chat message history notebook and also expands the title from "Momento" to "Momento Chat History", inline with other chat history storage providers.


#### Before submitting

<!-- If you're adding a new integration, please include:

1. a test for the integration - favor unit tests that does not rely on network access.
2. an example notebook showing its use


See contribution guidelines for more information on how to write tests, lint
etc:

https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
-->

#### Who can review?

cc @dev2049 who reviewed the original integration
I want ssh tunnel support in SQLDatabaseChain

### Feature request

Due to security reasons, it is not always possible to connect directly to the DB. I want SQLDatabaseChain to support ssh tunnel because I have to connect to a specific kind of stepping stone server before I can connect to the DB.

If there is already a way to accomplish this, I apologize.

### Motivation

I want to use llm with all our own data to create an innovative experience, and SQLDatabaseChain makes that possible.

### Your contribution

I can contribute with testing. I do not yet have a good understanding of langchain's source code structure and inner workings, so coding contributions would not be appropriate.
Add knn and query search field options to ElasticKnnSearch
Labels: name: lgtm

in the `ElasticKnnSearch` class added 2 arguments that were not exposed properly

`knn_search` added:
- `vector_query_field: Optional[str] = 'vector'`
-- vector_query_field: Field name to use in knn search if not default 'vector'

`knn_hybrid_search` added:
- `vector_query_field: Optional[str] = 'vector'`
-- vector_query_field: Field name to use in knn search if not default 'vector'
- `query_field: Optional[str] = 'text'`
-- query_field: Field name to use in search if not default 'text'



Fixes # https://github.com/hwchase17/langchain/issues/5633


cc: @dev2049 @hwchase17
Replace exec with wasm_exec

This replaces the usage of the insecure `exec` function with a more secure library called [wasm_exec](https://github.com/Jflick58/wasm_exec) TL;DR: Use `chroot` jails, `wasmtime` and a standalone Python3.11 WASM interpreter to safely execute arbitrary code

Some questions before I officially submit this pr:

- **How critical is it that we support the execution of code containing 3rd-party dependencies?** The team behind the wasm interpreter I vendor with the library is working on porting `numpy` along with some of the c libs needed to enable broader package support. 
       
  - *If it is critical, then how critical is it to support Python versions below 3.11?* Right now the interpreter does support the use of an existing venv for accessing installed 3rd party deps, but the interpreter is only distributed in 3.11 which means I would either a) have to only support 3.11 b) have a 3rd split in the PythonRepl code base (base, AST, Wasm) or b) try to write some hacky and unpythonic code to try to dynamically create a new 3.11-based venv off of the currently active Python environment. 

   - *If it is not critical* then is it okay if I refactor the test cases to avoid the tests containing `numpy` and `pandas` code? Should we also update the docs to reflect that limitation?

I also strongly welcome feedback on the wasm_exec library if there are implementation ideas that make it more functional. 

Fixes #1026 
Fixes #5294 
Fixes #5388 
Fixes https://nvd.nist.gov/vuln/detail/CVE-2023-29374

#### Who can review?

  - @hwchase17 
  - @vowelparrot
Retitles Bedrock doc to appear in correct alphabetical order in site nav

<!--
Thank you for contributing to LangChain! Your PR will appear in our release under the title you set. Please make sure it highlights your valuable contribution.

Replace this with a description of the change, the issue it fixes (if applicable), and relevant context. List any dependencies required for this change.

After you're done, someone will review your PR. They may suggest improvements. If no one reviews your PR within a few days, feel free to @-mention the same people again, as notifications can get lost.

Finally, we'd love to show appreciation for your contribution - if you'd like us to shout you out on Twitter, please also include your handle!
-->

<!-- Remove if not applicable -->

Fixes #5638. Retitles "Amazon Bedrock" page to "Bedrock" so that the Integrations section of the left nav is properly sorted in alphabetical order.

#### Who can review?

Tag maintainers/contributors who might be interested:

<!-- For a quicker response, figure out the right person to tag with @

  @hwchase17 - project lead

  Tracing / Callbacks
  - @agola11

  Async
  - @agola11

  DataLoaders
  - @eyurtsev

  Models
  - @hwchase17
  - @agola11

  Agents / Tools / Toolkits
  - @vowelparrot

  VectorStores / Retrievers / Memory
  - @dev2049

 -->
DOC: "Amazon Bedrock" is not sorted in Integrations section of nav

### Issue with current documentation:

In the left nav of the docs, "Amazon Bedrock" is alphabetized after "Beam integration for langchain" and before "Cerebrium AI", not with the rest of the A-named integrations.

<img width="254" alt="image" src="https://github.com/hwchase17/langchain/assets/93281816/20836ca0-3946-4614-8b44-4dcf67e27f7e">


### Idea or request for content:

Retitle the page to "Bedrock" so that its URL remains unchanged and the nav is properly sorted.
In memory audio blob loading and splitting

This PR creates a `BaseBlobSplitter` and a `BaseBlobTransformer`, which can be used to split and transform blobs. It creates an instance of the `BaseBlobTransformer` called `YoutubeToAudioTransformer`, which can be used to download a YouTube URL to audio. In addition, it creates an instance of `BaseBlobSplitter` called `AudioSplitter`, which can be used to split an audio file into smaller blobs. Both are useful for an end-to-end workflow that combines (1) YouTube link to Audio, (2) Audio to Audio splits, and (3) then use `OpenAIWhisperParser` [here](https://github.com/hwchase17/langchain/pull/5580) to create `Documents` from the splits.
Allow query field to be passed in ElasticKnnSearch.knnSearch

### System Info

The [_default_knn_query](https://github.com/jeffvestal/langchain/blob/10f34bf62e2d53f0b1a7b15ba21c2328b64862cd/langchain/vectorstores/elastic_vector_search.py#L402) takes `field` arg to specify the field to use when performing knn and hybrid search.

both [knn_search](https://github.com/jeffvestal/langchain/blob/10f34bf62e2d53f0b1a7b15ba21c2328b64862cd/langchain/vectorstores/elastic_vector_search.py#L475) and [hybrid_search](https://github.com/jeffvestal/langchain/blob/10f34bf62e2d53f0b1a7b15ba21c2328b64862cd/langchain/vectorstores/elastic_vector_search.py#L537) need to allow that to be passed in as an arg and pass it to `_default_knn_query`


I'll make the update

cc: @hw

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Try to search a field other than `vector`, can't do it 

### Expected behavior

Pass a `qeuery_field` value to search over a different field name other than the default `vector`
Make embedding object optional for ElasticKnnSearch

### Feature request

Make `embedding` an [optional arg](https://github.com/jeffvestal/langchain/blob/10f34bf62e2d53f0b1a7b15ba21c2328b64862cd/langchain/vectorstores/elastic_vector_search.py#L332) when creating `ElasticKnnSearch` class

### Motivation

If a user is only going to run knn/hybrid search AND use the `query_vector_builder` having an embedding object is not needed as ES will generate the embedding during search



### Your contribution

I will make the changes
ConversationalRetrievalChain unable to use ConversationBufferMemory when return_source_documents=True

### System Info

LangChain 0.0.188
Python 3.8


### Who can help?

@hwchase17 @agola11 

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

memory = ConversationBufferMemory(memory_key="chat_history", return_messages=True)
qa = ConversationalRetrievalChain.from_llm(OpenAI(temperature =0), vectorstore.as_retriever(), memory=memory, return_source_documents=True)
qa({"question": query})
#ValueError" One output key expected, got dict_keys(['answer', 'source_documents'])

qa = ConversationalRetrievalChain.from_llm(OpenAI(temperature =0), vectorstore.as_retriever(), return_source_documents=True)
qa({"question": query, "chat_history": []})
#...(runs as expected)



### Expected behavior

ConversationalRetrievalChain does not work with ConversationBufferMemory and return_source_documents=True.   

ValueError: One output key expected, got dict_keys(['answer', 'source_documents'])
Change Anthropic LLM and Chat to be LangFlow friendly
Labels: name: auto:improvement

Change the Anthropic LLM and Chat classes to use model_name and default parameters the same way others do like OpenAI, Cohere, etc.  Added an alias for model to model_name for backwards compatibility.  By following the common pattern this will enable the Anthropic PR I'm working on for LangFlow (https://github.com/jimwhite/langflow/tree/anthropic) to work as expected.

#### Who can review?

Tag maintainers/contributors who might be interested:

  Models
  - @hwchase17
  - @agola11
DOC: repetitive parts in Modules pages

### Issue with current documentation:

Pages in Modules: Models, Prompts, Memory, ... 
They all have repeated parts. See a picture. 

![image](https://github.com/hwchase17/langchain/assets/2256422/05514908-87b5-453a-9795-8304612f42bf)


### Idea or request for content:
The whole "Go Deeper" section can be removed and instead, the links from removed items added to the above items. For example "Prompt Templates" link is added to the "LLM Prompt Templates" in the above text. Etc.
This significantly decreases the size of the page and improves user experience. No more repetitive items.
_No response_
Refactor SQL dialect prompts to eliminate code duplication.
Labels: name: auto:refactor

Refactored code to eliminate multiple copies of the query prompt (prompt per dialect). Now, there is a single copy of the prompt and all placeholders are encapsulated in **SqlQueryDialect** enumeration class. This should make it much easier and less error to add new SQL dialects and/or extend prompt template (e.g. adding new hints for the LLM). Code changes have been formatted, linted, and tested.

#### Who can review?
@hwchase17
cannot import name 'FigmaFileLoader'

### System Info

langchain==0.0.189
os:windows11
python=3.10.11



### Who can help?

_No response_

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [X] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

from langchain.document_loaders import FigmaFileLoader

### Expected behavior

expected:
load the module

error:
ImportError: cannot import name 'FigmaFileLoader' from 'langchain.document_loaders' (C:\Users\xxx\AppData\Local\miniconda3\envs\xxx\lib\site-packages\langchain\document_loaders\__init__.py)

comments:
checked the langchain\document_loaders\__init__.py and there is no reference to FigmaFileLoader
[RFC] Run Evaluator Chains

Will try to land https://github.com/hwchase17/langchain/pull/5750 first
Pros:
- Fits many but not all feedback applications.
- Maintains a flexible base with configurable implementations.
- Doesn't implement THAT many new classes for the core logic

Cons:
- doesn't handle pairwise/group evals
- trajectory evals are a bit hacky ( you have to remember to turn on when tracing rather than building it from the run object)
- possibly too opinionated by forcing an input mapper and output parser?
- Doesn't (yet) allow easy composability of the reasoning step (most do COT built in)
- Most of the scores are binary - no nuance or more advanced logic. Doesn't (yet) decouple the scoring logic from the core of the prompt
- Not sure if the creation functions are actually much better than creating a child and making a complicated constructor

Example making 5 different evaluators:

<img width="971" alt="image" src="https://github.com/hwchase17/langchain/assets/130414180/901d8d78-5dee-4f43-acd6-7b0589182cb7">
feat: add `UnstructuredExcelLoader` for `.xlsx` and `.xls` files

# Unstructured Excel Loader

Adds an `UnstructuredExcelLoader` class for `.xlsx` and `.xls` files. Works with `unstructured>=0.6.7`. A plain text representation of the Excel file will be available under the `page_content` attribute in the doc. If you use the loader in `"elements"` mode, an HTML representation of the Excel file will be available under the `text_as_html` metadata key. Each sheet in the Excel document is its own document.

### Testing

```python
from langchain.document_loaders import UnstructuredExcelLoader

loader = UnstructuredExcelLoader(
    "example_data/stanley-cups.xlsx",
    mode="elements"
)
docs = loader.load()
```

## Who can review?

@hwchase17
@eyurtsev
Opensearch for Indexing and Retrieval 

### System Info

I am using Python for INdexing Document to OpenELastic search 


Python for Indexing---->
 documents = loader.load()
 text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)
 docs = text_splitter.split_documents(documents)
vs = OpenSearchVectorSearch.from_documents(
            docs, 
            embeddings,
            opensearch_url="https://search-bterai-opensearch-gmtgf5ukflbg77xlwenvwko4gm.us-west-2.es.amazonaws.com", 
            http_auth=("XXXX", "XXXX"),     
            use_ssl = False,
            verify_certs = False,
            ssl_assert_hostname = False,
            ssl_show_warn = False,
            index_name="test_sidsfarm")

NodejS for Querying

const client = new Client({
    nodes: [process.env.OPENSEARCH_URL ?? "https://X:X@search-bterai-opensearch-gmtgf5ukflbg77xlwenvwko4gm.us-west-2.es.amazonaws.com"],        
});

const vectorStore = new OpenSearchVectorStore(new OpenAIEmbeddings(), {
    client,
    indexName: "test_sidsfarm"
});
const chain = VectorDBQAChain.fromLLM(model, vectorStore, {
    returnSourceDocuments: true,
  });

res = await chain.call({ query: input });


Its Giving me error below , Please help!!

failed to create query: Field 'embedding' is not knn_vector type.

### Who can help?

@hwchase17 

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

ALready mentioned steps above

### Expected behavior

It should work out of the box irrespective of platforms