delayedkarma/langchain-issues huggingface dataset:
test_stream_log_retriever Unit Test + Tool names fix
Labels: name: area: agent, name: auto:improvement

## Description



| Tool         | Original Tool Name       |
|-----------------------------|---------------------------|
| open-meteo-api              | Open Meteo API            |
| news-api                    | News API                  |
| tmdb-api                    | TMDB API                  |
| podcast-api                 | Podcast API               |
| golden_query                | Golden Query              |
| dall-e-image-generator      | Dall-E Image Generator    |
| twilio                      | Text Message              |
| searx_search_results        | Searx Search Results      |
| dataforseo                  | DataForSeo Results JSON   |

When using these tools through `load_tools`, I encountered the following validation error:

```console
openai.error.InvalidRequestError: 'TMDB API' does not match '^[a-zA-Z0-9_-]{1,64}$' - 'functions.0.name'
```

In order to avoid this error, I replaced spaces with hyphens in the tool names:

| Tool           | Corrected Tool Name       |
|-----------------------------|---------------------------|
| open-meteo-api              | Open-Meteo-API            |
| news-api                    | News-API                  |
| tmdb-api                    | TMDB-API                  |
| podcast-api                 | Podcast-API               |
| golden_query                | Golden-Query              |
| dall-e-image-generator      | Dall-E-Image-Generator    |
| twilio                      | Text-Message              |
| searx_search_results        | Searx-Search-Results      |
| dataforseo                  | DataForSeo-Results-JSON   |

This correction resolved the validation error.

Additionally, a unit test, `tests/unit_tests/schema/runnable/test_runnable.py::test_stream_log_retriever`, was failing at random. Upon further investigation, I confirmed that the failure was not related to the above-mentioned changes. The `stream_log` variable was generating the order of logs in two ways at random The reason for this behavior is unclear, but in the assertion, I included both possible orders to account for this variability.
Issue: : Issue with HuggingFacePipeline while trying to use a Falcon model for chat with text files
Labels: name: auto:bug, name: area: models

### Issue you'd like to raise.

I'm encountering an error while trying to use a Falcon model to chat with a bunch of text files. I've downloaded the model locally on my Google Drive, and everything seems fine up to this point. However, when I try to define the LLM model in Langchain using the following code:

```
from langchain.llms import HuggingFacePipeline

llm = HuggingFacePipeline.from_model_id(
    model_id='/content/gdrive/MyDrive/Colab Notebooks/falcon model/898df1396f35e447d5fe44e0a3ccaaaa69f30d36',
    task="text-generation",
    model_kwargs={"temperature": 0, "max_length": 64},
)

```
I encounter the following error:

```
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-31-902f2455d555> in <cell line: 3>()
      1 from langchain.llms import HuggingFacePipeline
      2 
----> 3 llm = HuggingFacePipeline.from_model_id(
      4     model_id='/content/gdrive/MyDrive/Colab Notebooks/falcon model/898df1396f35e447d5fe44e0a3ccaaaa69f30d36',
      5     task="text-generation",
AttributeError: 'FalconForCausalLM' object has no attribute 'is_quantized'
```


the whole code 
```

device = torch.device('cuda')
checkpoint = "tiiuae/falcon-7b"
print(f"Checkpoint path: {checkpoint}")  # Add this line for debugging
tokenizer = AutoTokenizer.from_pretrained(pretrained_model_name_or_path="/content/gdrive/MyDrive/Colab Notebooks/falcon model/898df1396f35e447d5fe44e0a3ccaaaa69f30d36")
base_model = AutoModel.from_pretrained(
    pretrained_model_name_or_path='/content/gdrive/MyDrive/Colab Notebooks/falcon model/898df1396f35e447d5fe44e0a3ccaaaa69f30d36',
    torch_dtype=torch.float32
)



loader = DirectoryLoader('/content/gdrive/MyDrive/Colab Notebooks/chatbot 3.0/', glob="./*.txt", loader_cls=TextLoader)
documents = loader.load()
text_splitter = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=500)
texts = text_splitter.split_documents(documents)
embeddings = HuggingFaceEmbeddings(cache_folder='/content/gdrive/MyDrive/Colab Notebooks/falcon model/898df1396f35e447d5fe44e0a3ccaaaa69f30d36',)


from langchain.llms import HuggingFacePipeline

llm = HuggingFacePipeline.from_model_id(
    model_id='/content/gdrive/MyDrive/Colab Notebooks/falcon model/898df1396f35e447d5fe44e0a3ccaaaa69f30d36',
    task="text-generation",
    model_kwargs={"temperature": 0, "max_length": 64},
)

```


### Suggestion:

_No response_
DOC: LangChain Cookbook RAG typo
Labels: name: area: doc loader, name: auto:documentation

### Issue with current documentation:

[Link](https://python.langchain.com/docs/expression_language/cookbook/retrieval#with-memory-and-returning-source-documents)

Hi, I've been exploring the Cookbook RAG (link provided above) and noticed that `expanded_memory` is not defined anywhere in the code snippet(**With Memory and returning source documents**). I was wondering if a portion of the code is missing or if it's just a typo. Thank you in advance for looking over my issue!

### Idea or request for content:

_No response_
Issue: unit tests, a quick question.
Labels: name: auto:question

### Issue you'd like to raise.

Sorry to raise this as an issue, but I have a quick question. I am preparing come code for `langchain` and noticed that most of the code I wish to build on lacks unit tests. I was considering first submitting a PR of purely unit tests for the existing code I wish to use in order to keep subsequent PRs more digestible. _Is submitting a PR of just unit tests acceptable?_

### Suggestion:

_No response_
Previous version was not respecting the max_token_limit field.
Labels: name: area: memory, name: auto:bug

**Description:** The new version now pops the buffer when the messages are requested, keeping the original messages safe inside the chat history.

The scenario where the previous version did not work for me was with DynamoDB memory implementation, since I was loading the chat history on initialization only and never called the save_context method before my first llm call.
This version now also keeps the original chat history intact, which was desirable for my case.
Add support for AWS Kendra `ScoreConfidence` filter. 
Labels: name: auto:improvement, name: integration: aws

### Feature request

- If user wants to get content from document which are having confidence HIGH or HIGHEST then with help of this feature Kendra will return only more relevance records. 

### Motivation

- Sometimes kendra return content with LOW `ScoreConfidence` which may lead to inaccurate results.

### Your contribution

- I've checked the `_get_relevant_documents` method from `AmazonKendraRetriever` class but it is only returning top_k docs not actually filtering based on `ScoreConfidence`.
- With the help of `boto3` we can achieve that. 
- I would like to do more research and open PR for it.
`_dalle_image_url` returns list of urls if n>1
Labels: name: lgtm, name: auto:improvement

  - **Description:** Updated the `_dalle_image_url` method to return a list of URLs if self.n>1, 
  - **Issue:** #10691,
  - **Dependencies:** unsure,
  - **Tag maintainer:** @eyurtsev,
  - **Twitter handle:** @silvhua

I am new to development and do not know how to do any of the make format, make lint and make test steps or how to test the code locally. I would appreciate help with that.
feat: add NASA API tool
Labels: name: auto:enhancement

### Feature request

The feature proposal is to integrate a new Langchain tool that would provide support for queries on [https://api.nasa.gov/](NASA APIs)

Here is the list of NASA APIs that will be integrated into the new tool:
- APOD
- Asteroids NeoWs
- Earth
- EONET
- EPIC
- GeneLab
- Insight
- Mars Rover Photos
- NASA Image and Video Library
- TechTransfer
- TLE API
- Vesta/Moon/Mars Trek WMTS

### Motivation

Currently, Langchain does not provide access to a large data pool of reliable information related to astronomy. The integration of a NASA API tool would allow users to retrieve updated and accurate data related to satellite imagery, mission data and other space-related information. In cases where accuracy of data is critical, the NASA API tool may also be preferred over its GPT-generated counterpart.

### Your contribution

I am part of a group of 3 other students that will be able to submit a PR for this issue sometime in November.
Create pattern_using_functions.c
Labels: name: auto:documentation

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
fix recent docs integrations file loc
Labels: name: area: doc loader, name: auto:documentation, name: integration: elasticsearch

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Issue: <add_texts method in AzureSearch class throwing a metadata error>
Labels: name: area: vector store, name: auto:bug

### Issue you'd like to raise.

my Azure search index has 4 fields
id, user_id, content, content_vector.

I am trying to add a conversation to the azure search vector store for a user_id ="msunkara"

The code is given below:

    memory_vector_store: AzureSearch = AzureSearch(
        azure_search_endpoint=vector_store_address,
        azure_search_key=vector_store_password,
        index_name=memory_index_name,
        embedding_function=embeddings.embed_query)
     
    history = []
    user_id = "msunkara"
    session_id = "1"

query = Hi
res.content = Hi, How can I assist you today?

    history.append("User: "+ query + " Assistant: " + res.content)

    memory_id = session_id + user_id
    metadata = [{"user_id": user_id}]
    memory_vector_store.add_texts(history,keys = [memory_id],metadatas=metadata)

The above code is throwing the below error:
Result: Failure Exception: HttpResponseError: () The request is invalid. Details: The property 'metadata' does not exist on type 'search.documentFields'. Make sure to only use property names that are defined by the type. 


Help me solve this problem

### Suggestion:

_No response_
LLMRouterChain should easily support multiple inputs
Labels: name: auto:improvement, name: area: models

### Feature request

As it stands, `LLMRouterChain` basically only supports a single input. Here is the template:
https://github.com/langchain-ai/langchain/blob/v0.0.314/libs/langchain/langchain/chains/router/multi_prompt_prompt.py

While it is currently supported to define multiple input variables on a `PromptTemplate`, the `LLMRouterChain` basically doesn't support it because of the way the output from the LLM call is handled.

```
{{{{
    "destination": string \\ name of the prompt to use or "DEFAULT"
    "next_inputs": string \\ a potentially modified version of the original input
}}}}
```

With a lot of massaging, you can get to a place where an LLM causes `next_inputs` to be a comma-separated dict-like object but, down the chain (pun intended), you end up with an explosion because the way inputs are passed to chains doesn't handle this.

Overall, this implementation is odd. It expects the LLM to massage the original input(s) and then wants to use that output directly in future things.

Wouldn't it make more sense to simply ask the LLM to reason out which destination to send to (alone) and then take the original inputs and pass them along?

This current implementation is combining two things into one:
* a routing decision
* paraphrasing user input

The whole purpose of Langchain is to chain things, so why wouldn't you have "paraphrase/rephrase user input" as a chain step, instead of smushing it into the router implementation?

### Motivation

I have a use case where various inputs are collected separately and need to be maintained separately for a custom-tuned LLM later on. This caused me to attempt to use `LLMRouterChain` with multiple inputs, whereby I hit the roadblock described above.

It doesn't seem that wild to want to be able to make a routing decision based on input A, input B, or multiple inputs, and subsequently send all of those inputs to different templates later.

### Your contribution

I don't really have sufficient coding skills to help. I could maybe make some diagrams that help illustrate how this could work.
Video imagery to text (Closed Captioning)
Labels: name: auto:enhancement, name: area: models

### Feature request

Implement a feature using Langchain's image_captions.py and audio_speech_to_text.py to produce .srt files. This system will provide both subtitles and visual scene descriptions, essentially creating closed captioning. 

### Motivation

Current closed captioning primarily captures spoken content, omitting crucial visual elements of videos. This lack of comprehensive description hinders a full understanding, especially for the hearing-impaired community. Our goal is to elevate video accessibility by integrating both visual and auditory components in closed captioning, ensuring a more inclusive viewing experience.

### Your contribution

We are students at the University of Toronto, interested in contributing to Langchain for a school project. We would like to submit a PR that implements this feature sometime in November.
adding parameter for changing the language in SpacyEmbeddings
Labels: name: area: embeddings, name: auto:improvement, name: size:M

  - **Description:** Added the parameter for a possibility to change a language model in SpacyEmbeddings. The default value is still the same: "en_core_web_sm", so it shouldn't affect a code which previously did not specify this parameter, but it is not hard-coded anymore and easy to change in case you want to use it with other languages or models.
  
  - **Issue:** At Barcelona Supercomputing Center in Aina project (https://github.com/projecte-aina), a project for Catalan Language Models and Resources, we would like to use Langchain for one of our current projects and we would like to comment that Langchain, while being a very powerful and useful open-source tool, is pretty much focused on English language. We would like to contribute to make it a bit more adaptable for using with other languages.
  
  - **Dependencies:**  This change requires the Spacy library and a language model, specified in the model parameter.
  - **Tag maintainer:** @dev2049
  - **Twitter handle:** @projecte_aina
Retrying langchain.embeddings.openai.embed_with_retry.<locals>._embed_with_retry in 4.0 seconds as it raised APIConnectionError: Error communicating with OpenAI: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')).
Labels: name: area: embeddings, name: auto:bug

### System Info

llm = ChatOpenAI(model_name="gpt-4", temperature=0.5, request_timeout=300)

qa_chain = ConversationalRetrievalChain.from_llm(
    verbose=True,
    llm=llm,
    retriever=retriever,
    return_source_documents=True,
    combine_docs_chain_kwargs={"prompt": prompt},
    rephrase_question=False,
)

given code I get error:
Retrying langchain.embeddings.openai.embed_with_retry.<locals>._embed_with_retry in 4.0 seconds as it raised APIConnectionError: Error communicating with OpenAI: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')).

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [X] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

llm = ChatOpenAI(model_name="gpt-4", temperature=0.5, request_timeout=300)

qa_chain = ConversationalRetrievalChain.from_llm(
    verbose=True,
    llm=llm,
    retriever=retriever,
    return_source_documents=True,
    combine_docs_chain_kwargs={"prompt": prompt},
    rephrase_question=False,
)

### Expected behavior

successfully call OpenAI embedding api
Recursive loader user agent header
Labels: name: area: doc loader, name: auto:improvement

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** Default User-Agent added to RecursiveUrlLoader headers, 
  - **Issue:**  Fixes the following issue raised. https://github.com/langchain-ai/langchain/issues/11541 ,
  - **Dependencies:** Needed fake_useragent python library . pip install fake_useragent,
  - **Tag maintainer:** @baskaryan, @eyurtsev, @hwchase17,
Include information on the tools for creating gbnf grammar files in the llama-cpp notebook
Labels: name: lgtm, name: area: doc loader, name: auto:documentation

Hi,

I recently experimented with grammar-based sampling and discovered two methods for speeding up the creation of gbnf grammar files:
1. [Online grammar generator app](https://github.com/ggerganov/llama.cpp/discussions/2494) introduced [here](https://github.com/ggerganov/llama.cpp/discussions/2494)
2. [Script](https://github.com/ggerganov/llama.cpp/blob/master/examples/json-schema-to-grammar.py) for parsing json schema to gbnf grammar

I believe it is a good idea to include the information that leads to them in the `llama-cpp` notebook.

***

Codespell check fails but due to the unrelated script
Add a new vector store hippo to langchain
Labels: name: area: vector store, name: auto:enhancement

### Feature request

I would like to propose a new vector database to be included in the Langchain ecosystem to expand Langchain's influence. Below is the official link to our vector database.

https://www.transwarp.cn/starwarp


### Motivation

I hope that everyone can more conveniently use Hippo + Langchain to build AI Agents.

### Your contribution

Integrating the Hippo vector database into the Langchain ecosystem can expand Langchain's influence.
Use a less specific return type for | on Runnables
Labels: name: lgtm, name: auto:improvement

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Use conversation history with MultiRetrievalQAChain
Labels: name: auto:enhancement, name: area: models

### Feature request

It would be great to make it possible to use multiple retrievers, LLM, and conversation history together.

### Motivation

Currently, when using `MultiRetrievalQAChain.fromLLMAndRetrievers`, you can provide an LLM and multiple retrievers for context. If there was a possibility to add a conversation/chat history to this flow, it could help further understand the context, especially in cases where there was a previous conversation between a user and a human agent/bot.

### Your contribution

I'm not familiar with Python but could provide help with the TypeScript implementation on top of the base logic.
AttributeError: 'BloomForCausalLM' object has no attribute 'is_quantized'
Labels: name: auto:bug, name: area: models

### System Info

Google Colab
langchain==0.0.313
transformers==4.34.0
Python 3.10.12

[Dependencies](https://pastebin.com/raw/T1BqzVq1)

I am trying to download huggingface model and feed them to the chain. It was working prior to few weeks. I have tried to change model id to other models.

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction
```
!pip install langchain transformers

from langchain import HuggingFacePipeline
llm = HuggingFacePipeline.from_model_id(model_id = "bigscience/bloom-560m", task = "text-generation")
```
### Expected behavior

Successful execution
## Screenshot
![Screenshot_2023-10-13_09-37-20](https://github.com/langchain-ai/langchain/assets/86911396/51f46569-6bcc-4057-81a6-68eba060af50)
DOC:**Page Not Found**
Labels: name: auto:documentation, name: integration: aws

### Issue with current documentation:

From the page 
https://python.langchain.com/docs/integrations/platforms/aws

If you select  See a [usage example.](https://python.langchain.com/docs/integrations/llms/amazon_api_gateway_example) the following message is generated:

**Page Not Found**
We could not find what you were looking for.

Please contact the owner of the site that linked you to the original URL and let them know their link is broken.

https://python.langchain.com/docs/integrations/llms/amazon_api_gateway_example

### Idea or request for content:

_No response_