delayedkarma/langchain-issues huggingface dataset:
Retry limit is not respected
Labels: name: auto:bug, name: area: models

### System Info

Langchain==0.0305

Mac

python == 3.9.6

### Who can help?

@hwchase17  @agola11 

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

I have a simple web page summarize function basacilly followed example
```python
    llm = OpenAI(temperature=0)
    loader = WebBaseLoader(link)
    docs = loader.load()

    MAP_PROMPT = PromptTemplate(
        template="""Write a concise summary of the following web page(keep all identifiable information about the main entities):
    {text}

    CONCISE SUMMARY:
    """,
        input_variables=["text"],
    )
    REDUCE_PROMPT = PromptTemplate(
        template="""Based on the following text, write a concise summary(keep all identifiable information about the main entities)):
    {text}

    CONCISE SUMMARY:
    """,
        input_variables=["text"],
    )
    chain = load_summarize_chain(
        llm,
        chain_type="map_reduce",
        return_intermediate_steps=True,
        map_prompt=MAP_PROMPT,
        combine_prompt=REDUCE_PROMPT,
    )
    text_splitter = RecursiveCharacterTextSplitter()
    docs = text_splitter.split_documents(docs)
    if len(docs) == 0:
        return ""
    result = chain(
        text_splitter.split_documents(documents=docs),
        return_only_outputs=True,
    )
    logging.debug(result)
    return result["output_text"]
```
my program loaded in a pdf url link (https://www.fiserv.com/content/dam/fiserv-ent/final-files/marketing-collateral/case-studies/regions-bank-case-study-0623.pdf)

web based loader created 1 page document with a large number of tokens (13524539 characters)
I then started to get openai library error:
```
error_code=rate_limit_exceeded error_message='Rate limit reached for default-text-davinci-003 in organization org-a6hHivOPvKe9X60G8E6YqM5m on tokens_usage_based per min. Limit: 250000 / min. Current: 245411 / min. Contact us through our help center at help.openai.com if you continue to have issues.' error_param=None error_type=tokens_usage_based message='OpenAI API error received' stream_error=False
```

langchain kept retrying on this error definetly more than the default 6 times, I had `14` retry log from langchain in my terminal.

I think this is a quite critical bug, my OpenAI usage limit blew up the $120 limit under 10mins because of this.




### Expected behavior

retry limit should kick it to short circuit this situation or there should be some other method to prevent unexpected length of document being passed to openAI
Redis Vectorestore.add_documents() should use embed_documents() instead of  embed_query()
Labels: name: area: embeddings, name: area: vector store, name: auto:improvement, name: integration: redis

### Feature request

The Redis Vectorstore `add_documents()` method calls `add_texts()` which embeds documents one by one like:
```
embedding = (
                embeddings[i] if embeddings else self._embeddings.embed_query(text)
            )
```
The `add_documents()` method would seem to imply that it is a good way to jointly embed and upload a larger list of documents, but using an API call for each document as above is slow and prone to run into rate limits. 

Redis `add_documents()` could pass a kwarg `from_documents=True`to `add_texts()` which would change embedding to 

```
embedding = (
                embeddings[i] if embeddings else self._embeddings.embed_documents(documents)
            )
```


### Motivation

This would simplify the overall use of the Redis Vectorstore and would be more inline with how the example documents imply the add_documents() method should be used. Currently `add_documents()` is not suitable for something like a csv or a long list of smaller documents unless it is used like:
```
add_documents(
        documents=split_documents,
        embeddings=embedder.embed_documents(texts),
    )
```

### Your contribution

I could create a PR for this issue if it makes sense!
Rename RunnableMap to RunnableParallel
Labels: name: auto:refactor, name: area: models

- keep alias for RunnableMap
- update docs to use RunnableParallel and RunnablePassthrough.assign

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Issue: load_llm_from_config doesn't working with Chat Models
Labels: name: auto:improvement, name: area: models

### Issue you'd like to raise.

The llm loader `load_llm_from_config` defined [here](https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/llms/loading.py#L12) only returns `BaseLLM`s. There is no equivalent chat_model loader in langchain to load Chat Model LLMs from a config that I'm aware of. 

What do you think of either extending the `load_llm_from_config` such that it returns both `BaseLLM` and `BaseChatModel` 
```
load_llm_from_config(config: dict) -> Union[BaseLLM, BaseChatModel]
```

or create a chat_model loader in `langchain.chat_models.loading.py`

```
load_chat_llm_from_config(config: dict) -> BaseChatModel
```


### Suggestion:

_No response_
Add SemaDB VST wrapper
Labels: name: lgtm, name: area: vector store, name: auto:enhancement

- **Description**: Adding vectorstore wrapper for [SemaDB](https://rapidapi.com/semafind-semadb/api/semadb).
- **Issue**: None
- **Dependencies**: None
- **Twitter handle**: semafind

Checks performed:
- [x] `make format`
- [x] `make lint`
- [x] `make test`
- [x] `make spell_check`
- [x] `make docs_build`

Documentation added:

- SemaDB vectorstore wrapper tutorial

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Sentiment analysis chain 
Labels: name: auto:enhancement

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Sentimental analysis  chain 
Labels: name: auto:enhancement, name: area: models

### Feature request

The SentimentAnalysis chain is designed to analyze the sentiment of textual data using langchain. It uses a language model like OpenAI's GPT-3.5 & 4 to process text inputs and provide sentiment labels and scores. This chain can be used for various applications, including sentiment analysis of product reviews, social media comments, or customer feedback.

### Motivation

To contribute to open-source 

### Your contribution

yes , i am ready with my PR
'RunnableLambda' object has no attribute 'run'
Labels: name: auto:bug, name: area: models

### System Info

langchain==0.0.294

### Who can help?

@hwchase17 , @agola11 

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Build a `SimpleSequentialChain` including in its chains a `RunnableLambda`

It throws: `AttributeError: 'RunnableLambda' object has no attribute 'run'`

### Expected behavior

A RunnableLambda to have an attribute `run`...
Or rename the concept of Runnable.
Fix: Correct trailing commas in JSON before parsing
Labels: name: auto:improvement

This commit addresses the issue where JSON strings with trailing commas would cause parsing to fail. We have added a preprocessing step to correct these commas before attempting to parse the JSON string using the `json.loads()` function.

Changes:
- Added a `correct_trailing_commas` function that removes trailing commas from a given JSON string.
- Used this function to preprocess the JSON string inside the `parse` method of `PydanticOutputParser`.

Reason:
Some JSON producing systems or manual entries might inadvertently add trailing commas, which are not valid in standard JSON. This change aims to make our parser more robust and forgiving of such common mistakes, thereby reducing potential disruptions or failures due to invalid JSON formats.

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Parser Error with Langchain/agents/Conversational/Output_parser.py
Labels: name: area: agent, name: auto:bug, name: area: models

### System Info

langchain==0.0.279

### Who can help?

_No response_

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [X] Prompts / Prompt Templates / Prompt Selectors
- [X] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [X] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

With GPT-3.5 turbo 16k model, 

Query - What is universal workflow type?

Thought: Do I need to use a tool? Yes
Action: chat_with_datasource
Action Input: What is universal workflow type?
Observation: The Universal Workflow type represents logic flows in a system. It includes two types: Flows and Subflows.

Thought: Do I need to use a tool? No

raise OutputParserException(f"Could not parse LLM output: `{text}`")
langchain.schema.output_parser.OutputParserException: Could not parse LLM output: `Do I need to use a tool? No`

Analysis - 

Although when I checked i found, its using 
Langchain/agents/Conversational/prompt.py

--------------------------------------------------
FORMAT_INSTRUCTIONS = """To use a tool, please use the following format:

```
Thought: Do I need to use a tool? Yes
Action: the action to take, should be one of [{tool_names}]
Action Input: the input to the action
Observation: the result of the action
```

When you have a response to say to the Human, or if you do not need to use a tool, you MUST use the format:

```
Thought: Do I need to use a tool? No
{ai_prefix}: [Your response here]
```"""
-----------------------------------------------------

So, when it does not get any tool to use, it should generate some response in "ai_prefix}:"may be previous response of llm/observation at last step"
but, its unable to generate any response and resulted into parser error.


However, when I run the same code with GPT-4, below is the response I get, and its able to parse the output as well, since llm is able to generate 
{ai_prefix}: [Your response here] as per the prompt instruction, but GPT-3.5 model is unable to generate it.


-----------------------------------------------------------
Thought: Do I need to use a tool? Yes
Action: chat_with_datasource
Action Input: What is universal workflow type?
Observation: The Universal Workflow type represents logic flows in a system. It includes two types: Flows and Subflows.

Thought: Do I need to use a tool? No
AI: The Universal Workflow is a system that represents logic flows. It consists of different types such as Flows and Subflows.
------------------------------------------------------------

### Expected behavior

With GPT 3.5 model as well, it should generate AI: response, when agent found that it does not need to use a tool, which would fix the parse error.
Add condition in Prompt based on the retriever
Labels: name: area: memory, name: auto:question, name: auto:improvement, name: area: models

### Issue you'd like to raise.

Hello, 
I'm trying to change dynamically a prompt, based on the value of the context in a ConversationalRetrievalChain

```
prompt_template = """You are an AI assistant.
Use the following pieces of context to answer the question, in a precise way, at the end. 

Context: {context if context else "some default: I don't have information about it"}

Question: {question}
Helpful Answer :"""
QA_PROMPT = PromptTemplate(
    template=prompt_template, input_variables=["context", "question"]
)

llm = AzureChatOpenAI(
        temperature=0,
        deployment_name="gpt-35-turbo",
    )

memory = ConversationBufferWindowMemory(
    k=1,
    output_key='answer',
    memory_key='chat_history',
    return_messages=True)

retriever = vector.as_retriever(search_type="similarity_score_threshold", search_kwargs={"score_threshold": .70})

qa_chain = ConversationalRetrievalChain.from_llm(llm=llm,
                                                memory=memory,
                                                retriever=retriever,
                                                condense_question_prompt=CONDENSE_QUESTION_PROMPT,
                                                combine_docs_chain_kwargs={'prompt': QA_PROMPT},
                                                return_source_documents=True,
                                                verbose=True)
```

The idea is when the retriever gets 0 documents, to have something in the context that push the model to say I don't know and don't make up and answer. I already try by adding in the prompt "if you don't know the answer don't make it up" and didn't work.

Any idea how to add a condition in the template or control the LLM based on the output of the retriever?


### Suggestion:

_No response_
Qdrant - Load the saved vector db/store ?
Labels: name: area: vector store, name: auto:question, name: auto:improvement

### Issue you'd like to raise.

I use Qdrant through langchain  to store vectors.  But I can't find any example in docs where the dataset is searched based on a previously created collection. How to load the data? I found that Pinecone has "from_existing_index" function, which probably does the thing. But Qdrant doesn't have such a function. I created my own solution using qdrant_client, but I would like to use Langchain to simplify the script. How to do it?



### Suggestion:

_No response_
Vectorstore and ConversationalRetrievalChain should return the certainty of the relevance 
Labels: name: area: vector store, name: auto:improvement

### Feature request

I am using Weaviate with ConversationalRetrievalChain 
I am using `similarity_score_threshold` search type. 

It will call `similarity_search_with_score` in `vectorstore/weaviate.py` 
Currently it's written like this
`        if not self._by_text:
            vector = {"vector": embedded_query}
            result = (
                query_obj.with_near_vector(vector)
                .with_limit(k)
                .with_additional("vector")
                .do()
            )
        else:
            result = (
                query_obj.with_near_text(content)
                .with_limit(k)
                .with_additional("vector")
                .do()
            )
`

The `with_additional` function can accept a list of `AdditionalProperties`
So there's no point of hardcoding to be just the vector, and let the user add the additional arguments required. 

I've tested this just by changing ` .with_additional("vector")` to ` .with_additional(["vector", "certainty"] )` and it works just fine


### Motivation

I am using the Conversational Chain just because it's easier to manage than creating one. But it's limitations are apparent. I want to retrieve the scores it get which is currently not possible

### Your contribution

I don't know how to pass in the `_additional` arguments from the vectorstore.as_retriever() but just hard coding the args i want the tests worked fine.
Use InferenceClient instead of InferenceAPI for HuggingFaceHubEmbeddings
Labels: name: area: embeddings, name: auto:improvement

**Description:**
Replace deprecated `InferenceAPI` of Hugging Face Inference with `InferenceClient` in the module `embeddings.huggingface_hub`.

**Issue:**
#10483 

**Remarks:**
- A similar PR is open for the module `llms.huggingface_hub`: #10514 
- It's my first contribution to an open source project. Then don't hesitate to send me your comments 😃.
DOC: pydantic extraction example code not working
Labels: name: area: doc loader, name: auto:bug, name: auto:documentation

### Issue with current documentation:

The first code example following the Pydantic section, right after "Lets define a class with attributes annotated with types."

results for me in the following pydantic errors using gpt-4:

RuntimeError: no validator found for <class '__main__.Properties'>, see `arbitrary_types_allowed` in Config

The following versions of langchain and pydantic were used:
langchain==0.0.309
pydantic==2.4.2
pydantic_core==2.10.1


### Idea or request for content:

It would be nice to have the code updated so that it works out of the box.

Perhaps a reference to the extraction functions in the 'openai functions' section would be nice.
DOCX loader is not working properly in js
Labels: name: area: doc loader, name: auto:bug

### System Info

langchain latest version: 0.0.161
"mammoth": "^1.6.0",


### Who can help?

_No response_

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [X] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Creating blob data with the use of fetch
   const blobData = await fetch(totalFiles[i].url).then((res) => res.blob());
After this use this blob data in 
 const loader = new DocxLoader(blobData);
 
 But When I hit the next step

      const data = await loader.load();
      
      It give me an error:
      
      Error: Could not find file in options
    at Object.openZip (unzip.js:10:1)
    at extractRawText (index.js:82:1)
    at DocxLoader.parse (docx.js:25:1)
    at async langChainInitialization (index.js:88:20)
    at async handleRefDocUpload (index.js:136:9)
      

### Expected behavior

It should load the document.
Quote tool names
Labels: name: area: agent, name: auto:improvement

  - **Description:** Quote tool names in prompt templates
  - **Issue:** https://github.com/langchain-ai/langchain/issues/1657 https://github.com/langchain-ai/langchain/issues/1477 https://github.com/langchain-ai/langchain/issues/1358
  - **Tag maintainer:** @hwchase17

The original prompt template is ambiguous because it does not tell whether the tool names should be quoted or not. This PR makes it clear that the tool names should be quoted. It fixes #1358 for Nous-Hermes-Llama2-13b according to my tests.
Issue:  how to use multiple inputs with custom llm agent
Labels: name: area: agent, name: auto:question

### Issue you'd like to raise.

we are following https://python.langchain.com/docs/modules/agents/how_to/custom_llm_agent however, by default it's using one input only, how to make it accept multiple inputs same as STRUCTURED_CHAT_ZERO_SHOT_REACT_DESCRIPTION, thanks.

### Suggestion:

_No response_