delayedkarma/langchain-issues huggingface dataset:
Issue: value error in BooleanOutputParser

### Issue you'd like to raise.

when use the BooleanOutputParser in the chain_filter.py, the LLMs output 'yes|no'，the function 'parse' will throw ValueError。

### Suggestion:

I fix it,as follows:

def parse(self, text: str) -> bool:
        """Parse the output of an LLM call to a boolean.

        Args:
            text: output of language model

        Returns:
            boolean

        """
        cleaned_text = text.upper().strip()
        if cleaned_text not in (self.true_val, self.false_val):
            raise ValueError(
                f"BooleanOutputParser expected output value to either be "
                f"{self.true_val} or {self.false_val}. Received {cleaned_text}."
            )
        return cleaned_text == self.true_val
Parser output may not always produce a string (based on what GPT returns), any valid json construct is possible

### System Info

langchain 0.183

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [X] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

If GPT returns a message formatted this way:

Thought: I will produce a list of things.
Action:
```
{
  "action": "Final Answer",
  "action_input": [
    { "thing" : "A", "attribute" : "X" },
    { "thing" : "B", "attribute" : "Y" }
]
}
```

Where action_input is a list instead of a string, the output parser will recognize it as valid and return it all the way back.

output_parser.py: line 23->32:
```
    def parse(self, text: str) -> Union[AgentAction, AgentFinish]:
        try:
            action_match = re.search(r"```(.*?)```?", text, re.DOTALL)
            if action_match is not None:
                response = json.loads(action_match.group(1).strip(), strict=False)
                if isinstance(response, list):
                    # gpt turbo frequently ignores the directive to emit a single action
                    logger.warning("Got multiple action responses: %s", response)
                    response = response[0]
                if response["action"] == "Final Answer":
```

reponse = json.loads only checks that the response is a valid json, not that the type of "reponse['action']" is a string.

While not necessarily an issue functionally speaking, this means that the function prototype
in base.py:225
```
    def run(self, *args: Any, callbacks: Callbacks = None, **kwargs: Any) -> str:
        """Run the chain as text in, text out or multiple variables, text out."""
```
is incorrect. the return value usually is a string, but could be a dictionary or a list based on the output of gpt.





### Expected behavior

Either force GPT to reformulate, or change the function prototype
DeepSpeed Support And Device Map Fix For HuggingFacePipeline
Labels: name: needs documentation

# DeepSpeed Support For HuggingFacePipeline

This PR adds support for DeepSpeed with the Huggingface pipelines for local models.  DeepSpeed allows over 2x faster model inference for a wide variety of models.

To use this, you need a modern GPU and have CUDA installed that is the same version as the one used to compile Pytorch  

You also need to install deepspeed with the following command:
```pip install deepspeed```

Fixes # (issue)

A test has been added for this new feature.  As a side note, some existing tests seem to not be working due to the fact that ```max_new_tokens``` is not a supported keyword.

## Who can review?

Community members can review the PR once tests pass. Tag maintainers/contributors who might be interested:

  @hwchase17 - project lead
  Models
  - @hwchase17
  - @agola11
RFC: Use wasm-exec package to sandbox code execution by the Python REPL tool

### Issue you'd like to raise.

Issues #1026  and #5294 raise legitimate concerns with the security of executing LLM-generated code via the `exec()` function. Out of that discussion came a link to [this blog](https://til.simonwillison.net/webassembly/python-in-a-wasm-sandbox) that demonstrated a way to use Wasm to execute Python code in an isolated interpreter. 

I have developed a library based on that code called [wasm-exec](https://github.com/Jflick58/wasm_exec) to provide a clean interface to this solution. I’d like to next make a PR in LangChain replacing instances of `exec()` with `wasm_exec()` but wanted to get some feedback on this solution before doing so. 

Right now, the largest unknown is the extent of support for arbitrary packages, which may make running something like Pandas in the sandbox untenable until a solution is found. I believe I have a path forward on that (via installing when configuring the wasm runtime), but will need to continue to experiment. 

### Suggestion:

_No response_
Failure when running pip install langchain[all]

### System Info

Langchain version: NA
Python 3.10.9
WSL2

### Who can help?

_No response_

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

1. pip install langchain[all]
2. Wait
3. Observe:

```
  Preparing metadata (setup.py) ... done
  Downloading openai-0.0.2.tar.gz (741 bytes)
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [6 lines of output]
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/tmp/pip-install-9c_7ozkg/openai_9e8b55b2ec17406d8b64d964c29099f3/setup.py", line 6, in <module>
          raise RuntimeError(
      RuntimeError: This package is a placeholder package on the public PyPI instance, and is not the correct version to install. If you are having trouble figuring out the correct package to install, please contact us.
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
```

### Expected behavior

Installation complete.
fix: compatibility with old version FAISS vector store file
Labels: name: wontfix

# fix: compatibility with old version FAISS vector store file

This PR addresses the compatibility issues with langchain when processing vector files generated by older versions of langchain and FAISS. The older files lack the required internal member variables (`deployment, allowed_special, disallowed_special, request_timeout`) that are needed by the new version, leading to compatibility issues as reported in #3251.

In addition, this PR also adds a new environment variable `OPENAI_EMBEDDINGS_DEPLOYMENT` which enables langchain to dynamically set the deployment id for older versions of the data files.

This fix does not introduce any functional changes. Instead, it sets default values for these new variables so that they can handle older data files and prevent compatibility bugs from occurring.

Fixes #3251 

## Who can review?

Community members can review the PR once tests pass. Tag maintainers/contributors who might be interested:

  @hwchase17 - project lead

  Async
  - @agola11

  DataLoaders
  - @eyurtsev

  Models
  - @hwchase17
  - @agola11

  VectorStores / Retrievers / Memory
  - @dev2049
Implements support for Personal Access Token Authentication in the ConfluenceLoader
Labels: name: lgtm, name: maintainer-to-merge

# Implements support for Personal Access Token Authentication in the ConfluenceLoader

Fixes #5191

Implements a new optional parameter for the ConfluenceLoader: `token`.
This allows the use of personal access authentication when using the on-prem server version of Confluence. 

## Who can review?

Community members can review the PR once tests pass. Tag maintainers/contributors who might be interested:
@eyurtsev @Jflick58 

Twitter Handle: felipe_yyc
Running into this error while creating embeddings out of pdf file

### System Info

File "d:\langchain\pdfqa-app.py", line 46, in _upload_data
    Pinecone.from_texts(self.doc_chunk,embeddings,batch_size=16,index_name=self.index_name)
  File "E:\anaconda\envs\langchain\lib\site-packages\langchain\vectorstores\pinecone.py", line 232, in from_texts
    embeds = embedding.embed_documents(lines_batch)
  File "E:\anaconda\envs\langchain\lib\site-packages\langchain\embeddings\openai.py", line 297, in embed_documents
    return self._get_len_safe_embeddings(texts, engine=self.deployment)
  File "E:\anaconda\envs\langchain\lib\site-packages\langchain\embeddings\openai.py", line 221, in _get_len_safe_embeddings
    token = encoding.encode(
  File "E:\anaconda\envs\langchain\lib\site-packages\tiktoken\core.py", line 117, in encode
    if match := _special_token_regex(disallowed_special).search(text):
TypeError: expected string or buffer

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [X] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [X] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

def _load_docs(self):
        loader=PyPDFLoader("D:\langchain\data_source\\1706.03762.pdf")
        self.doc= loader.load()
        

    def _split_docs (self):
        text_splitter = RecursiveCharacterTextSplitter(chunk_size=self.chunk_size, chunk_overlap=self.chunk_overlap,separators= ["\n\n", ""])
        self.doc_chunk = text_splitter.split_documents(self.doc)
    
    def _upload_data(self):
        embeddings = OpenAIEmbeddings()
        list_of_index = pinecone.list_indexes()
        if self.index_name in list_of_index:
            Pinecone.from_texts(self.doc_chunk,embeddings,batch_size=16,index_name=self.index_name)
        else:
            pinecone.create_index(self.index_name, dimension=1024) # for open AI
            Pinecone.from_texts(self.doc_chunk,embeddings,batch_size=16,index_name=self.index_name)
    def dataloader(self):
        self._load_docs()
        self._split_docs()
        self._upload_data()

### Expected behavior

Please help with the solution
Update confluence.py to return spaces between elements
Labels: name: lgtm, name: maintainer-to-merge

# Update confluence.py to return spaces between elements like headers and links.

Please see https://stackoverflow.com/questions/48913975/how-to-return-nicely-formatted-text-in-beautifulsoup4-when-html-text-is-across-m

Given:

```html
<address>
        183 Main St<br>East Copper<br>Massachusetts<br>U S A<br>
        MA 01516-113
    </address>
```

The document loader currently returns:

```
'183 Main StEast CopperMassachusettsU S A        MA 01516-113'
```

After this change, the document loader will return:

```
183 Main St East Copper Massachusetts U S A MA 01516-113
```


@eyurtsev would you prefer this to be an option that can be passed in?
Supabase similarity search with metadata filtering
Labels: name: area: vector store, name: auto:improvement

# Supabase similarity search with metadata filtering

<!--
Thank you for contributing to LangChain! Your PR will appear in our release under the title you set. Please make sure it highlights your valuable contribution.

Replace this with a description of the change, the issue it fixes (if applicable), and relevant context. List any dependencies required for this change.

After you're done, someone will review your PR. They may suggest improvements. If no one reviews your PR within a few days, feel free to @-mention the same people again, as notifications can get lost.
-->

<!-- Remove if not applicable -->

From what I understood, `SupabaseVectorStore`'s similarity searches did not implement any sort of filtering based on the metadata stored in Supabase. Furthermore, I could not find any way of using `metadata_filter` to limit the results given by a `RetrievalQA`.

This is my first open source contribution so if I happened to overlook anything please let me know!

### Usage

Metadata can be defined in the constructor.

```
vectorStore = SupabaseVectorStore(client=supabase, embedding=embeddings, table_name="documents", metadatas=[{"key": "value"}])

# unchanged
qa = RetrievalQA.from_chain_type(llm=ChatOpenAI(), chain_type="stuff", retriever=vectorStore.as_retriever())
res = qa.run("What is langchain?")
```

@dev2049
BaseConversationalRetrievalChain error on 0 docs found

### System Info

BaseConversationalRetrievalChain._call() and ._acall() run into errors eventually in LLMChain.prep_prompts() on referencing input_list[0].  This will trigger an index error with an empty input_list, which happens when there are no Documents retrieved during _call() by BaseConversationalRetrievalChain._get_docs().  There should be a check for this.

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Set up a qa call to ConversationalRetrievalChain.from_llm() on an retriever to which a filter has been applied that will result in no Documents being matched.

### Expected behavior

If 0 docs are found, don't bother running the rest of the process.
error thrown when trying to implement BabyAGI with tools with multiple inputs (requiring Structured Tool Chat Agent) 

Windows 11, Anaconda, python 3.9.16 LanchChain 0.0.183
My goal is to extend the tools used by baby AGI, more specifically to use At least the basic    WriteFileTool() and ReadFileTool(). they use two inputs though, so I cannot stick with the vanilla ZeroShotAgent. forgive my poor understanding of Python but I have tried to use AgentType.STRUCTURED_CHAT_ZERO_SHOT_REACT_DESCRIPTION and replace it in the code. Agent initialization is the only modification I've made to the code in the link below, though it throws an error. Could anyone kindly help updating the code in order to be able to leverage multiple input tools or provide guidance or resources? 

https://python.langchain.com/en/latest/use_cases/agents/baby_agi_with_agent.html?highlight=babyagi%20with%20tools 


```python 

from langchain.agents import AgentType 

from langchain.agents import initialize_agent 

    @classmethod 

    def from_llm( 

        cls, llm: BaseLLM, vectorstore: VectorStore, verbose: bool = False, **kwargs 

    ) -> "BabyAGI": 

        """Initialize the BabyAGI Controller.""" 

        task_creation_chain = TaskCreationChain.from_llm(llm, verbose=verbose) 

        task_prioritization_chain = TaskPrioritizationChain.from_llm( 

            llm, verbose=verbose 

        ) 

        llm_chain = LLMChain(llm=llm, prompt=prompt) 

        tool_names = [tool.name for tool in tools] 

        agent = initialize_agent(agent=AgentType.STRUCTURED_CHAT_ZERO_SHOT_REACT_DESCRIPTION,llm=llm_chain, tools=tool_names) 

        agent_executor = AgentExecutor.from_agent_and_tools( 

            agent=agent, tools=tools, verbose=True 

        ) 

        return cls( 

            task_creation_chain=task_creation_chain, 

            task_prioritization_chain=task_prioritization_chain, 

            execution_chain=agent_executor, 

            vectorstore=vectorstore, 

            **kwargs, 

        ) 

``` 

I get the error 

```python 

 File ~\anaconda3\envs\aagi\lib\site-packages\langchain\agents\structured_chat\base.py:83 in create_prompt  args_schema = re.sub("}", "}}}}", re.sub("{", "{{{{", str(tool.args))) 
AttributeError: 'str' object has no attribute 'args' 
``` 

### Suggestion:

_No response_
Formatter chain

### Feature request

Creation of a chain or wrapper that uses two LLMs with different models to force correct formatting:
- do the "difficult" work with a model like GPT-4: output format is free text.
- have a second LLM (davinci-003 or a specialized model for this purpose) that is good at following formatting instructions to convert the output of the first model into (say) JSON.


### Motivation

When a model is set up to be creative, or the use case demands that the output is consistently formatted correctly, we are often out of luck, and the output breaks an app.  This proposed use of two models will most likely increase the accuracy of the output a lot.


### Your contribution

If maintainers think this is a good idea, please add your input, and I will be happy to provide a PR.
Add async support to routing chains

# Add async support for (LLM) routing chains

<!--
Thank you for contributing to LangChain! Your PR will appear in our release under the title you set. Please make sure it highlights your valuable contribution.

Replace this with a description of the change, the issue it fixes (if applicable), and relevant context. List any dependencies required for this change.

After you're done, someone will review your PR. They may suggest improvements. If no one reviews your PR within a few days, feel free to @-mention the same people again, as notifications can get lost.
-->

<!-- Remove if not applicable -->

Add asynchronous LLM calls support for the routing chains. More specifically:
- Add async `aroute` function (i.e. async version of `route`) to the `RouterChain` which calls the routing LLM asynchronously
- Implement the async `_acall` for the `LLMRouterChain`
-  Implement the async `_acall` function for `MultiRouteChain` which first calls asynchronously the routing chain with its new `aroute` function, and then calls asynchronously the relevant destination chain.

<!-- If you're adding a new integration, please include:

1. a test for the integration - favor unit tests that does not rely on network access.
2. an example notebook showing its use


See contribution guidelines for more information on how to write tests, lint
etc:

https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
-->

## Who can review?

- @agola11

<!-- For a quicker response, figure out the right person to tag with @

  @hwchase17 - project lead
  Async
  - @agola11
        
 -->
HtmlTextSplitter

# HtmlTextSplitter

I am submitting a new HtmlTextSplitter class, which attempts to split text along HTML layout elements.

This PR addresses the need for HTML text splitting functionality in the LangChain library. There are no additional dependencies required for this change.

## Examples

An [example notebook](docs/modules/indexes/text_splitters/examples/html.ipynb) showing its use

## Who can review?

Community members can review the PR once tests pass. Tag maintainers/contributors who might be interested:
 - @vowelparrot
Fixes iter error in FAISS add_embeddings call

# Remove re-use of iter within add_embeddings causing error

As reported in https://github.com/hwchase17/langchain/issues/5336 there is an issue currently involving the atempted re-use of an iterator within the FAISS vectorstore adapter

Fixes # https://github.com/hwchase17/langchain/issues/5336

## Who can review?

Community members can review the PR once tests pass. Tag maintainers/contributors who might be interested:

  VectorStores / Retrievers / Memory
  - @dev2049
ImportError: DLL load failed while importing interpreter: The specified module could not be found.
Labels: name: stale

I cannot figure out why this is happening:   File "c:\Users\Yaseen\Documents\AVA\main.py", line 2, in <module>
    import langchain
  File "C:\Users\Yaseen\AppData\Local\Programs\Python\Python311\Lib\site-packages\langchain\__init__.py", line 6, in <module>
    from langchain.agents import MRKLChain, ReActChain, SelfAskWithSearchChain
  File "C:\Users\Yaseen\AppData\Local\Programs\Python\Python311\Lib\site-packages\langchain\agents\__init__.py", line 2, in <module>    
    from langchain.agents.agent import (
  File "C:\Users\Yaseen\AppData\Local\Programs\Python\Python311\Lib\site-packages\langchain\agents\agent.py", line 16, in <module>      
    from langchain.agents.tools import InvalidTool
  File "C:\Users\Yaseen\AppData\Local\Programs\Python\Python311\Lib\site-packages\langchain\agents\tools.py", line 8, in <module>       
    from langchain.tools.base import BaseTool, Tool, tool
  File "C:\Users\Yaseen\AppData\Local\Programs\Python\Python311\Lib\site-packages\langchain\tools\__init__.py", line 45, in <module>    
    from langchain.tools.powerbi.tool import (
  File "C:\Users\Yaseen\AppData\Local\Programs\Python\Python311\Lib\site-packages\langchain\tools\powerbi\tool.py", line 10, in <module>
    from langchain.chains.llm import LLMChain
  File "C:\Users\Yaseen\AppData\Local\Programs\Python\Python311\Lib\site-packages\langchain\chains\__init__.py", line 18, in <module>
    from langchain.chains.llm_math.base import LLMMathChain
  File "C:\Users\Yaseen\AppData\Local\Programs\Python\Python311\Lib\site-packages\langchain\chains\llm_math\base.py", line 9, in <module>
    import numexpr
  File "C:\Users\Yaseen\AppData\Local\Programs\Python\Python311\Lib\site-packages\numexpr\__init__.py", line 24, in <module>
    from numexpr.interpreter import MAX_THREADS, use_vml, __BLOCK_SIZE1__
ImportError: DLL load failed while importing interpreter: The specified module could not be found.
DOC: the example on setting model_kwrgs in HuggingFaceEmbeddings does not work

### Issue with current documentation:

The example in https://python.langchain.com/en/latest/reference/modules/embeddings.html 
```
from langchain.embeddings import HuggingFaceEmbeddings

model_name = "sentence-transformers/all-mpnet-base-v2"
model_kwargs = {'device': 'cpu'}
hf = HuggingFaceEmbeddings(model_name=model_name, model_kwargs=model_kwargs)
```
does not work.
I get an error: 
> pydantic.error_wrappers.ValidationError: 1 validation error for HuggingFaceEmbeddings
model_kwargs
  extra fields not permitted (type=value_error.extra)


### Idea or request for content:

fixed example