delayedkarma/langchain-issues huggingface dataset:
Added matching async load func to PlaywrightURLLoader


<!--
Thank you for contributing to LangChain! Your PR will appear in our release under the title you set. Please make sure it highlights your valuable contribution.

Replace this with a description of the change, the issue it fixes (if applicable), and relevant context. List any dependencies required for this change.

After you're done, someone will review your PR. They may suggest improvements. If no one reviews your PR within a few days, feel free to @-mention the same people again, as notifications can get lost.

Finally, we'd love to show appreciation for your contribution - if you'd like us to shout you out on Twitter, please also include your handle!
-->

<!-- Remove if not applicable -->

Fixes # (issue)

The existing PlaywrightURLLoader load() function uses a synchronous browser which is not compatible with jupyter.
This PR adds a sister function aload() which can be run insisde a notebook.

#### Before submitting

<!-- If you're adding a new integration, please include:

1. a test for the integration - favor unit tests that does not rely on network access.
2. an example notebook showing its use


See contribution guidelines for more information on how to write tests, lint
etc:

https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
-->

#### Who can review?

Tag maintainers/contributors who might be interested:

<!-- For a quicker response, figure out the right person to tag with @

  @hwchase17 - project lead

  Tracing / Callbacks
  - @agola11

  Async
  - @agola11

  DataLoaders
  - @eyurtsev

  Models
  - @hwchase17
  - @agola11

  Agents / Tools / Toolkits
  - @vowelparrot

  VectorStores / Retrievers / Memory
  - @dev2049

 -->
Update schema.py
Labels: name: auto:improvement

# Override add_user_message func to support saving dicts passed in through the agent.run(input={}) pattern.

<!--
Thank you for contributing to LangChain! Your PR will appear in our release under the title you set. Please make sure it highlights your valuable contribution.

Replace this with a description of the change, the issue it fixes (if applicable), and relevant context. List any dependencies required for this change.

After you're done, someone will review your PR. They may suggest improvements. If no one reviews your PR within a few days, feel free to @-mention the same people again, as notifications can get lost.

Finally, we'd love to show appreciation for your contribution - if you'd like us to shout you out on Twitter, please also include your handle!
-->

<!-- Remove if not applicable -->

Fixes # (issue)

#### Before submitting

<!-- If you're adding a new integration, please include:

1. a test for the integration - favor unit tests that does not rely on network access.
2. an example notebook showing its use


See contribution guidelines for more information on how to write tests, lint
etc:

https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
-->

#### Who can review?

Tag maintainers/contributors who might be interested:

<!-- For a quicker response, figure out the right person to tag with @

  @hwchase17 - project lead

  Tracing / Callbacks
  - @agola11

  Async
  - @agola11

  DataLoaders
  - @eyurtsev

  Models
  - @hwchase17
  - @agola11

  Agents / Tools / Toolkits
  - @vowelparrot

  VectorStores / Retrievers / Memory
  - @dev2049

 -->
Define map only for chain_type of RetrievalQA

### Feature request

Hi,

I would like to use `RetrievalQA` chain with `chain_type` equal to map only. Is it actually possible with LangChain library? When generating answer I also get logits scores for each answer, so I would be able to choose the best answer based on these logits scores. Consequently, I only need the `RetrievalQA` chain to generate answer for each document retrieved. 

### Motivation

Logits scores is a new feature of hugging Face ecosystem for generative model. It would be interesting to enable the user to generate answer for each retrieved document then let the user decided the best answer based on these scores. So a `chain_type` of type map only is needed.

### Your contribution

Let me know if this exists, otherwise I would be glad to test.
try fix issue add fault tolerance
Labels: name: auto:bug

<!--
Thank you for contributing to LangChain! Your PR will appear in our release under the title you set. Please make sure it highlights your valuable contribution.

Replace this with a description of the change, the issue it fixes (if applicable), and relevant context. List any dependencies required for this change.

After you're done, someone will review your PR. They may suggest improvements. If no one reviews your PR within a few days, feel free to @-mention the same people again, as notifications can get lost.

Finally, we'd love to show appreciation for your contribution - if you'd like us to shout you out on Twitter, please also include your handle!
-->

<!-- Remove if not applicable -->

Fixes # (issue)

#### Before submitting

<!-- If you're adding a new integration, please include:

1. a test for the integration - favor unit tests that does not rely on network access.
2. an example notebook showing its use


See contribution guidelines for more information on how to write tests, lint
etc:

https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
-->

#### Who can review?

Tag maintainers/contributors who might be interested:

<!-- For a quicker response, figure out the right person to tag with @

  @hwchase17 - project lead

  Tracing / Callbacks
  - @agola11

  Async
  - @agola11

  DataLoaders
  - @eyurtsev

  Models
  - @hwchase17
  - @agola11

  Agents / Tools / Toolkits
  - @vowelparrot

  VectorStores / Retrievers / Memory
  - @dev2049

 -->
Combination of RetrievalQA with custom embeddings API

### Feature request

Hi,

I would like to use `RetrievalQA` chain with custom remote embeddings API. As far as I could see when searching through documentation, I couldn't find any way on how to specify the url and payload to send to the custom API in the argument of the `RetrievalQA` object.

One solution would be to be able to take the url and payload into the `langchain.embeddings` module. The second path would be to be able to specify directly the query embeddings within the `RetrievalQA` object.

### Motivation

Using a custom API for embeddings is highly desirable for anyone who doesn't want to use paying APIs or who wants to optimize inference computations.

### Your contribution

Please indicate me if this already exists in the library.
Facing issue while trying to Stream output for QLoRa model using TextIteratorStreamer 

### System Info

A4000 GPU

transformer 4.30.0
langchain latest

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [X] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [X] Async

### Reproduction

streamer = TextIteratorStreamer(tokenizer, skip_special_tokens=True, use_multiprocessing=False)

pipe = pipeline(
                "text-generation", model=model, tokenizer=tokenizer, max_new_tokens=1000, streamer=streamer, temperature=0, top_p=0.2
            )
hf = HuggingFacePipeline(pipeline=pipe)

llm_chain.run("Where is New delhi?").lstrip()

### Expected behavior

While loading a QLoRa model and passing TextIteratorStreamer  to iterate over generated words from LLM i am getting TypeError: cannot pickle '_thread.lock' object.

While using only TextStreamer it is running fine but could not able to stream the result at API.
Update ArXiv tool.py to fix issue #5901

<!--
Thank you for contributing to LangChain! Your PR will appear in our release under the title you set. Please make sure it highlights your valuable contribution.

Replace this with a description of the change, the issue it fixes (if applicable), and relevant context. List any dependencies required for this change.

After you're done, someone will review your PR. They may suggest improvements. If no one reviews your PR within a few days, feel free to @-mention the same people again, as notifications can get lost.

Finally, we'd love to show appreciation for your contribution - if you'd like us to shout you out on Twitter, please also include your handle!
-->

<!-- Remove if not applicable -->

Fixes #5901

  - @hwchase17
  - @vowelparrot
add dashscope text embedding
Labels: name: lgtm



#### What I do
Adding embedding api for [DashScope](https://help.aliyun.com/product/610100.html), which is the DAMO Academy's multilingual text unified vector model based on the LLM base. It caters to multiple mainstream languages worldwide and offers high-quality vector services, helping developers quickly transform text data into high-quality vector data. Currently supported languages include Chinese, English, Spanish, French, Portuguese, Indonesian, and more. 

#### Who can review?

  Models
  - @hwchase17
  - @agola11
Langchain model using wikipedia tool fails to return response, but generates it successfully

### Langchain agent error

I have the following error while trying to load an llm agent and connecting to wikipedia tool, it seems that the response is generated but throws an error when trying to convert the response to the right format.
Am I missing something or doing something wrong?
Thank you in advanced :)

```python
from langchain.agents import load_tools
from langchain.agents import initialize_agent
from langchain.agents import AgentType
from langchain.memory import ConversationBufferMemory
from langchain.llms import AzureOpenAI

import openai

openai.api_type = "azure"
openai.api_base = "api_base"
openai.api_version = "2022-12-01"
openai.api_key = "key"

memory = ConversationBufferMemory(memory_key="chat-history", return_messages=True,output_key='answer')
llm = AzureOpenAI(deployment_name="text-davinci-003", model_name="text-davinci-003", openai_api_key="key",openai_api_version = "2022-12-01")
tools = load_tools (['wikipedia'], llm=llm)

agent = initialize_agent(
            tools,
            agent=AgentType.CONVERSATIONAL_REACT_DESCRIPTION,
            verbose=True,
            memory=memory,
            llm=llm)

response = {}
response['input'] = "When did Cristobal Colon discover America?"
response['chat_history'] = []
res = agent.run(response) 
```
![errrrCapture](https://github.com/hwchase17/langchain/assets/89514540/9b570bf7-00b4-41b7-8e83-4bdb7a2be49b)




### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [X] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [X] Memory
- [X] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Execute the code snippet above to reproduce the error

### Expected behavior

We are expecting the generated response as an output but seems to not be working
DOC: Inconsistent unit of measure for chunk_size and chunk_overlap

### Issue with current documentation:

It's not clear in the documentation ([JS/TS](https://js.langchain.com/docs/modules/indexes/text_splitters/#:~:text=Parameters%E2%80%8B,default%20value%20is%201000%20tokens.)) if the unit of measure used for chunks is in single text characters or in tokens. It's incosistent because it explicits both units in the same phrase.
As far as I know a token should be around 4 characters, so it's an important size difference.

### Idea or request for content:

Please change the documentation explaining whether it's in text characters or in tokens.
SQLDatabaseChain has SQL injection issue

### System Info

There is no safeguard in SQLDatabaseChain to prevent a malicious user from sending a prompt such as "Drop Employee table".

SQLDatabaseChain should have a facility to intercept and review the SQL before sending it to the database.

Creating this separately from https://github.com/hwchase17/langchain/issues/1026 because the SQL injection issue and the Python exec issues are separate. For example SQL injection cannot be solved with running inside an isolated container. 

[LangChain version: 0.0.194. Python version 3.11.1]

<img width="596" alt="image" src="https://github.com/hwchase17/langchain/assets/227187/3ced0139-490f-4e41-a880-71dc864ee12c">

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Here is a repro using the Chinook sqlite database used in the example ipynb. Running this will drop the Employee table from the SQLite database.

```python
chinook_sqlite_uri = "sqlite:///Chinook_Sqlite_Tmp.sqlite"
from langchain import OpenAI, SQLDatabase, SQLDatabaseChain
llm = OpenAI(temperature=0)
db = SQLDatabase.from_uri(chinook_sqlite_uri)
db.get_usable_table_names()
db_chain = SQLDatabaseChain.from_llm(llm=llm, db=db, verbose=True)
db_chain.run("How many employees are there?")
db_chain.run("Drop the employee table")
```

### Expected behavior

LangChain should provide a mechanism to intercept SQL before sending it to the database. During this interception the SQL can be examined and rejected if it performs unsafe operations.
feat: add content_format param to ConfluenceLoader.load()
Labels: name: lgtm

Confluence API supports difference format of page content. The storage format is the raw XML representation for storage. The view format is the HTML representation for viewing with macros rendered as though it is viewed by users. 

Add the `content_format` parameter to `ConfluenceLoader.load()` to specify the content format, this is
set to `ContentFormat.STORAGE` by default.

#### Who can review?

Tag maintainers/contributors who might be interested: @eyurtsev
Fix: change the chatgpt plugin retriever metadata format
Labels: name: lgtm

the current implement put the doc itself as the metadata, but the document  chatgpt plugin retriever returned already has a `metadata` field, it's better to use that instead. 

the original code will throw the following exception when using `RetrievalQAWithSourcesChain`, becuse it can not find the field `metadata`:

```python
Exception has occurred: ValueError       (note: full exception trace is shown but execution is paused at: _run_module_as_main)
Document prompt requires documents to have metadata variables: ['source']. Received document with missing metadata: ['source'].
  File "/home/wangjie/anaconda3/envs/chatglm/lib/python3.10/site-packages/langchain/chains/combine_documents/base.py", line 27, in format_document
    raise ValueError(
  File "/home/wangjie/anaconda3/envs/chatglm/lib/python3.10/site-packages/langchain/chains/combine_documents/stuff.py", line 65, in <listcomp>
    doc_strings = [format_document(doc, self.document_prompt) for doc in docs]
  File "/home/wangjie/anaconda3/envs/chatglm/lib/python3.10/site-packages/langchain/chains/combine_documents/stuff.py", line 65, in _get_inputs
    doc_strings = [format_document(doc, self.document_prompt) for doc in docs]
  File "/home/wangjie/anaconda3/envs/chatglm/lib/python3.10/site-packages/langchain/chains/combine_documents/stuff.py", line 85, in combine_docs
    inputs = self._get_inputs(docs, **kwargs)
  File "/home/wangjie/anaconda3/envs/chatglm/lib/python3.10/site-packages/langchain/chains/combine_documents/base.py", line 84, in _call
    output, extra_return_dict = self.combine_docs(
  File "/home/wangjie/anaconda3/envs/chatglm/lib/python3.10/site-packages/langchain/chains/base.py", line 140, in __call__
    raise e
```

Additionally, the `metadata` filed in the `chatgpt plugin retriever` have these fileds by default:
```json
{
    "source":  "file",   //email, file or chat
    "source_id": "filename.docx", // the filename
    "url": "", 
    ...
}
```
so, we should set `source_id` to `source` in the langchain metadata.

```python
metadata = d.pop("metadata", d)
if(metadata.get("source_id")):
    metadata["source"] = metadata.pop("source_id")
```

#### Who can review?
@dev2049

<!-- For a quicker response, figure out the right person to tag with @

  @hwchase17 - project lead

  Tracing / Callbacks
  - @agola11

  Async
  - @agola11

  DataLoaders
  - @eyurtsev

  Models
  - @hwchase17
  - @agola11

  Agents / Tools / Toolkits
  - @vowelparrot

  VectorStores / Retrievers / Memory
  - @dev2049

 -->
[RFC] [Breaking] Retriever Callbacks

This is one way we could implement it. I try to break as few external APIs as possible, but I'll call out a few (of many more) places that get broken.

I don't leap for joy at how much code each change requires when we add new callbacks and how daunting the function signature starts to look for these interfaces. And I don't love how strict things have to be in passing the callback to each component, though maintaining a top level stack that's both thread safe and async context aware probably has other problems.

I haven't updated all the experimental autogpt/ baby agi memory things. and since memory currently doestn't require callbacks, a retriever memory (in experimental) wouldn't get hit
arxiv: Correct name of search client attribute to 'arxiv_search' from incorrect 'arxiv_client'

+ this private attribute is referenced as `arxiv_search` in internal usage and is set when verifying the environment

twitter: @spazm 


#### Who can review?

Any of @hwchase17, @leo-gan, or @bongsang might be interested in reviewing.

+ Mismatch between `arxiv_client` attribute vs `arxiv_search` in validation and usage is present in the initial commit by @hwchase17.  
+ @leo-gan has made most of the edits.
+ @bongsang implemented pdf download.
commit two document_loaders to support xml file and java file

<!--
Thank you for contributing to LangChain! Your PR will appear in our release under the title you set. Please make sure it highlights your valuable contribution.

Replace this with a description of the change, the issue it fixes (if applicable), and relevant context. List any dependencies required for this change.

After you're done, someone will review your PR. They may suggest improvements. If no one reviews your PR within a few days, feel free to @-mention the same people again, as notifications can get lost.

Finally, we'd love to show appreciation for your contribution - if you'd like us to shout you out on Twitter, please also include your handle!
-->

<!-- Remove if not applicable -->

Fixes # (issue)

#### Before submitting

<!-- If you're adding a new integration, please include:

1. a test for the integration - favor unit tests that does not rely on network access.
2. an example notebook showing its use


See contribution guidelines for more information on how to write tests, lint
etc:

https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
-->

####@eyurtsev

Tag maintainers/contributors who might be interested:

<!-- For a quicker response, figure out the right person to tag with @

  @hwchase17 - project lead

  Tracing / Callbacks
  - @agola11

  Async
  - @agola11

  DataLoaders
  - @eyurtsev

  Models
  - @hwchase17
  - @agola11

  Agents / Tools / Toolkits
  - @vowelparrot

  VectorStores / Retrievers / Memory
  - @dev2049

 -->
Fix openai proxy error

Fixes proxy error.
Since openai does not parse proxy parameters and uses openai.proxy directly, the proxy method needs to be modified.

https://github.com/openai/openai-python/blob/7610c5adfaebe3ffdb9927a551a741a3fab1b62e/openai/api_requestor.py#LL90

#### Who can review?
  @hwchase17 - project lead

  Models
  - @hwchase17
  - @agola11
How can I make langchain run asynchronously? 

### Issue you'd like to raise.

I'd like to have my server that uses langchain to run asynchronously, specifically, I'd like to have `await` when running `chain({})`. My understanding of this is that the server will start handling other requests made to it, and not wait for `chain({})` to finish before handling other requests.

```
@app.post("/document/qa")
async def qa(document: Document):
    try:
        api_key = os.environ.get("OPENAI_API_KEY")
        embedding = OpenAIEmbeddings()    
        collection_name = document.user_id + "/" + document.pdf_title
        connection_result = check_if_connection_exists(collection_name)

        if connection_result == False:
            raise Exception("Collection does not exist")

        store = PGVector(
            connection_string=connection_string, 
            embedding_function=embedding, 
            collection_name=collection_name,
            distance_strategy=DistanceStrategy.COSINE
        )

        memory_response = get_conversation_memory(collection_name)
        if (memory_response["status-code"] == 404):
            raise Exception("Failed to retrieve conversation memory")

        json_response = memory_response["request-response"]
        chat_memory = ChatMemory.from_pg_conversation(
            response_json=json_response)

        retriever = store.as_retriever()
        query = document.query

        llm = ChatOpenAI(streaming=False, openai_api_key=api_key)
        # don't mind the streaming lol it's a work in progress
        streaming_llm = ChatOpenAI(streaming=False, openai_api_key=api_key, verbose=True,)

        question_generator = LLMChain(llm=llm, prompt=CONDENSE_QUESTION_PROMPT)
        doc_chain = load_qa_with_sources_chain(llm=streaming_llm, chain_type="map_reduce")

        chain = ConversationalRetrievalChain(
            retriever=retriever,
            question_generator=question_generator,
            combine_docs_chain=doc_chain,
        )

        # I'd like to have await here
        result = chain({
                "question": query,  
                "chat_history": chat_memory.history
            }, 
            return_only_outputs=True
        )

        ai_split_response = result["answer"].split("SOURCES: ")
        ai_answer = ai_split_response[0]
        ai_sources = ai_split_response[1].split(", ")
        # removes the period at the end of the last source
        ai_sources[-1] = ai_sources[-1].replace(".", "")
        # posts conversation result to the database
        post_response = post_conversation_memory(
            title=collection_name,
            user_message=query,
            ai_response=ai_answer,
            ai_sources=ai_sources,
        )

        if (post_response["status-code"] == 404):
            raise Exception("Failed to post conversation memory")

        return {
            "question": query,
            "result": ai_answer,
            "sources": ai_sources,
        }
    except Exception as e:
        raise HTTPException(status_code=404, detail=str(e))
```

### Suggestion:

_No response_