delayedkarma/langchain-issues huggingface dataset:
Structured tools cannot properly infer function schema

### System Info

```
$ pip show langchain
Name: langchain
Version: 0.0.186
Summary: Building applications with LLMs through composability
Home-page: https://www.github.com/hwchase17/langchain
Author:
Author-email:
License: MIT
Location: /home/mteoh/temp_venv/venv/lib/python3.10/site-packages
Requires: PyYAML, pydantic, tenacity, dataclasses-json, numexpr, numpy, openapi-schema-pydantic, aiohttp, async-timeout, requests, SQLAlchemy
Required-by:
```

```
$ python --version
Python 3.10.2
```

### Who can help?

@vowelpa

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [X] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

1. install langchain version 0.0.186, python version 3.10.2
2. Run the code below (I put in a file called `mwe.py`)
```python
from langchain.tools import StructuredTool
from typing import Dict
from pydantic import BaseModel

def foo(args_dict: Dict[str, str]):
    return "hi there"

class FooSchema(BaseModel):
    args_dict: Dict[str, str]

foo_tool = StructuredTool.from_function(
    foo,
    name="FooTool",
    description="min working example of a bug?",
    # args_schema=FooSchema  # inferring this schema does not work
)

result = foo_tool.run(tool_input={
    "args_dict": {"aa": "bb"}
})

print(result)
```
4. observe the error below:
```
Traceback (most recent call last):
  File "/home/mteoh/temp_venv/mwe.py", line 18, in <module>
    result = foo_tool.run(tool_input={
  File "/home/mteoh/temp_venv/venv/lib/python3.10/site-packages/langchain/tools/base.py", line 247, in run
    parsed_input = self._parse_input(tool_input)
  File "/home/mteoh/temp_venv/venv/lib/python3.10/site-packages/langchain/tools/base.py", line 190, in _parse_input
    result = input_args.parse_obj(tool_input)
  File "pydantic/main.py", line 526, in pydantic.main.BaseModel.parse_obj
  File "pydantic/main.py", line 341, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 1 validation error for FooToolSchemaSchema
args_dict
  str type expected (type=type_error.str)
```

### Expected behavior

We expect to see the output of `foo()` which is `"hi there"`.

You can get this result by uncommenting `args_schema=FooSchema` above. This is a problem, because this line below in `StructuredTool.from_function()` https://github.com/hwchase17/langchain/blob/58e95cd11e2c2fc31ed6551b5a2b876143d57429/langchain/tools/base.py#L469 suggests that the schema gets inferred, if not provided one. Instead, what's happening is that the tool "infers" that the arguments involve just one string, which is incorrect.

I don't mind fixing this myself. In that case, any guidance is very welcome. Thank you!
conversationalRetrievalChain - how to set the template

### Issue you'd like to raise.

I am using conversationalRetrievalChain. I cannot seem to change the system template. Any suggestion how to do this? 

`retriever = vectorstore.as_retriever(search_kwargs={"k": source_amount},  qa_template=QA_PROMPT, question_generator_template=CONDENSE_PROMPT)`

`qa = ConversationalRetrievalChain.from_llm(llm=model, retriever=retriever, return_source_documents=True)`

When printing QA:
[SystemMessagePromptTemplate(prompt=PromptTemplate(input_variables=['context'], output_parser=None, partial_variables={}, template="Use the following pieces of context to answer the users question. \nIf you don't know the answer, just say that you don't know, don't try to make up an answer.\n----------------\n{context}", template_format='f-string', validate_template=True), additional_kwargs={}), HumanMessagePromptTemplate(prompt=PromptTemplate(input_variables=['question'],

Whatever I try, I seem to be unable to change the template "Use the following pieces of context to answer..."



### Suggestion:

_No response_
Brute force web research
Labels: name: area: doc loader, name: auto:enhancement

# Research Chain

This is an experimental research chain that tries to answer "researchy" questions using information on the web.

For example, 

```
Compile information about Albert Einstein.
Ignore if it's a different Albert Einstein. 
Only include information you're certain about.

Include:
* education history
* major contributions
* names of spouse 
* date of birth
* place of birth
* a 3 sentence short biography

Format your answer in a bullet point format for each sub-question.
```

Or replace `Albert Einstein` with another person of interest (e.g., John Smith of Boston).


The chain is composed of the following components:

1. A searcher that searches for documents using a search engine.
    - The searcher is responsible to return a list of URLs of documents that
      may be relevant to read to be able to answer the question.
2. A downloader that downloads the documents.
3. An HTML to markdown parser (hard coded) that converts the HTML to markdown.
    * Conversion to markdown is lossy
    * However, it can significantly reduce the token count of the document
    * Markdown helps to preserve some styling information
      (e.g., bold, italics, links, headers) which is expected to help the reader
      to answer certain kinds of questions correctly.
4. A reader that reads the documents and produces an answer.

## Limitations

* Quality of results depends on LLM used, and can be improved by providing more specialized parsers (e.g., parse only the body of articles).
* If asking about people, provide enough information to disambiguate the person.
* Content downloader may get blocked (e.g., if attempting to download from linkedin) -- may need to read terms of service / user agents appropriately.
* Chain can be potentially long running (use initialization parameters to control how many options are explored) -- use async implementation as it uses more concurrency.
* This research chain only implements a single hop at the moment; i.e.,
  it goes from the questions to a list of URLs to documents to compiling answers.
  Without continuing the crawl, web-sites that require pagination will not be explored fully.
* The reader chain must match the type of question. For example, the QA refine chain 
  isn't good at extracting a list of entries from a long document.
  
## Extending

* Continue crawling documents to discover more relevant pages that were not surfaced by the search engine.
* Adapt reading strategy based on nature of question.
* Analyze the query and determine whether the query is a multi-hop query and change search/crawling strategy based on that.
* Break components into tools that can be exposed to an agent. :)
* Add cheaper strategies for selecting which links should be explored further (e.g., based on tf-idf similarity instead of gpt-4)
* Add a summarization chain on top of the individually collected answers.
* Improve strategy to ignore irrelevant information.
Tools: Inconsistent callbacks/run_manager parameter 

### System Info

MacOS Ventura 13.3.1 (a)
python = "^3.9"
langchain = "0.0.185"


### Who can help?

@agola11 @vowelparrot

### Related Components
- Agents / Agent Executors
- Tools / Toolkits
- Callbacks/Tracing

### Reproduction

I want to use the CallbackManager to save some info within a tool. So, as per the [`create_schema_from_function`](https://github.com/hwchase17/langchain/blob/64b4165c8d9b8374295d4629ef57d4d58e9af7c8/langchain/tools/base.py#L99) that is used to create the tool schema, I define the function as: 


```python
def get_list_of_products(
        self, profile_description: str, run_manager: CallbackManagerForToolRun
    ): 
```

Nonetheless, once the tool is run the[ expected parameter](https://github.com/hwchase17/langchain/blob/64b4165c8d9b8374295d4629ef57d4d58e9af7c8/langchain/tools/base.py#L493) in the function's signature is `callbacks`,

```python
new_argument_supported = signature(self.func).parameters.get("callbacks")
```

So the tool can't run, with the error being:

```bash
TypeError: get_list_of_products() missing 1 required positional argument: 'run_manager'
```

This behavior applies to Structured tool and Tool.

### Expected behavior

Either the expected function parameter is set to `run_manager` to replicate the behavior of the [`run` function](https://github.com/hwchase17/langchain/blob/64b4165c8d9b8374295d4629ef57d4d58e9af7c8/langchain/tools/base.py#L256) from the `BaseTool` or a different function is used instead of [`create_schema_from_function`](https://github.com/hwchase17/langchain/blob/64b4165c8d9b8374295d4629ef57d4d58e9af7c8/langchain/tools/base.py#L99) to create a tool's schema expecting the `callbacks` parameter.
DOC: Return Source Documents to Vectorstore Agent

### Issue with current documentation:

Unable to recreate return source documents from prompt in the current [Vectorstore Agent Documentation](https://python.langchain.com/en/latest/modules/agents/toolkits/examples/vectorstore.html). I tried adding `return_source_documents=True` to my `create_vectorstore_agent` (as discussed [here](https://github.com/hwchase17/langchain/issues/4562)) method and explicitly asking for the source document:

`agent_executor.run("What did biden say about ketanji brown jackson is the state of the union address? Show me the source document")`

But this only returns the content of the `answer`, i.e. 
```
{
   "answer":"message returned here.\n",
   "sources":"13421341235123"
}
```

### Idea or request for content:

Would like either a way to link to a custom output parser / memory for this use case ([memory does seem to work out of the box](https://python.langchain.com/en/latest/modules/agents/agent_executors/examples/sharedmemory_for_tools.html)) or a demo of how to configure the underlying tools to force output to string or something.
Localization langchain prompt （zh_CN and agents)
Labels: name: area: agent, name: auto:improvement

# Localization langchain prompt （now is zh_CN and agents)
Create Localization prompt for  zh_CN angets/* and will add more language and whole langchain prompt after ....

all PROMPT translated by gpt-3.5-turbo。



Fixes # (issue)
#5075

## Who can review?

Community members can review the PR once tests pass. Tag maintainers/contributors who might be interested:

  Agents / Tools / Toolkits
   @vowelparrot
Why raise an error in conversation retrieval chain if the chat history is a string?

https://github.com/hwchase17/langchain/blob/64b4165c8d9b8374295d4629ef57d4d58e9af7c8/langchain/chains/conversational_retrieval/base.py#L34

It seems the `_get_chat_history` is building the chat_history string but if the history is already a string then it should.
The check might even be in the BaseConversationalRetrievalChain `_call` methods.

What would be the correct way of using this if the chat_history is already a string?
Text Fragments from text splitters for deep linking with browsers (or compatible systems) to specific text chunks in source documents

### Feature request

Example URL of a Text Fragment to the README of this project that highlights the About: 

https://github.com/hwchase17/langchain#:~:text=About-,%E2%9A%A1,%E2%9A%A1,-Resources

A SO: https://stackoverflow.com/questions/62989058/how-does-text-in-url-works-to-highlight-text

Example of splitter I'm talking about: https://python.langchain.com/en/latest/modules/indexes/text_splitters/examples/markdown.html

It'll be awesome if these text splitters could get the ability to try and generate [Text Fragments](https://web.dev/text-fragments/) for the text that was split up so that a URL could be generated that a user can click through and have the browser auto-scroll to the highlighted fragment. I'm sure the system could also be used outside the browser world for some tooling that could itself scroll to as well as it should be a well developed pattern/algorithm.

The system wouldn't be perfect due to issues such as duplicate text on page, impossible to generate unique split text, but I'm sure most citations would still find it useful.

### Motivation

I'm a little disappointed at the [notion db employee handbook example](https://github.com/hwchase17/notion-qa) where the sources are just filenames. 

What if the info was in a big doc? `Source: Office d0ebcaaa2074442ba155c67a41d315dd.md` ? Eh. How about as an option:

```
Source: Office%20d0ebcaaa2074442ba155c67a41d315dd.md#:~:text=~12%20o%27%20clock%2C%20there%20is%20lunch%20in%20the%20canteen%2C%20free%20of%20cost.%20Jo%C3%ABlle%20is%20in%20charge%20of%20lunch%20%E2%80%94%C2%A0ask%20her%20if%20you%20need%20anything%20(allergies%20for%20example). 
```

[Hyperlink to raw with text fragment](https://github.com/hwchase17/notion-qa/blob/71610847545c97041b93ecb3b19d9746623ce80f/Notion_DB/Blendle's%20Employee%20Handbook%20a834d55573614857a48a9ce9ec4194e3/Office%20d0ebcaaa2074442ba155c67a41d315dd.md#:~:text=~12%20o%27%20clock%2C%20there%20is%20lunch%20in%20the%20canteen%2C%20free%20of%20cost.%20Jo%C3%ABlle%20is%20in%20charge%20of%20lunch%20%E2%80%94%C2%A0ask%20her%20if%20you%20need%20anything%20(allergies%20for%20example).)

Of course, that looks ugly in a terminal, but on a web page where links can be hyperlinks like above, it'll be a much better experience.

edit: Hmm, that link doesn't work very well on GitHub and it's turbolink'd pages.

### Your contribution

I wish, I'm still trying to grasp Langchain itself. I'm particularly interested in Langchain and friends or rivals for Q/A answering and some of my personal hobby's, my work's, and the notion DB example's pages are quite long.
`encoding_kwargs` for InstructEmbeddings
Labels: name: lgtm

# What does this PR do?

Bring support of `encode_kwargs` for ` HuggingFaceInstructEmbeddings`, change the docstring example and add a test to illustrate with `normalize_embeddings`.

Fixes #3605
(Similar to #3914)

Use case:
```python
from langchain.embeddings import HuggingFaceInstructEmbeddings

model_name = "hkunlp/instructor-large"
model_kwargs = {'device': 'cpu'}
encode_kwargs = {'normalize_embeddings': True}
hf = HuggingFaceInstructEmbeddings(
    model_name=model_name,
    model_kwargs=model_kwargs,
    encode_kwargs=encode_kwargs
)
```

## Before submitting

- [x] Tests were added and passed
- [x] Actions are passed

## Who can review?

Community members can review the PR once tests pass. Tag maintainers/contributors who might be interested:

  Models
  - @hwchase17
  - @agola11
Removes duplicated call from langchain/client/langchain.py
Labels: name: lgtm

This removes duplicate code presumably introduced by a cut-and-paste error, spotted while reviewing the code in ```langchain/client/langchain.py```. The original code had back to back occurrences of the following code block:

```
        response = self._get(
            path,
            params=params,
        )
        raise_for_status_with_text(response)
```

Here are the ```make test``` results:

```
================= 693 passed, 52 skipped, 27 warnings in 8.38s =================
```
Issue: Get the verbose messages from chain

### Issue you'd like to raise.

We can get the intermediate messages printed when the verbose is set to True in the chains. But is there a way we can get the intermediate messages from the chain as a return value?


### Suggestion:


Take the code below as example.

```
from langchain import OpenAI, ConversationChain
from langchain.llms import OpenAI
llm = OpenAI(engine="text-davinci-003", temperature=0.9)
conversation = ConversationChain(llm=llm, verbose=True)
conversation.predict(input="How are you?")
conversation.predict(input="I am Ricardo Kaka, what is your name?")
conversation.predict(input="What is the first thing I said to you?")
```
We get the messages below printed in the shell. But I am wondering if there is a way I can get the messages as an return value, something like conversation.verbose_message, or conversation.get_verbose_message()?
DOC: Small typo in the docs, "and" should be removed, and maybe a period in the end would be ok.

### Issue with current documentation:

In https://docs.langchain.com/docs/components/agents/agent-executor the statement:

The agent executor is responsible for calling the agent, getting back **and** action and action input, calling the tool that the action references with the corresponding input, getting the output of the tool, and then passing all that information back into the Agent to get the next action it should take


### Idea or request for content:

The agent executor is responsible for calling the agent, getting back action and action input, calling the tool that the action references with the corresponding input, getting the output of the tool, and then passing all that information back into the Agent to get the next action it should take.
Feature: Qdrant filters supports
Labels: name: lgtm

# Support Qdrant filters

Qdrant has an [extensive filtering system](https://qdrant.tech/documentation/concepts/filtering/) with rich type support. This PR makes it possible to use the filters in Langchain by passing an additional param to both the `similarity_search_with_score` and `similarity_search` methods.

## Who can review?

@dev2049 @hwchase17
Support for Meta ToT

### Feature request

An interesting takeway for Meta TOT - Meta Tree of Thoughts.  aims to enhance the Tree of Thoughts (TOT) language algorithm by using a secondary agent to critique and improve the primary agent's prompts. This innovative approach allows the primary agent to generate more accurate and relevant responses based on the feedback from the secondary agent.

https://github.com/kyegomez/Meta-Tree-Of-Thoughts

I would like add it to the overall offering. If ok, I can pick it up.

### Motivation

Optimization on the continuous feedback loop. 

### Your contribution

I will like to work on this issue.
Feat: Add batching to Qdrant
Labels: name: lgtm

# Add batching to Qdrant

Several people requested a batching mechanism while uploading data to Qdrant. It is important, as there are some limits for the maximum size of the request payload, and without batching implemented in Langchain, users need to implement it on their own. This PR exposes a new optional `batch_size` parameter, so all the documents/texts are loaded in batches of the expected size (64, by default).

The integration tests of Qdrant are extended to cover two cases:
1. Documents are sent in separate batches.
2. All the documents are sent in a single request.

## Who can review?

@dev2049 @hwchase17
Added async _acall to FakeListLLM
Labels: name: lgtm

# Added Async _acall to FakeListLLM

FakeListLLM is handy when unit testing apps built with langchain. This allows the use of FakeListLLM inside concurrent code with [asyncio](https://docs.python.org/3/library/asyncio.html).

I also changed the pydocstring which was out of date.

## Who can review?

@hwchase17 - project lead
@agola11 - async
Allow ElasticVectorSearch#add_texts to explicitely set the _ids

### Issue you'd like to raise.

ElasticVectorSearch currently assigns a uuid as identifier while indexing documents.
This is not idempotent: if we run the code twice duplicates are created.
Also it would be beneficial to be able to insert new docs, update existing ones and ignore unchanged.

### Suggestion:

I propose to check first if _id or id is present in metadata before setting it to a UUID.
LLM from hugging face not working

### Issue with current documentation:

I am trying to use LLM from hugging face as shown in the documentation below, it is working only when write the same prompt in the documentation but when I  am changing it I don't have responses.

https://github.com/hwchase17/langchain/blob/master/docs/modules/models/llms/integrations/huggingface_hub.ipynb

### Idea or request for content:

_No response_
Memory with multi input

### Issue you'd like to raise.

I cant seem to add memory to llmChain which takes in 3 inputs as the input_key param only takes in one input so the program looses context
template = """"
             As a helpful chatbot agent of the company, provide a  answer to the customer query.
             Strictly limit to the information provided.
             {chat_history}
             calculate out of pocket or cost if deductable is available in the user's plan info 
             Question: {query}
             f"Information from user's plan: {plan_info}
             f"Information from company database: {faq_info}
    """

    prompt = PromptTemplate(
        input_variables=["chat_history","query", "plan_info", "faq_info"],
        template=template
    )
    memory = ConversationBufferMemory(memory_key="chat_history", input_key=["query","plan_info","faq_info"])
    chain = LLMChain(llm=model, prompt=prompt,memory=memory)

    return chain.predict(query=query, plan_info=plan_info,faq_info=faq_info)

### Suggestion:

is there a way to get memory working in my case or memory with more than 1 inputs isnt implemented yet?
FinalStreamingStdOutCallbackHandler not working with ChatOpenAI LLM

### System Info

Hi :)

I tested the new callback stream handler `FinalStreamingStdOutCallbackHandler` and noticed an issue with it.
I copied the code from the documentation and made just one change - use `ChatOpenAI` instead of `OpenAI`

### Who can help?

@hwchase17 

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [X] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

`llm = ChatOpenAI(streaming=True, callbacks=[FinalStreamingStdOutCallbackHandler()], temperature=0)` here is my only change
`tools = load_tools(["wikipedia", "llm-math"], llm=llm)`
`agent = initialize_agent(tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=False)`
`agent.run("It's 2023 now. How many years ago did Konrad Adenauer become Chancellor of Germany.")`

### Expected behavior


The code above returns the response from the agent but does not stream it. In my project, I must use the `ChatOpenAI` LLM, so I would appreciate it if someone could fix this issue, please.
MRKL output parser no longer breaks well formed queries
Labels: name: lgtm, name: maintainer-to-merge

# Handles the edge scenario in which the action input is a well formed SQL query which ends with a quoted column

There may be a cleaner option here (or indeed other edge scenarios) but this seems to robustly determine if the action input is likely to be a well formed SQL query in which we don't want to arbitrarily trim off `"` characters

Fixes #5423

## Who can review?

Community members can review the PR once tests pass. Tag maintainers/contributors who might be interested:

For a quicker response, figure out the right person to tag with @

  @hwchase17 - project lead

  Agents / Tools / Toolkits
  - @vowelparrot