delayedkarma/langchain-issues huggingface dataset:
Using Open source LLM models in SQL Chain
Labels: name: auto:bug, name: auto:question

### Using Open source LLM models in SQL Chain

Is it possible to use open source LLM models in SQL chain ?
I have tried using tapex/Flan models in SQL Chain, but getting a serialization error on dict[] classes.

Error:
```
 File "pydantic\main.py", line 341, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 1 validation error for SQLDatabaseChain
__root__ -> llm
  **_value is not a valid dict (type=type_error.dict)_**
```

Are there any samples/snippets are available for using open source LLM models in SQL Chain ?

Sample code snippet I tried that is throwing the error:
```
tokenizer = TapexTokenizer.from_pretrained("microsoft/tapex-base-finetuned-wtq")
model = BartForConditionalGeneration.from_pretrained("microsoft/tapex-base-finetuned-wtq")
chain = SQLDatabaseChain(llm=model, database=db, verbose=True)
chain.run("context query ?")
```

### Suggestion:

_No response_
SelfQueryRetriever, Add option to return all when user asks
Labels: name: auto:improvement

### Feature request

I tested the enable_limit to True and asking thins like
1. Get 3 residents who were born in xxx
2.  Get 5 Residens who were bon in xxx

It works pretty well.

However in my use case, users can ask also:  List all the residents who were born in xxx. 
When questions like this are done, by default it will return only 4 documents and not all.

![image](https://github.com/hwchase17/langchain/assets/6962857/aaeec974-56d3-47c4-90eb-9bde4782cb22)


### Motivation

My use case requires sometimes to list all documents that match the criteria, not only 4. 

### Your contribution

I am a beginner in langchain (only used it for 2 months), so not sure where in the code this can be fixed, but with the proper guidance I should be able to contribute (if somebody is willing to guide me)
RetrievalQAWithSourcesChain acall does not write fromatted prompt to terminal if verbose=True
Labels: name: auto:bug

### System Info

python==3.10
langchain==0.0.169

### Who can help?

@agola11 

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [X] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [X] Async

### Reproduction

Steps to reproduce:
1. Open the [example notebook](https://colab.research.google.com/drive/1ut3LVSSxsN_C52Pn1ceqWdHjSzhxuZol?usp=sharing)  
2. Replace ```insert API key here```with your API key  
3. Run all cells

### Expected behavior

Asynchronously calling the RetrievalQAWithSourcesChain with ```chain_type_kwargs = {"prompt": prompt, "verbose": True}``` should result in the same terminal output as the synchronous version instead of skipping "Prompt after formatting: ..."

```result = chain(query)``` output:

```
> Entering new StuffDocumentsChain chain...


> Entering new LLMChain chain...
Prompt after formatting:
PROMPT_AFTER_FORMATTING

> Finished chain.

> Finished chain.
```


Wrong ```result = await chain.acall(query)``` output:
```
> Entering new StuffDocumentsChain chain...


> Entering new LLMChain chain...

> Finished chain.

> Finished chain.
```
fix: openapi controller prompt, when bot is unable to resolve an api …
Labels: name: lgtm, name: auto:bug

…call, it needs retry

<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the same people again.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->
Specific name of the current chain is not displayed
Labels: name: auto:bug

### System Info

LangChain v0.0.229, Python v3.10.12, Ubuntu 20.04.2 LTS

### Who can help?

@hwchase17 @agola11 

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [X] Callbacks/Tracing
- [ ] Async

### Reproduction

I am encountering an issue where the specific name of the current chain is not being displayed in the console output, even though I have set 'verbose=True' in the MultiPromptChain and other Chains. When the program enters a new chain, it only prints 'Entering new  chain...' without specifying the name of the chain. This makes it difficult to debug and understand which chain is currently being used. Could you please look into this issue and provide a way to display the name of the current chain in the console output? Thank you.

The output could be
```
> Entering new  chain...


> Entering new  chain...
lib/python3.10/site-packages/langchain/chains/llm.py:275: UserWarning: The predict_and_parse method is deprecated, instead pass an output parser directly to LLMChain.
  warnings.warn(

> Finished chain.
math: {'input': 'What is the derivative of a function?'}

> Entering new  chain...
Prompt after formatting:
You are a very good mathematician. You are great at answering math questions. \nYou are so good because you are able to break down hard problems into their component parts, \nanswer the component parts, and then put them together to answer the broader question.
Here is a question:
What is the derivative of a function?

> Finished chain.

> Finished chain.
```

### Expected behavior

```
> Entering new MultiPromptChain chain...


> Entering new LLMRouterChain chain...
lib/python3.10/site-packages/langchain/chains/llm.py:275: UserWarning: The predict_and_parse method is deprecated, instead pass an output parser directly to LLMChain.
  warnings.warn(

> Finished chain.
math: {'input': 'What is the derivative of a function?'}

> Entering new LLMChain[math] chain...
Prompt after formatting:
You are a very good mathematician. You are great at answering math questions. \nYou are so good because you are able to break down hard problems into their component parts, \nanswer the component parts, and then put them together to answer the broader question.
Here is a question:
What is the derivative of a function?

> Finished chain.

> Finished chain.
```
AttributeError: 'str' object has no attribute 'upsert' in Pinecone Hybrid Search
Labels: name: auto:bug

### System Info

0.0.228

### Who can help?

@lbsnrs

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Following tutorial here:
https://python.langchain.com/docs/modules/data_connection/retrievers/integrations/pinecone_hybrid_search

```
bm25_encoder = BM25Encoder().default()
embed = OpenAIEmbeddings(deployment=OPENAI_EMBEDDING_DEPLOYMENT_NAME, model=OPENAI_EMBEDDING_MODEL_NAME, chunk_size=1)
 
retriever = PineconeHybridSearchRetriever(
        embeddings=embed, sparse_encoder=bm25_encoder, index="cubigometadatanotindexed"
)
retriever.add_texts(["foo", "bar", "FirstName0003384 is a guy", "FirstName0003381 is a girl"])
result = retriever.get_relevant_documents("Who is FirstName0003381?")
```
 
I get this error:

```

AttributeError: 'str' object has no attribute 'upsert'
Traceback:
File "C:\Users\xx\anaconda3\envs\xxChatbotv3\Lib\site-packages\streamlit\runtime\scriptrunner\script_runner.py", line 552, in _run_script
    exec(code, module.__dict__)
File "C:\Users\xx\repos\xxChatbotv1\app\pages\Pinecone Hybrid Search.py", line 116, in <module>
    main()
File "C:\Users\xx\repos\xxChatbotv1\app\pages\Pinecone Hybrid Search.py", line 112, in main
    retriever.add_texts(["foo", "bar", "FirstName0003384 is a guy", "hello"])
File "C:\Users\xx\anaconda3\envs\zzChatbotv3\Lib\site-packages\langchain\retrievers\pinecone_hybrid_search.py", line 121, in add_texts
    create_index(
File "C:\Users\xx\anaconda3\envs\zzChatbotv3\Lib\site-packages\langchain\retrievers\pinecone_hybrid_search.py", line 98, in create_index
    index.upsert(vectors)
    ^^^^^^^^^^^^
```

### Expected behavior

The texts should be added to the index without error
Adds OpenAI functions powered document metadata tagger
Labels: name: lgtm, name: auto:enhancement, name: auto:documentation

Adds a new document transformer that automatically extracts metadata for a document based on an input schema. I also moved `document_transformers.py` to `document_transformers/__init__.py` to group it with this new transformer - it didn't seem to cause issues in the notebook, but let me know if I've done something wrong there.

Also had a linter issue I couldn't figure out:

```
MacBook-Pro:langchain jacoblee$ make lint
poetry run mypy .
docs/dist/conf.py: error: Duplicate module named "conf" (also at "./docs/api_reference/conf.py")
docs/dist/conf.py: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#mapping-file-paths-to-modules for more info
docs/dist/conf.py: note: Common resolutions include: a) using `--exclude` to avoid checking one of them, b) adding `__init__.py` somewhere, c) using `--explicit-package-bases` or adjusting MYPYPATH
Found 1 error in 1 file (errors prevented further checking)
make: *** [lint] Error 2
```

@rlancemartin @baskaryan
Lost in the middle: We have been ordering documents the WRONG way. (for long context)
Labels: name: auto:improvement, name: auto:documentation

Motivation, it seems that when dealing with a long context and "big" number of relevant documents we must avoid using out of the box score ordering from vector stores.
See: https://arxiv.org/pdf/2306.01150.pdf

So, I added an additional parameter that allows you to reorder the retrieved documents so we can work around this performance degradation. The relevance respect the original search score but accommodates the lest relevant document in the middle of the context.
Extract from the paper (one image speaks 1000 tokens):
![image](https://github.com/hwchase17/langchain/assets/1821407/fafe4843-6e18-4fa6-9416-50cc1d32e811)
This seems to be common to all diff arquitectures. SO I think we need a good generic way to implement this reordering and run some test in our already running retrievers.
It could be that my approach is not the best one from the architecture point of view, happy to have a discussion about that.
For me this was the best place to introduce the change and start retesting diff implementations.

@rlancemartin, @eyurtsev
Add Google GenAppBuilder as Tools
Labels: name: needs documentation, name: auto:enhancement

<!-- Thank you for contributing to LangChain!

  - Description: Add the support of Google AppBuilder as tools.
  - Dependencies: None
  - Twitter handle: @MiquelJulien

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the same people again.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->
Minor update to reference other sql tool by tool names instead of hard coded string.
Labels: name: lgtm, name: auto:improvement


<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the same people again.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->

Currently there are 4 tools in SQL agent-toolkits, and 2 of them have reference to the other 2.

This PR change the reference from hard coded string to `{tool.name}`
Update chat agents' output parser to extract action by regex
Labels: name: needs test, name: auto:improvement

<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the same people again.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->

Currently `ChatOutputParser` extracts actions by splitting the text on "```", and then load the second part as a json string.

But sometimes the LLM will wrap the action in markdown code block like:

````markdown
```json
{
  "action": "foo",
  "action_input": "bar"
}
```
````

Splitting text on "```" will cause `OutputParserException` in such case.

This PR changes the behaviour to extract the `$JSON_BLOB` by regex, so that it can handle both ` ``` ``` ` and ` ```json ``` `

@hinthornw
Glob patterns not finding documents when using it as an argument to DirectoryLoader
Labels: name: auto:bug

### System Info

I'm using langchain 0.0.218 in python 3.10.0 and when I use glob patterns as a direct argument to initialize the class this does not load anything. e.g. DirectoryLoader(path = root_dir + 'data', glob = "**/*.xml") 

But when I use it in loader_kwargs it works perfect. 
e.g. DirectoryLoader(path = path, loader_kwargs={"glob":"**/*.xml"}
May this be a bug that when class is initialized in the line?https://github.com/hwchase17/langchain/blob/master/langchain/document_loaders/directory.py#L33
It seems to always be set as "**/[!.]*" when using it as an arg but not when using it inside loader_kwargs

### Who can help?

@eyurtsev 

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [X] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

1. Make a directory called data
2. Inside that directory store all kind of supported documents  (docx, text, etc) excepting -for example- xml files and also a folder that only contains all the xml files 
3.  Use loader = Directoryloader = (path = root_dir + 'data', glob = "**/*.xml") 
5. execute loader.load() will not load any documents

Then use loader = DirectoryLoader(path = path, loader_kwargs={"glob": "**/*.xml"} 
loader.load() and will work perfectly

### Expected behavior

Must work using it like loader = Directoryloader(path = root_dir + 'data', glob = "**/*.xml") 

*NOTE* This happens with all kind of glob patterns passed through glob argument. It does not has to do with the file extension or something. 

Let me know if you need more info :)
only add handlers if they are new
Labels: name: auto:improvement

When using callbacks, there are times when callbacks can be added redundantly: for instance sometimes you might need to create an llm with specific callbacks, but then also create and agent that uses a chain that has those callbacks already set. This means that "callbacks" might get passed down again to the llm at predict() time, resulting in duplicate calls to the `on_llm_start` callback. 

For the sake of simplicity, I made it so that langchain never adds an exact handler/callbacks object in `add_handler`, thus avoiding the duplicate handler issue.

Tagging @hwchase17 for callback review


<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the same people again.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->
HuggingFaceEndpoint incorrectly assumes that the endpoint returns the prompt prepended to the response
Labels: name: auto:bug

  - Description: HuggingFaceEndpoint truncates the text because it assumes the endpoint returns the prompt together with generated text (but that's not the case, so the the code wrongly truncates the answer)
  - Issue: Fixes #7353 
  - Dependencies: None
  - Tag maintainer: @baskaryan
  - Twitter handle: @juanan

This issue has also been discussed here:
https://huggingface.co/tiiuae/falcon-40b-instruct/discussions/51
MAKE langchain AN ORGANIZATION
Labels: name: auto:enhancement

### Issue you'd like to raise.

It is soooooo **weird** that this repo is still under a personal GitHub account 😕

At this moment in time (1:45pm 7-10-2023)
https://github.com/lang-chain is still available. 

It would feel more professional if this repo became an organization. 

### Suggestion:

Convert this personal repo to an organizations repo.
Fix AsyncFinalIteratorCallbackHandler's token yield issue with streaming enabled
Labels: name: auto:bug

When utilizing the AsyncFinalIteratorCallbackHandler in conjunction with streaming functionality, the handler.aiter() failed to yield new tokens that were being generated.

@agola11

The code snippet provided below highlights the issue:

```python
async def wrap_done(fn: Awaitable, event: asyncio.Event):
    try:
        await fn
    except Exception:
        print("Exception in wrap_done")
    finally:
        event.set()

async def create_query_handler(query: str) -> AsyncIterable[str]:
    tools = []
    handler = AsyncFinalIteratorCallbackHandler()
    llm = ChatOpenAI(temperature=0, model="gpt-3.5-turbo", verbose=True, streaming=True, callbacks=[
        handler
    ])
    agent = initialize_agent(tools=tools, llm=llm, agent=AgentType.OPENAI_FUNCTIONS, verbose=True)
    template = PromptTemplate.from_template("".join([
        "You're are Jeff. Please respond to my queries and then laugh at me"  
    ]))

    task = asyncio.create_task(wrap_done(
        agent.arun(template.format(query=query, podcast_name=namespace)),
        handler.done)
    )

    async for token in handler.aiter():
        print(token, end="")
        yield f"data: {token}\n\n"

    await task
```