delayedkarma/langchain-issues huggingface dataset:
Confluence document loader not handling pagination correctly anymore (when using Confluence Cloud)
Labels: name: area: doc loader, name: auto:bug, name: auto:improvement

### System Info

LangChain 0.0.317
atlassian-python-api 3.41.3

### Who can help?

@hwchase17 

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [X] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

When playing with documents from Confluence (Cloud hosting in my case), I have noticed that I am getting multiple copies of the same document in my vector store or from my Confluence retriever (custom implementation not in LangChain).  In both cases, the vector store is populated with LangChain's Confluence document loader and my retriever is also using that same document loader.  The problem happens in `ConfluenceLoader.paginate_request()` in the `while` loop until we reach `max_pages`.  In summary, you will get copies of the same documents until you reach `max_pages`.  For example, if your query (I'm using CQL in my use-case) returns 10 documents and `max_pages` is set to `200`, you will get 20 copies of each document.  This also makes the search process much slower.

The previous pagination system of Confluence REST server was leveraging an index controlled by the `start` parameter.  According to this [page](https://developer.atlassian.com/cloud/confluence/change-notice-moderize-search-rest-apis/), it has been deprecated in favor of a cursor system.  It is now recommended to follow `_links.next` instead of relying on the `start` parameter and getting an empty result when documents to be returned have been exhausted.  

This change is now in effect for the Cloud hosting of Confluence, not sure about private deployments.  You can see for yourself by running a script that looks like this:

```python
from atlassian import Confluence

site = "templates" # Public site 
confluence = Confluence(url=f"https://{site}.atlassian.net/")

cql='type=page AND text~"is"'
response_1 = confluence.cql(cql=cql, start=0)
print(response_1) # Easier to see in debug mode
next_start = len(response_1["results"])*1000
response_2 = confluence.cql(cql=cql, start=next_start)
print(response_2) # Notice returned documents are the same as in response_1
```

### Expected behavior

The Confluence document loader should handle properly pagination for both the old (using `start`) and new way (cursor) to maintain backward compatibility for private Confluence deployments.  It should also properly stop when `max_pages` has been reached.
Fixed bug in PGVector database connection
Labels: name: area: vector store, name: auto:bug

Description: This change turns on futures in the PGVector connection. With futures turned on, commits are enabled.
Issue: A bug occurs when you try to create an empty PGVector vector store, rather than creating it from_documents which occurs much more frequently. Under these conditions, you will not be able to commit changes to the database, leading to errors as soon as this is tried.
Dependencies: None
Tag maintainer: @hwchase17
Issue: TokenTextSplitter with local tokenizer ?
Labels: name: area: embeddings, name: auto:bug, name: auto:documentation

### Issue you'd like to raise.

Hello,

I am using langchain offline on a local machine. I'd like to split document over tokens user TokenTextSplitter.

Unfortunately, I can't get to make the class to user a local tokenizer. 

I tried to do 

`text_splitter = TokenTextSplitter(model_name='/my/path/to/my/tokenizer/', chunk_size = 50, chunk_overlap = 10`

like I did for the HuggingFaceEmbedding (and it worked pretty well).

But I get the following error: 
Could not automaticcaly map '/my/path/to/my/tokenizer/' to a tokenizer. Please use 'tiktoken.get_encoding' to explicitly get the tokenizer you expect


Couldn't find any info in the documentation about setup an offline / local tokenizer.


### Suggestion:

_No response_
Running SQLDatabaseChain adds prefix "SQLQuery:\n" infront of returned SQL by LLM, causing invalid query when ran on Database using chain
Labels: name: auto:bug, name: area: models, name: integration: aws

### System Info

Running SQLDatabaseChain with LangChain version 0.0.319 and Snowflake return SQL query which is to be executed on snowflake database in next step. But returned query contains the prefix "SQLQuery:\n" which will broke the whole chain when the query gets executed on snowflake. How to get rid of this "SQLQuery:\n" prefix from query.

Note: Using AWS Bedrock endpoint with Anthropic Claudev2 LLM.

Using default prompt provided in the langchain:

`Given an input question, first create a syntactically correct {dialect} query to run, then look at the results of the query and return the answer. Unless the user specifies in his question a specific number of examples he wishes to obtain, always limit your query to at most {top_k} results. You can order the results by a relevant column to return the most interesting examples in the database.

Never query for all the columns from a specific table, only ask for a the few relevant columns given the question.

Pay attention to use only the column names that you can see in the schema description. Be careful to not query for columns that do not exist. Also, pay attention to which column is in which table.

Use the following format:

Question: Question here
SQLQuery: SQL Query to run
SQLResult: Result of the SQLQuery
Answer: Final answer here
`

Getting following response in debug/verbose mode:

`"text": " SQLQuery:\nSELECT top 5 p.productname, sum(od.quantity) as total_sold\nFROM products p\nJOIN orderdetails od ON p.productid = od.productid \nGROUP BY p.productname\nORDER BY total_sold DESC\n"
}`

Any help would be appreciated.

### Who can help?

_No response_

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Just use SQLDatabaseChain with AWS Bedrock Anthropic Claude 2 to produce this one.

### Expected behavior

Should not prefix  "SQLQuery:\n" in front returned SQL query.
Function get_relevant_docs() returning AttributeError
Labels: name: area: vector store, name: auto:bug

### System Info

MacOS Ventura 13.6
Python 3.10.13
langchain 0.0.306


### Who can help?

@hwchase17
@agola11 

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

```
  llm = AzureOpenAI(temperature=0, model="gpt-35-turbo")
  compressor = LLMChainExtractor.from_llm(llm)
  base_retriever = vectorstores.as_retriever()
  compression_retriever = ContextualCompressionRetriever(base_compressor=compressor, base_retriever=base_retriever)
  compression_retriever.get_relevant_documents("Owner")
```

### Expected behavior

Expected to return the docs compressed from the vectorstore, but I'm getting the `AttributeError: 'str' object has no attribute 'get'`
DOC: Deprecated Import Statement in Documentation for OpenAI
Labels: name: auto:documentation, name: area: models

### Issue with current documentation:

#### **Description**

In various places within the documentation, the import statement is being used:

```python
from langchain.llms import OpenAI
```

However, as hinted in https://github.com/langchain-ai/langchain/commit/779790167e37f49b3eec5d04dfd30b0447d4a32a, this statement has been deprecated. The correct and updated import statement should be:

```python
from langchain.chat_models import ChatOpenAI
```

#### **Problem**

The use of the deprecated import statement leads to problems, particularly when interfacing with Pydantic. Users following the outdated documentation might experience unexpected errors or issues due to this.
system_message in agent_kwargs not updating System Message in AgentType.STRUCTURED_CHAT_ZERO_SHOT_REACT_DESCRIPTION
Labels: name: area: agent, name: auto:bug, name: area: models

### System Info

Langchain Version: 0.0.316 - Python 3.8.10


### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [X] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [X] Agents / Agent Executors
- [X] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

@hwchase17 @agola11 
I am using StructuredTool for multi input support. Below is my initialize_agent

```
    sys_msg = "Assistant’s main duty is to decide which tool to use......"
    system_message = SystemMessage(content=sys_msg)

    agent_executor = initialize_agent(
            agent=AgentType.STRUCTURED_CHAT_ZERO_SHOT_REACT_DESCRIPTION,
            tools=tools,
            llm=llm,
            verbose=True,
            max_iterations=3,
            early_stopping_method='generate',
            handle_parsing_errors=True,
            agent_kwargs = { "system_message": system_message }
    )
```
I dont see the above (sys_msg) as the system_message in the openAI query. Instead I see this in the debug logs
```
2023-10-20 08:54:01 DEBUG    api_version=None data='{"messages": [{"role": "system", "content": "Respond to the human as helpfully and accurately as possible. You have access to the following tools:\\n\\n
```
I believe this is the default system message. How do I change this to my custom system message in AgentType.STRUCTURED_CHAT_ZERO_SHOT_REACT_DESCRIPTION. 

I see a similar ticket for OpenAI functions AgentType : https://github.com/langchain-ai/langchain/issues/6334
I also tried specifying system_message=system_message as discussed in the above ticket. That did not help either


### Expected behavior

The custom system_message that I provide in agent_kwargs should replace the default one going out to openAI.
Takeoff pro support
Labels: name: auto:enhancement, name: area: models

**Description:**
This PR adds support for the [Pro version of Titan Takeoff Server](https://docs.titanml.co/docs/category/pro-features). Users of the Pro version will have to import the TitanTakeoffPro model, which is different from TitanTakeoff.

**Issue:**
Also minor fixes to docs for Titan Takeoff (Community version)

**Dependencies:**
No additional dependencies

 **Twitter handle:** @becoming_blake

@baskaryan @hwchase17
Support other embedder in Hugginface Hub
Labels: name: area: embeddings, name: auto:improvement

### Feature request

At the moment huggingface_hub.py supports only sentence-transformers because there is a validation:

```
if not repo_id.startswith("sentence-transformers"):
                raise ValueError(
                    "Currently only 'sentence-transformers' embedding models "
                    f"are supported. Got invalid 'repo_id' {repo_id}."
                )
```

### Motivation

At the moment there are other higher-performing embedders on the hub, like e5 or bge family.

### Your contribution

I think you should relax the constraint by also including embedders supported by sentence-transformer

```
if not repo_id.startswith(("sentence-transformers", "intfloat", "BAAI")):
                raise ValueError(
                    "Currently only 'sentence-transformers' embedding models "
                    f"are supported. Got invalid 'repo_id' {repo_id}."
                )
```
feat: Add Linearized output to Textract PDFLoader
Labels: name: area: doc loader, name: auto:enhancement, name: area: models, name: integration: aws

### Feature request

Textract released the [LAYOUT](https://docs.aws.amazon.com/textract/latest/dg/layoutresponse.html) feature, which identifies different layout elements like tables, lists, figures, text-paragraphs and titles. This should be used by the AmazonTextractPDFParser to generate a linearized output to improve downstream LLMs accuracy with those hints.

Text output should render tables and key/value pairs and text in reading order for multi-column text and prefix lists with a *, when features like LAYOUT, TABLES, FORMS are passed to the textract call



### Motivation

Improve downstream LLM accuracy

### Your contribution

I'll submit a PR for this feature.
DOC: Improve MarkDown Splitting and Metadata as Part of RAG.
Labels: name: area: embeddings, name: area: doc loader, name: auto:improvement, name: auto:documentation, name: area: models

### Issue with current documentation:

The current documentation for rag does not focus on adding metadata to chunks to make sure before even doing similarity search on vector db only the relevant docs with correct metadata are retrieved from vector db. 
The issue is with open source embedding models having only mostly 512 sequence length. 
Lot of context is lost if correct metadata is not present in each chunk.
This improves ability of local model to  improve results by factor of magnitude. We tested the same with local hosted embedded model and codellama to improve queries on mermaid and currently getting results onpar with gpt4.

### Idea or request for content:

This notebook covers the same.

https://github.com/unoplat/unoplat-lamp/blob/1-mermaid-expert/llm-rag/Mermaid%20Expert%20RAG.ipynb 

The comparison is done with codellama without the KB and gpt4.
Added new feature (new classes) to parse and load pdfs using PyMuPDF from bytes buffer
Labels: name: area: doc loader, name: auto:enhancement

Hi, here is the required info for this pull request.

Description: 
I've added a new feature (new classes) to parse and load pdfs using PyMuPDF from bytes buffer. This is important when the PDF is not a file but is a bytes buffer in memory. Especially, it's useful when working and using Langchain with Spark and Databricks Autoloader. 
I've also added a unit test.

Dependencies: 
No new dependencies is required. Just io library which is in the native python packaging.
Support runnable factories in .configurable_alts()
Labels: name: auto:improvement

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Fix typo in stuff_prompt's system_template
Labels: name: lgtm, name: auto:nit

- **Description:** 

Add missing apostrophe in `user's` in stuff_prompt's system_template. The first sentence in the system template went from:

> Use the following pieces of context to answer the users question.

to

> Use the following pieces of context to answer the user's question.

- **Issue:** 
- **Dependencies:** none
- **Tag maintainer:** @baskaryan
- **Twitter handle:** ojohnnyo

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Fix superfluous Auto-fixing parser documents
Labels: name: auto:documentation

Replace this entire comment with:
  - **Description:** Fix superfluous [Auto-fixing parser](https://python.langchain.com/docs/modules/model_io/output_parsers/output_fixing_parser) docs. Also switching to `langchain.pydantic_v1` from the direct reference to `pydantic`, 
  - **Issue:** N/A,
  - **Dependencies:** N/A,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** @dosuken123 

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
Shadowfaxproject patch 1
Labels: name: area: doc loader, name: auto:improvement

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
The LCEL memory example returns KeyError
Labels: name: area: lcel

### System Info

langchain==0.0.319, mac, python 3.10

### Who can help?

@hwchase17 @agola11 

I'm trying to use this exact example from: https://python.langchain.com/docs/expression_language/cookbook/memory

```
  model = ChatOpenAI()
  prompt = ChatPromptTemplate.from_messages([
      ("system", "You are a helpful chatbot"),
      MessagesPlaceholder(variable_name="history"),
      ("human", "{input}")
  ])

  memory = ConversationBufferMemory(return_messages=True)

  memory.load_memory_variables({})

  chain = RunnablePassthrough.assign(
      memory=RunnableLambda(memory.load_memory_variables) | itemgetter("history")
  ) | prompt | model

  inputs = {"input": "hi im bob"}
  response = chain.invoke(inputs)
```

and getting:
```
  File "/Users/name/.pyenv/versions/3.10.10/lib/python3.10/site-packages/langchain/schema/prompt_template.py", line 60, in <dictcomp>
    **{key: inner_input[key] for key in self.input_variables}
KeyError: 'history'
```

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [X] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Please run the code above.

### Expected behavior

To have it work like the documentation.
Add Async _acall and _astream to Bedrock
Labels: name: auto:enhancement, name: area: models

### Feature request

Currently Bedrock and BedrockChat models do not sypport async calls and streaming.
it would be very useful to have working ChatOpenAI methods like _acall and _astream in Bedrock llms too so that we can use Claude2 and other Bedrock Models in production easily

### Motivation

without async functionalities it is hard to build production level chatbots using Bedrock models, including claude2 which is one of the most desired models

### Your contribution

I am trying myself, but I am having difficulties. If someone can help, it would be very appreciated by many,
Issue: description for the custom tool's output
Labels: name: area: agent, name: auto:question, name: auto:improvement

### Issue you'd like to raise.

In the [documentation](https://python.langchain.com/docs/modules/agents/tools/custom_tools), it's mentioned that expected input parameters could be defined though `args_schema` for the custom tool:

```
class CalculatorInput(BaseModel):
    question: str = Field()
class CustomCalculatorTool(BaseTool):
    name = "Calculator"
    description = "useful for when you need to answer questions about math"
    args_schema: Type[BaseModel] = CalculatorInput
```

The output of my custom function is a complex table with multiple columns and diverse data types. I would like to provide clearer descriptions for each column, including the possible values that can be found in each column. I assume that this way, the agent can utilize the data more effectively. 
Can you please clarify if it is possible to describe the output from the custom tool in a similar manner as `args_schema` for the agent?


### Suggestion:

Would something like this work for the output description? (Of course, the need for such a solution would be for more complex outputs.)

```
class CalculatorInput(BaseModel):
     question: str = Field()

class CalculatorOutput(BaseModel):
     answer: str = Field()

class CustomCalculatorTool(BaseTool):
     name = "Calculator"
     description = "useful for when you need to answer questions about math"
     args_schema: Type[BaseModel] = CalculatorInput
     output_schema: Type[BaseModel] = CalculatorOutput
```

Thank you!`