delayedkarma/langchain-issues huggingface dataset:
KuzuGraph not working
Labels: name: auto:bug

### System Info

Name: kuzu
Version: 0.0.10 (latest)

Name: langchain
Version: 0.0.311

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

I am using the example from langchain documentation:
https://python.langchain.com/docs/use_cases/graph/graph_kuzu_qa

This line fails:
graph = KuzuGraph(db)

F```
ile [~/.local/lib/python3.10/site-packages/langchain/graphs/kuzu_graph.py:71](https://file+.vscode-resource.vscode-cdn.net/home/steve/Projects/homelab/homeapi/notebooks/~/.local/lib/python3.10/site-packages/langchain/graphs/kuzu_graph.py:71), in KuzuGraph.refresh_schema(self)
     69 for table in rel_tables:
     70     current_table_schema = {"properties": [], "label": table["name"]}
---> 71     properties_text = self.conn._connection.get_rel_property_names(
     72         table["name"]
     73     ).split("\n")
     74     for i, line in enumerate(properties_text):
     75         # The first 3 lines defines src, dst and name, so we skip them
     76         if i < 3:

AttributeError: 'kuzu._kuzu.Connection' object has no attribute 'get_rel_property_names'
```

### Expected behavior

should work like example
Denodo connector for langchain
Labels: name: auto:enhancement

### Feature request

connect langchain to denodo database. while support for mysql, postgres etc exists, there is currently none for more niche platforms like denodo. Somthing like this 
![image](https://github.com/langchain-ai/langchain/assets/64059206/2603677e-0acd-4ea8-b800-bde0dab11179)

![image](https://github.com/langchain-ai/langchain/assets/64059206/856741e5-4712-4ed0-acbe-e7c086f43aab)


### Motivation

many organizations now use denodo for storing massive datasets , providing denodo connectivity will greatly enhance the reach of llamaindex and add immense value

### Your contribution

I can perform thorough testing of the feature and then update the documentation.
Dead Links in Retrieval-Augmented Generation (RAG) Tutorial
Labels: name: auto:bug, name: auto:documentation

### System Info

N/A (issue pertains to the web tutorial, [Retrieval-augmented generation (RAG)](https://python.langchain.com/docs/use_cases/question_answering/#quickstart))

### Who can help?

@hwchase17 

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

1.  Navigate to [Retrieval-augmented generation (RAG)](https://python.langchain.com/docs/use_cases/question_answering/#quickstart)].

2.  Attempt to access the links provided below:

- "Open in Colab" link, which is intended to direct users to a Colab notebook: [Open in Colab](https://colab.research.google.com/github/langchain-ai/langchain/blob/master/docs/docs/use_cases/question_answering/qa.ipynb)

- "See details in here on the multi-vector retriever for this purpose" link, which should provide more information on the multi-vector retriever: [See details in here on the multi-vector retriever for this purpose](https://python.langchain.com/docs/use_cases/question_answering/docs/modules/data_connection/retrievers/multi_vector)

3. Observe that these links do not lead to the intended resources and instead result in 404 errors or lead to pages that no longer exist.

### Expected behavior

All links in the tutorial lead to active, relevant pages.
createJsonAgent returns answer based on top 5 data only
Labels: name: area: agent, name: area: doc loader, name: auto:bug, name: area: models

### System Info

Subject: Bug Report - Langchain API Query Limitation

I am currently utilizing the following code snippet:

```typescript
let data: JsonObject;
try {
  const jsonFile = await fs.readFileSync(
    './src/openAI/data/formattedResults.json',
    'utf8',
  );
  data = JSON.parse(jsonFile) as JsonObject;
  if (!data) {
    throw new Error('Failed to load JSON spec');
  }
} catch (e) {
  console.error(e);
  return;
}

const toolkit = new JsonToolkit(new JsonSpec(data));
const executor = await createJsonAgent(model, toolkit);
const res = await executor.call({ input: question }).catch((err) => {
  console.log(err);
});
```

While interacting with the Langchain API, I noticed an issue. When I asked any question, it only returned the top few projects as answers. For instance, when I queried the agent, 'How many projects are there?' it correctly identified that there are 289 projects. However, it incorrectly responded with 'There are 5 projects' instead of providing the accurate count.

Upon further investigation, I realized that the Langchain API appears to be limiting the results to the top 5 data entries, rather than considering the entire dataset of 289 projects. This issue is affecting the accuracy of the responses.

I kindly request assistance in resolving this limitation so that the Langchain API can process and return the complete dataset accurately. Please advise on the necessary steps to address this issue and ensure that all relevant data is considered in responses to queries.

Thank you for your prompt attention to this matter.

### Who can help?

@eyurtsev
@hwchase17
@agola11
@dosu-beta

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [X] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [X] Agents / Agent Executors
- [X] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

let data: JsonObject;
try {
  const jsonFile = await fs.readFileSync(
    './src/openAI/data/formattedResults.json',
    'utf8',
  );
  data = JSON.parse(jsonFile) as JsonObject;
  if (!data) {
    throw new Error('Failed to load JSON spec');
  }
} catch (e) {
  console.error(e);
  return;
}

const toolkit = new JsonToolkit(new JsonSpec(data));
const executor = await createJsonAgent(model, toolkit);
const res = await executor.call({ input: question }).catch((err) => {
  console.log(err);
});
 
 
 
<img width="735" alt="image" src="https://github.com/langchain-ai/langchain/assets/82230052/a0d70ea4-b236-4bb7-9450-52838aff97c5">

Totally there were 289 projects 
<img width="178" alt="image" src="https://github.com/langchain-ai/langchain/assets/82230052/881bff40-0ca4-4729-83c5-83f2fe982fe9">


### Expected behavior

The agent should take into consideration all the data, rather than top few data
Fix: Sitemap Document Loader Tests and Documentation
Labels: name: lgtm, name: area: doc loader, name: auto:documentation

**Description:**
While working on the Docusaurus site loader #9138, I noticed some outdated docs and tests for the Sitemap Loader. 

**Issue:** 
This is tangentially related to #6691 in reference to doc links. I plan on digging in to a few of these issue when I find time next.
load_qa_chain succeeds where load_qa_with_sources_chain fails to return result
Labels: name: area: embeddings, name: area: vector store, name: auto:bug, name: area: models

### System Info

Windows 10

Python 3.10.11

I am using FAISS as the vectorstore, HuggingFaceEmbeddings, and the llm is HuggingFaceHub with google/flan-t5-small. There are no changes to any of these elements when running with and without 'with_sources'.

All libraries were installed within the last week.

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

embeddings = HuggingFaceEmbeddings()
db = FAISS.load_local("FAISS-DB", embeddings)
llm = HuggingFaceHub(repo_id="google/flan-t5-small", model_kwargs={"temperature": 0.2, "max_length": 256})
query = "what does the company sodexo provide?"

similarity_docs = db.similarity_search(query)
chain = load_qa_chain(llm, chain_type="stuff")
results = chain.run(input_documents=similarity_docs, question=query)

print(query)
print(results)

similarity_docs = db.similarity_search(query)
chain = load_qa_with_sources_chain (llm, chain_type="stuff")
results = chain.run(input_documents=similarity_docs, question=query)

print(query)
print(results)

------------------------------
Output:
what does the company sodexo provide?
integrated food, facilities management and other services
what does the company sodexo provide?
0

[similarity_docs.txt](https://github.com/langchain-ai/langchain/files/12918308/similarity_docs.txt)


### Expected behavior

The expected result was the same answer with the source, similar to {'answer': 'integrated food, facilities management and other services', 'source': 'Inputs\\bodyExtract_71756.txt'}
Fix HuggingFace notebook link
Labels: name: lgtm, name: auto:documentation

  - **Description:** While reading the docs (https://python.langchain.com/docs/integrations/providers/huggingface), I noticed the notebook linked in https://python.langchain.com/docs/use_cases/evaluation/huggingface_datasets.html was giving back 404. I made a search in the docs to see whether it was available, so this PR updates the link in the docs.
  - **Issue:** I haven't opened an issue for this change.
  - **Dependencies:** -
  - **Tag maintainer:** -,
  - **Twitter handle:** -
default user agent is added in recursive url loader header
Labels: name: area: doc loader, name: auto:bug

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** Experiencing a 403 Forbidden error when using the RecursiveUrlLoader on certain websites, and that this issue is resolved when we manually set a User-Agent in the headers. The WebBaseLoader in LangChain has a default User-Agent set in the session headers but RecursiveUrlLoader doesn't have default User-Agent. Updated  RecursiveUrlLoader code to include default user-agent, if it is not provided.
  - **Issue:** This fixes 403 Forbidden error on certain websites. This code update will solve the issue mentioned in the link https://github.com/langchain-ai/langchain/issues/11540,https://github.com/langchain-ai/langchain/issues/11541.
  - **Dependencies:** fake_useragent library is needed,
  - **Tag maintainer:** @baskaryan, @eyurtsev, @hwchase17,
Issue: When using ChatGLM, I can't connect to the endpoint_url.
Labels: name: auto:bug, name: auto:question, name: area: models

### Issue you'd like to raise.

When I use the chatglm code from the official documentation, I can't connect to the endpoint URL provided there.

The official website is: https://python.langchain.com/docs/integrations/llms/chatglm

The specific code is as follows：

          from langchain.llms import ChatGLM
          from langchain.prompts import PromptTemplate
          from langchain.chains import LLMChain
          
          template = """{question}"""
          prompt = PromptTemplate(template=template, input_variables=["question"])
          
          endpoint_url = "http://127.0.0.1:8000"
          
          llm = ChatGLM(
              endpoint_url=endpoint_url,
              max_token=80000,
              history=[["我将从美国到中国来旅游，出行前希望了解中国的城市", "欢迎问我任何问题。"]],
              top_p=0.9,
              model_kwargs={"sample_model_args": False},
          )
          
          llm_chain = LLMChain(prompt=prompt, llm=llm)
          
          question = "北京和上海两座城市有什么不同？"
          
          llm_chain.run(question)


The code is encountering the following error：

        ValueError: Error raised by inference endpoint: HTTPConnectionPool(host='127.0.0.1', port=8000): Max retries exceeded with 
url: / (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x00000163A6691F70>: Failed to establish a new connection: [WinError 10061] 由于目标计算机积极拒绝，无法连接。'))

I have tried many methods I found online, but they didn't have much effect. I'm not sure if it's necessary to open a specific server through cmd to connect to 127.0.0.1:8000.

### Suggestion:

_No response_
lancedb integration improvements
Labels: name: area: vector store, name: auto:improvement

  - **Description:** 
  Hi, I'm from lancedb team.
 - Improves lancedb integration by making it easier to use - now you don't require to create tables manually and pass it in the constructor, although that is still backwards compatible.
 - Bug fix - pandas was being used even though its not a dependency for lancedb or langchain

  - **Tag maintainer:** 
  @baskaryan

  - **Twitter handle:** @loldedxd
Issue: Not clear through docs: how we can pass variale to the input_variable in prompt 
Labels: name: auto:question, name: auto:documentation, name: area: models

### Issue you'd like to raise.

hi  @dosu-bot , @dosu-beta  : 
I am not able to figure out how we can pass variale to the input_variable in prompt : 

prompt_with_loader = PromptTemplate(
    input_variables=["query", "username", "password"],
    template=
    """
    Based on the instruction you are tasked to construct a Python script that performs the following action: {query}. Utilize the specifics in the given API documentation to understand the required endpoint, method, and parameters.

    Credentials for authentication:
    Username: {username}
    Password: {password}

    Please create a Python script using the 'requests' library to execute the desired action. Ensure the script includes:
    - Parsing of the necessary details from the API specification
    - Basic Authentication with the provided username and password (encoded in Base 64)
    - Proper use of the relevant HTTP method, headers, body, and parameters as per the API specification
    - print the response of the api
    - Error handling for the request
    """
)
chain_type_kwargs = {"prompt": prompt_with_loader} , 

**qa_chain = RetrievalQAWithSourcesChain.from_chain_type(
        llm=llm,
        chain_type="stuff",
        retriever=retriever,
        chain_type_kwargs=chain_type_kwargs,
        return_source_documents=False,
        verbose=True
        )**. // when i am goining to hit this block it gives me this ERRR :  document_variable_name summaries was not found in llm_chain input_variables: ['query', 'username', 'password'] (type=value_error)   
        
        pls can you guide us here 
        
        

### Suggestion:

_No response_
S3 Directory Loader not working as expected 
Labels: name: area: doc loader, name: auto:bug, name: integration: aws

### System Info

langchain :  0.0.306
python : 3.10.12
platform : Ubuntu 

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [X] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

`loader = S3DirectoryLoader(
        bucket="", prefix="data/",
        aws_access_key_id="",
        aws_secret_access_key="")
data = loader.load()`

It should have load all the files with in the bucket from a given prefix which may contain multiple sub-folders but it does not 

### Expected behavior

It should have loaded all the files as list of Document of langchain schema , but it does not
ConfluenceLoader's keep_newlines is not passed correctly
Labels: name: area: doc loader, name: auto:bug

### System Info

LangChain version: v0.0.314

### Who can help?

@eyurtsev @baskaryan 

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [X] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

```python
    from langchain.document_loaders import ConfluenceLoader

    loader = ConfluenceLoader(url="xxx", session=get_session())
    docs = loader.load(page_ids=["xxx"], keep_newlines=True)
    for doc in docs:
        print(doc.page_content)
        have_newlines = "\n" in doc.page_content
        print(f"{have_newlines=}")
```

read the source code, I found the `keep_newlines` arguments are not passed to `process_page` function when load with `cql` and `page_ids` 
file: `langchain.document_loaders.confluence.py`
function: `Confluence.load`
lines: 308-353

<img width="781" alt="image" src="https://github.com/langchain-ai/langchain/assets/19882756/1d11d7dd-f883-4782-8b5d-3365ca2dff87">
<img width="821" alt="image" src="https://github.com/langchain-ai/langchain/assets/19882756/b87f3a16-65a4-490c-851f-0c51a534d171">


### Expected behavior

The `keep_newlines` parameter is passed correctly and empty lines are correctly retained
Encapsulate alicloud pai-eas access method for chatmodels and llms
Labels: name: auto:enhancement, name: area: models

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
### Description: 
To provide an eas llm service access methods in this pull request by impletementing `PaiEasEndpoint` and `PaiEasChatEndpoint` classes in `langchain.llms` and `langchain.chat_models` modules. Base on this pr, langchain users can build up a chain to  call remote eas llm service and get the llm inference results.

### About EAS Service
EAS is a Alicloud product on Alibaba Cloud Machine Learning Platform for AI which is short for AliCloud PAI. EAS provides model inference deployment services for the users. We build up a llm inference services on EAS with a general llm docker images. Therefore, end users can quickly setup their llm remote instances to load majority of the hugginface llm models, and serve as a backend for most of the llm apps. 

### Dependencies
This pr does't involve any new dependencies.
Adapt to the latest version of Alibaba Cloud OpenSearch vector store API
Labels: name: area: vector store, name: auto:improvement

Hello Folks,

Alibaba Cloud OpenSearch has released a new version of the vector storage engine, which has significantly improved performance compared to the previous version. At the same time, the sdk has also undergone changes, requiring adjustments alibaba opensearch vector store code  to adapt.

This PR includes:

Adapt to the latest version of Alibaba Cloud OpenSearch API.
More comprehensive unit testing.
Improve documentation.

I have read your contributing guidelines. And I have passed the tests below

- [x] make format
- [x]  make lint
- [x]  make coverage
- [x]  make test
Issue: Getting Error while integrating confluence attachment
Labels: name: area: doc loader, name: auto:bug

### Issue you'd like to raise.

I'm encountering an error while trying to integrate confluenece attachment by the keyword include_attachments=True
Created a chunk of size 494, which is longer than the specified 100
Created a chunk of size 230, which is longer than the specified 100
Created a chunk of size 121, which is longer than the specified 100
Created a chunk of size 224, which is longer than the specified 100
Created a chunk of size 103, which is longer than the specified 100
Created a chunk of size 638, which is longer than the specified 100
Created a chunk of size 112, which is longer than the specified 100
Created a chunk of size 228, which is longer than the specified 100
Created a chunk of size 147, which is longer than the specified 100
Created a chunk of size 127, which is longer than the specified 100
Created a chunk of size 306, which is longer than the specified 100
Created a chunk of size 105, which is longer than the specified 100

Below is my code:
text_splitter = CharacterTextSplitter(chunk_size=100, chunk_overlap=0)
texts = text_splitter.split_documents(documents)
text_splitter = TokenTextSplitter(chunk_size=1000, chunk_overlap=10, encoding_name="cl100k_base")
texts = text_splitter.split_documents(texts)

### Suggestion:

_No response_
Issue: Getting Error while integrating confluence attachment
Labels: name: area: doc loader, name: auto:bug

### Issue you'd like to raise.

I'm encountering an error while trying to integrate confluenece attachment by the keyword include_attachments=True
Created a chunk of size 494, which is longer than the specified 100
Created a chunk of size 230, which is longer than the specified 100
Created a chunk of size 121, which is longer than the specified 100
Created a chunk of size 224, which is longer than the specified 100
Created a chunk of size 103, which is longer than the specified 100
Created a chunk of size 638, which is longer than the specified 100
Created a chunk of size 112, which is longer than the specified 100
Created a chunk of size 228, which is longer than the specified 100
Created a chunk of size 147, which is longer than the specified 100
Created a chunk of size 127, which is longer than the specified 100
Created a chunk of size 306, which is longer than the specified 100
Created a chunk of size 105, which is longer than the specified 100

Below is my code:
    text_splitter = CharacterTextSplitter(chunk_size=100, chunk_overlap=0)
    texts = text_splitter.split_documents(documents)
    text_splitter = TokenTextSplitter(chunk_size=1000, chunk_overlap=10, encoding_name="cl100k_base")
    texts = text_splitter.split_documents(texts)

### Suggestion:

_No response_