delayedkarma/langchain-issues huggingface dataset:
Support extra tools argument for pandas agent toolkit
Labels: name: area: agent, name: lgtm, name: auto:improvement

**Description** 

We support adding new tools in some toolkits already like the [SQLAgent toolkit](https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/agents/agent_toolkits/sql/base.py#L27). 

Related [SO](https://stackoverflow.com/questions/76583163/are-langchain-toolkits-able-to-be-modified-can-we-add-tools-to-a-pandas-datafra) thread
This replicates the same functionality here, so users can add custom bespoke tools.
add HTMLHeaderTextSplitter
Labels: name: lgtm, name: area: doc loader, name: auto:enhancement

Description: Similar in concept to the `MarkdownHeaderTextSplitter`, the `HTMLHeaderTextSplitter` is a "structure-aware" chunker that splits text at the element level and adds metadata for each header "relevant" to any given chunk. It can return chunks element by element or combine elements with the same metadata, with the objectives of (a) keeping related text grouped (more or less) semantically and (b) preserving context-rich information encoded in document structures. It can be used with other text splitters as part of a chunking pipeline.

Dependency: lxml python package

Maintainer: @hwchase17

Twitter handle: @MartinZirulnik
`LlamaCppEmbeddings`: adds `verbose` parameter, similar to `llms.LlamaCpp` class
Labels: name: lgtm, name: area: embeddings, name: auto:improvement

## Description

As of now, when instantiating and during inference, `LlamaCppEmbeddings` outputs (a lot of) verbose when controlled from Langchain binding - it is a bit annoying when computing the embeddings of long documents, for instance.

This PR adds `verbose` for `LlamaCppEmbeddings` objects to be able **not** to print the verbose of the model to `stderr`. It is natively supported by `llama-cpp-python` and directly passed to the library – the PR is hence very small.

The value of `verbose` is `True` by default, following the way it is defined in [`LlamaCpp` (`llamacpp.py` #L136-L137)](https://github.com/langchain-ai/langchain/blob/c87e9fb2ce0ae617e3b2edde52421c80adef54cc/libs/langchain/langchain/llms/llamacpp.py#L136-L137)

## Issue

_No issue linked_

## Dependencies

_No additional dependency needed_

## To see it in action

```python
from langchain.embeddings import LlamaCppEmbeddings

MODEL_PATH = "<path_to_gguf_file>"

if __name__ == "__main__":
    llm_embeddings = LlamaCppEmbeddings(
        model_path=MODEL_PATH,
        n_gpu_layers=1,
        n_batch=512,
        n_ctx=2048,
        f16_kv=True,
        verbose=False,
    )
```
Call to langchain.vectorstores.pinecone.add_texts needs to use deepcopy for metadata
Labels: name: area: embeddings, name: area: vector store, name: auto:bug, name: stale, name: integration: pinecone

### System Info

LangChain 0.0.295
Python 3.11.5
pinecone-client 2.2.4


### Who can help?

@hwchase17 @agola11 

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [X] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction


I'm calling add_texts to add texts and their metadata to Pinecone. Many of my texts have identical metadata ({"chapter": chapter}), so I created a metadata object and passed that in for each to metadatas as its metadata, when calling add_texts.

`            # For each text, associate the chapter number as metadata
            metadata = {"chapter": chapter}

            # Add document embeddings to vector store first
            try:
                doc_ids = vector_store_service.add_texts(
                    doc_texts,
                    metadatas=[metadata for _ in doc_texts],
                    index_name=index_name,
                    namespace=namespace,
                )
 `

However, add_texts manipulates the items within metadatas, expecting them to be isolated objects. This includes writing a "text" metadata field.

https://api.python.langchain.com/en/latest/_modules/langchain/vectorstores/pinecone.html#Pinecone.add_texts

Because the texts I passed in were referencing the same metadata object that was passed in, when the LangChain code updated the "metadata" for that particular text, including adding the _text_key, that effectively overwrote the values for all of the texts that shared that same metadata object.

The result was that all texts which shared a metadata object that was input ended up having the text of the final text that was processed.


### Expected behavior

I worked around this on the client side by changing my metadatas argument so that the items in the list do not reference a shared object. It would be my expectation that inside add_texts, the function would use something like deepcopy to ensure that modifying one metadata (or text, for that matter) does not affect any of the others, which is a weird side effect.
Support for PDF in WebResearchRetriever (plus sources fix)
Labels: name: area: doc loader, name: auto:improvement

**Description:**
-  `WebResearchRetriever` would fail on PDF documents.  To address this I have created a new `async_pdf` document loader and integrated with `WebResearchRetriever`.  There is a lot of similarity with `async_html`, and certainly opportunity to share code via base class (or consolidate both into a single `async_web` class).  
- The `async_pdf` class has its own PDF download code, rather than use `PyPDFLoader` directly to download the remote file.  This is because `PyPDFLoader` does not currently support timeouts or optional SSL verify (no timeouts can result in infinite hang).
- Added a new method that inspects the content-type of the remote URL and only keeps known types.  This should reduce content decoding errors
- Also added a small fix for missing sources coming from `RetrievalQAWithSourcesChain`.  The regex in `_split_sources` would get confused if there is a newline between "SOURCES:" and the source strings in the answer.

**Issues:**  https://github.com/langchain-ai/langchain/issues/11031, https://github.com/langchain-ai/langchain/issues/10575
WebResearchRetriever fails on PDF files
Labels: name: area: doc loader, name: auto:bug, name: auto:question

### Discussed in https://github.com/langchain-ai/langchain/discussions/8630

<div type='discussions-op-text'>

<sup>Originally posted by **matthewosm** August  2, 2023</sup>
Hi I'm using WebResearchRetriever to do materials research, unfortunately the top results are often PDFs so I get the error:

`langchain.document_loaders.async_html:Failed to decode content from xyz.pdf`

I've seen there is a [PDF Loader](https://python.langchain.com/docs/modules/data_connection/document_loaders/pdf), but I'm not quite sure how to get it to work with my script.

Here is what I have so far:

```
import faiss, logging
from langchain.chains import RetrievalQAWithSourcesChain
from langchain.chat_models import ChatOpenAI
from langchain.retrievers.web_research import WebResearchRetriever
from langchain.vectorstores import FAISS 
from langchain.embeddings.openai import OpenAIEmbeddings
from langchain.docstore import InMemoryDocstore  
from langchain.utilities import GoogleSearchAPIWrapper

embedding_model = OpenAIEmbeddings()
index = faiss.IndexFlatL2(1536)
vectorstore = FAISS(embedding_model.embed_query, index, InMemoryDocstore({}), {})

llm = ChatOpenAI(model_name="gpt-3.5-turbo-16k", temperature=0.5, streaming=True)

search = GoogleSearchAPIWrapper()   

web_retriever = WebResearchRetriever.from_llm(vectorstore=vectorstore, llm=llm, search=search, num_search_results=6)

question = "Define THIS MATERIAL"

if question:
    logging.basicConfig()
    logging.getLogger("langchain.retrievers.web_research").setLevel(logging.INFO)    
    qa_chain = RetrievalQAWithSourcesChain.from_chain_type(llm, retriever=web_retriever)

    result = qa_chain({"question": question})
    answer, sources = result['answer'], result['sources']
    print(answer, sources)
```</div>
DOC: What's Serialized format of ChatPromptTemplate from langchain hub?
Labels: name: auto:documentation, name: stale, name: area: models

### Issue with current documentation:

There's no example on serialized format of chat template:
https://python.langchain.com/docs/modules/model_io/prompts/prompt_templates/prompt_serialization

Yet langchain hub already offer such template
https://smith.langchain.com/hub/homanp/superagent

What would be a serialized yaml file?

### Idea or request for content:

An example of serialized chat template
SearchApi integration
Labels: name: area: agent, name: auto:enhancement

Based on the customers' requests for native langchain integration, SearchApi is ready to invest in AI and LLM space, especially in open-source development.

- This is our initial PR and later we want to improve it based on customers' and langchain users' feedback. Most likely changes will affect how the final results string is being built. 
- We are creating similar native integration in Python and JavaScript.
- The next plan is to integrate into Java, Ruby, Go, and others.
- Feel free to assign @SebastjanPrachovskij as a main reviewer for any SearchApi-related searches. We will be glad to help and support langchain development. 

### Usage

```python
from langchain.utilities import SearchApiAPIWrapper
from langchain.llms.openai import OpenAI
from langchain.agents import initialize_agent, Tool
from langchain.agents import AgentType

import os
os.environ["OPENAI_API_KEY"] = ""
os.environ["SEARCHAPI_API_KEY"] = ""

llm = OpenAI(temperature=0)
search = SearchApiAPIWrapper()
tools = [
    Tool(
        name="Intermediate Answer",
        func=search.run,
        description="useful for when you need to ask with search"
    )
]

self_ask_with_search = initialize_agent(tools, llm, agent=AgentType.SELF_ASK_WITH_SEARCH, verbose=True)
self_ask_with_search.run("Who lived longer: Plato, Socrates, or Aristotle?")
```

### Output

```
> Entering new AgentExecutor chain...
 Yes.
Follow up: How old was Plato when he died?
Intermediate answer: eighty
Follow up: How old was Socrates when he died?
Intermediate answer: | Socrates | 
| -------- | 
| Born | c. 470 BC Deme Alopece, Athens | 
| Died | 399 BC (aged approximately 71) Athens | 
| Cause of death | Execution by forced suicide by poisoning | 
| Spouse(s) | Xanthippe, Myrto | 

Follow up: How old was Aristotle when he died?
Intermediate answer: 62 years
So the final answer is: Plato

> Finished chain.
'Plato'
```
Add last_edited_time and created_time props to NotionDBLoader
Labels: name: lgtm, name: area: doc loader, name: auto:improvement

# Description

Adds logic for NotionDBLoader to correctly populate `last_edited_time` and `created_time` fields from [page properties](https://developers.notion.com/reference/page#property-value-object).

There are no relevant tests for this code to be updated.
Issue: <Retrieval QA returns empty result>
Labels: name: auto:bug, name: area: models

### Issue you'd like to raise.

I have a simple Retrieval QA chain that used to work proprerly. Here's an example : 
![image](https://github.com/langchain-ai/langchain/assets/48955402/7242d7b9-e11a-4a00-8070-bf53b7d6c708)

But now, without any modifications to the code, it always returns me an empty string as a result : 
![image](https://github.com/langchain-ai/langchain/assets/48955402/9a71c72e-e7f8-4b37-a7bf-1298f0ef3811)

Any idea why ?

### Suggestion:

_No response_
Issue: How to use custom prompt template with retrievalqa chain
Labels: name: auto:question, name: area: models, name: integration: chroma

### Issue you'd like to raise.

The goal is to use custom prompt template with retrievalqa chain. I have included my code below. You can see that PromptTemplate includes the input variables ['context', 'persona', 'question'], but I can't figure out where to pass in the 'persona' variable. Some examples of a  'persona' variable would be "Pirate" or "Shakespeare" to get the LLM to respond like a pirate or Shakespeare. The 'persona' variable is a simplification of the real question, which is how to use custom input variables and custom PromptTemplate with RetrievalQA chain. 

This may stem from not understanding RetrievalQA chain under the hood. You'll notice that when I call the chain, I need to pass in only the "query" parameter (which is not an input variable from the custom promptTemplate). I don't understand this as well: where do the input variables "context" and "question" go?

```
from langchain.embeddings.openai import OpenAIEmbeddings
from langchain.vectorstores import Chroma
from langchain.chains import RetrievalQA
from langchain.document_loaders import PyPDFLoader
from langchain.chat_models import ChatOpenAI
from langchain.text_splitter import TokenTextSplitter
from langchain import PromptTemplate

import os
os.environ["OPENAI_API_KEY"] = OPENAI_API_KEY

# Define the LLM
# llm = ChatOpenAI(temperature=0,model_name="gpt-4")
llm = ChatOpenAI(temperature=0, model_name="gpt-3.5-turbo")

# Data Ingestion
loader = PyPDFLoader("../data/Batman_wiki.pdf")
pages = loader.load()

# Chunk and Embeddings
text_splitter = TokenTextSplitter(chunk_size=100, chunk_overlap=0)
texts = text_splitter.split_documents(pages)

embeddings = OpenAIEmbeddings()

# Vector Store
db = Chroma.from_documents(documents=texts, embedding=embeddings)

# Create Prompt
template = """Use the following pieces of context to answer the question at the end. 
If you don't know the answer, just say that you don't know.
Don't try to make up an answer.
{context}

Respond in the persona of {persona}

Question: {question}
Answer: 
"""

prompt = PromptTemplate.from_template(template)

# Initialise RetrievalQA Chain
chain = RetrievalQA.from_chain_type(
    llm,
    retriever=db.as_retriever(),
    return_source_documents=True,
    chain_type_kwargs={"prompt": prompt},
    # persona="Shakespeare"
)

response = chain({"query": question, "persona": "Shakespeare"})

print(response) 
```

### Suggestion:

_No response_
Make pairwise comparison chain more like LLM as a judge 
Labels: name: auto:enhancement, name: area: models

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:**: Adds LLM as a judge as an eval chain
  - **Tag maintainer:** @hwchase17 

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Fix issue where requests wrapper passes auth kwarg twice
Labels: name: auto:bug

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->

Closes #8842
Fix issue #10985 - Skip model.to(device) if it is instantiated with bitsandbytes config
Labels: name: auto:bug, name: area: models

Preventing error caused by attempting to move the model that was already loaded on the GPU using the Accelerate module to the same or another device. It is not possible to load model with Accelerate/PEFT to CPU for now

Addresses: [#10985](https://github.com/langchain-ai/langchain/issues/10985)
Support new version of tiktoken that are working with langchain (tag "^0.3.2" => "">=0.3.2,<0.6.0" and python "^3.9" =>">=3.9")
Labels: name: auto:improvement

  - **Description:**
       be able to use langchain with other version than tiktoken 0.3.3 i.e 0.5.1
  - **Issue:**
      cannot installed the conda-forge version since it applied all optional dependency:
       https://github.com/conda-forge/langchain-feedstock/pull/85  
      replace "^0.3.2" by "">=0.3.2,<0.6.0" and "^3.9" by  python=">=3.9"  
      Tested with python 3.10, langchain=0.0.288 and tiktoken==0.5.0
adding argument support for SupabaseVectorStore's postgres functions.
Labels: name: area: vector store, name: auto:improvement

  - **Description:** `SupabaseVectorStore` has some limitations using postgres functions since is limited to a function with only one parameter `query_embedding`, this PR intends to add as many arguments as you need, hence allowing to create functions with other parameters than `query_embedding`,  this PR also allows the use of `postgres_filter` from `SupabaseVectorStore`.
Usage
```python
vector_store = SupabaseVectorStoreMixin(embedding=embeddings, client=supabase, table_name="document", query_name="match_documents_mmr")
matched_docs = vector_store.max_marginal_relevance_search(query, k=2, fetch_k=5, sql_fn_args={
    "stack": "python", # stack is a parameter in the supabase postgres' function.
})
```
  - **Issue:** #10994 ,
  - **Dependencies:** None,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** AHardReset
<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** `SupabaseVectorStore` has some limitations using postgres functions since is limited to a function with only one parameter `query_embedding`, this PR intends to add as many arguments as you need, hence allowing to create functions with other parameters than `query_embedding`,  this PR also allows the use of `postgres_filter` from `SupabaseVectorStore`.
Usage
```python
vector_store = SupabaseVectorStoreMixin(embedding=embeddings, client=supabase, table_name="document", query_name="match_documents_mmr")
matched_docs = vector_store.max_marginal_relevance_search(query, k=2, fetch_k=5, sql_fn_args={
    "stack": "python", # stack is a parameter in the supabase postgres's function.
})
```
  - **Issue:** #10994 ,
  - **Dependencies:** None,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** AHardReset

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
UnstructuredMarkdownLoader  load 报错 Ran out of input 
Labels: name: area: doc loader, name: auto:bug, name: stale

### System Info

from langchain.document_loaders import DirectoryLoader, UnstructuredMarkdownLoader
from langchain.text_splitter import MarkdownTextSplitter

try:

  loader_pdf = DirectoryLoader('data/', glob="**/*.md", loader_cls=UnstructuredMarkdownLoader)
  documents = loader_pdf.load()

  print(documents)
  # loader = UnstructuredMarkdownLoader(filePath)
 # loader.load() 

except Exception as e:
  print(e)

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [X] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

任意的md 文件 都无法加载

### Expected behavior

能够获取到md 文件 load 获取非结构化数据
support add_embeddings for elasticsearch
Labels: name: area: vector store, name: auto:improvement

  - **Description:** Provide a way to use different text for embedding.
      -  For example, if you are ingesting stack-overflow Q&As for RAG, you would want to embed the questions and return the answer(s) for the hits. With this change, the consumer of langchain can implement that easily.
      - I noticed the similar function is added on faiss.py with #1912 which was for performance reason, but I see the same function can be used to achieve what I thought. So instead of changing Document class to have embedding_content, I mimicked the implementation of faiss.py.
      - The test should provide some guidance on how to use it. It would be more intuitive if I just pass texts and embedding_texts as separate arguments, but I chose to use `zip`-ed object for the consistency with faiss.py implementation.
      - I plan to make similar pull request for OpenSearch.
  - **Issue:** N/A
  - **Dependencies:** None other than the existing ones.
Issue: <This model's maximum context length is 4097 tokens, however you requested 4226 tokens>
Labels: name: auto:bug, name: auto:question, name: stale, name: area: models

### Issue you'd like to raise.

![image](https://github.com/langchain-ai/langchain/assets/75439567/70de0078-caa6-4dc1-902d-c911e6aa1c8e)


### Suggestion:

I am using model chatgpt 3.5 . But I got errror . Why i get this error . Can U help me to find solution for it . data in my database is so big . So upgrading model version is not santisfactory.
add new test file
Labels: name: area: vector store, name: auto:improvement

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->