delayedkarma/langchain-issues huggingface dataset:
Add into chat agent prompt to not prefix json blob with "json"
Labels: name: auto:bug

Fix bug where LLM was prefixing action JSON with the word "json" causing agent to not be able parse the action/action_input in the JSON. 

Response from ChatOpenAI LLM was:
```
json
{
"action": "Calculator",
"action_input": "25% of 300"
}
```

Bot asked me to submit pull request for this fix to issue I filed: https://github.com/hwchase17/langchain/issues/7554

This was breaking the DeepLeaning/Langchain tutorial on Agents: https://learn.deeplearning.ai/langchain/lesson/7/agents

@hinthornw
Adding LLM wrapper for Kobold AI
Labels: name: needs documentation, name: auto:enhancement

<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: adds two wrappers that lets you use Oobabooga's text-generation-webui api or KoboldAI api in lanchain
  - Issue: n/a
  - Dependencies: none extra, just what exists in lanchain
  - Tag maintainer: @baskaryan 
  - Twitter handle: @zanzibased
  
If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access, 
  I'm not sure if I followed this but I have been testing these for a while and using them with my discord langchain chatbot
  2. an example notebook showing its use. 
  https://colab.research.google.com/drive/1iAVfRnxHpwI4DAyD7rNGlKar212w24Uc?usp=sharing

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the same people again.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->
` ValueError: Could not parse output` when using `QAGenerateChain`'s `.apply_and_parse()` method
Labels: name: auto:bug

### System Info

python==3.9.17
langchain==0.0.190

Win 11 64 bit

### Who can help?

@hwchase17 @agol

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [X] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

```python
# Instantiate the chain
example_gen_chain = QAGenerateChain.from_llm(ChatOpenAI())

example_gen_chain.apply_and_parse([{'doc': data[2]}])

[Out]:

ValueError                                Traceback (most recent call last)
Cell In[36], line 1
----> 1 example_gen_chain.apply_and_parse([{'doc': data[2]}])

File ~\anaconda3\envs\nlp-openai-langchain\lib\site-packages\langchain\chains\llm.py:257, in LLMChain.apply_and_parse(self, input_list, callbacks)
    255 """Call apply and then parse the results."""
    256 result = self.apply(input_list, callbacks=callbacks)
--> 257 return self._parse_result(result)

File ~\anaconda3\envs\nlp-openai-langchain\lib\site-packages\langchain\chains\llm.py:263, in LLMChain._parse_result(self, result)
    259 def _parse_result(
    260     self, result: List[Dict[str, str]]
    261 ) -> Sequence[Union[str, List[str], Dict[str, str]]]:
    262     if self.prompt.output_parser is not None:
--> 263         return [
    264             self.prompt.output_parser.parse(res[self.output_key]) for res in result
    265         ]
    266     else:
    267         return result

File ~\anaconda3\envs\nlp-openai-langchain\lib\site-packages\langchain\chains\llm.py:264, in <listcomp>(.0)
    259 def _parse_result(
    260     self, result: List[Dict[str, str]]
    261 ) -> Sequence[Union[str, List[str], Dict[str, str]]]:
    262     if self.prompt.output_parser is not None:
    263         return [
--> 264             self.prompt.output_parser.parse(res[self.output_key]) for res in result
    265         ]
    266     else:
    267         return result

File ~\anaconda3\envs\nlp-openai-langchain\lib\site-packages\langchain\output_parsers\regex.py:28, in RegexParser.parse(self, text)
     26 else:
     27     if self.default_output_key is None:
---> 28         raise ValueError(f"Could not parse output: {text}")
     29     else:
     30         return {
     31             key: text if key == self.default_output_key else ""
     32             for key in self.output_keys
     33         }

ValueError: Could not parse output: QUESTION: What is the fabric composition of the Maine Expedition Shirt with PrimaLoft®?

ANSWER: The fabric composition of the Maine Expedition Shirt with PrimaLoft® is 85% premium wool and 15% nylon.
```

### Expected behavior

Returns parsed output.
Use `func` as a default `coroutine` method in Tool instantiation
Labels: name: auto:improvement

### Feature request

There are tools with `func` but whose implementation of coroutine would be just same. E.g.,
```python
def adder(x, y):
  return x+y

async def aadder(x, y):
  return x+y

adder_tool = Tool(func=adder, coroutine=aadder, ...)
```
I have to define `adder` and `aadder` redundantly.

Of course the logic can be abstracted within the two definitions, but I'd prefer just having the same function and reuse the `func` at the async calling. A possible implementation would look like this

at [this line](https://github.com/hwchase17/langchain/blob/master/langchain/tools/base.py#L453C1-L473C65)
```python
    async def _arun(
        self,
        *args: Any,
        run_manager: Optional[AsyncCallbackManagerForToolRun] = None,
        **kwargs: Any,
    ) -> Any:
        """Use the tool asynchronously."""
        if self.coroutine:
            ...
        elif self.default_coroutine: # introducing some parameter
            return self.run(*args, run_manager=run_manager, **kwargs)
        raise NotImplementedError("Tool does not support async")
```

with this the adding `adder` will look like this:
```python
def adder(x, y):
  return x+y

adder_tool = Tool(func=adder, default_coroutine=True, ...)
```

### Motivation

Simplify function documentations

### Your contribution

I can make a PR as proposed if it's the right approach.
Follow-up Query Recommender Callback
Labels: name: auto:enhancement

### Feature request

Introduce a follow-up query recommender callback to enhance user experience and engagement in case of chatbot use case.
The recommender callback will suggest relevant follow-up queries generated by LLM based on the user's conversation history, facilitating smoother interactions.

The proposed flow is as follows:
- Utilize the configured memory to recommend follow-up queries by analyzing the chat history.
- In the absence of configured memory, leverage the current question and answer to suggest follow-up queries.

Usage: It's up to the user how they want to utilize these recommended queries

### Motivation

The inclusion of this feature would greatly benefit various chatbot use cases.
By suggesting follow-up queries, the chatbot can proactively guide the conversation, helping users navigate complex interactions more efficiently. This feature has the potential to enhance user satisfaction and streamline the overall user experience. While the exact extent of its usefulness may vary, it is a valuable addition that can significantly improve the chatbot's capabilities.

### Your contribution

I can work on this, let me know your thoughts @hwchase17
Using Agent - ChatOpenAI - response can't be parsed because it starts with 'json' for next action (RC identified and fix proposed)
Labels: name: auto:bug

### System Info

LangChain 0.230
python 3.10

From first example of using calculator from: https://learn.deeplearning.ai/langchain/lesson/7/agents

> Entering new  chain...
ACTION:  json
{
  "action": "Calculator",
  "action_input": "25% of 300"
}

The text is what the ChatOutputParser in agents/chat/output_parser.py gets (I prefixed it with ACTION in my print statement).

The word 'json' is now prefixing the JSON blob from the ChatOpenAI LLM which causes the agent to fail. One possible solution that I verified work but not sure if it's the right one is checking if it prefixes with json and just remove that:

            if action.startswith("json"):
                action = action[4:]
                
 This seems to work and lets both the first two examples in the tutorial work. 

### Who can help?

@hwchase17  and @agola11 for LLM/Chat wrappers/Agents.

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [X] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Steps to reproduce behavior.

Run source code from @hwchase17 Deeplearning class: https://learn.deeplearning.ai/langchain/lesson/7/agents

The calculator and wikipedia examples don't run. 

### Expected behavior

The calculator and wikipedia examples work by parsing the "new?" output from ChatOpenAI in the tutorial.
0.0.229 VespaRetriver signature broken
Labels: name: auto:bug

### System Info

langchain 0.0.229

cannot instantiate VespaRetriever (error that it takes only 1 argument but 4 were given)

rolling version back 

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

from langchain.retrievers.vespa_retriever import VespaRetriever

vespa_query_body = {
    "yql": 'select * from abstracts where userQuery() or ({targetHits:1}nearestNeighbor(paragraph_embeddings,q))',
    'input.query(q)': 'embed(q)',
    'query': 'q',
    "hits": '3',
    "ranking": "hybrid",
}
vespa_content_field = "paragraph_embeddings"
retriever = VespaRetriever(app=vespa_app, body=vespa_query_body, content_field=vespa_content_field)

### Expected behavior

retriever should instantiate but does not
Issue: Passing auth object to LLMRequestsChain
Labels: name: auto:improvement

### Issue you'd like to raise.

Accessing many corporate resources requires special authentication, e.g. Kerberos. 
The `requests` library supports passing an auth object, e.g. 
`requests.get(url, auth=HttpNegotiateAuth(), verify=False)` to use SSPI. 

We're able to pass a `requests_wrapper `to `LLMRequestsChain`, but it only allows changing headers, not the actual get method that is used. 

### Suggestion:

Allow for more generic generic wrappers to be passed? Allow passing a requests-compatible auth object?
Update combined.py
Labels: name: lgtm, name: auto:bug

from my understanding, the `check_repeated_memory_variable` validator will raise an error if any of the variables in the `memories` list are repeated. However, the `load_memory_variables` method does not check for repeated variables. This means that it is possible for the `CombinedMemory` instance to return a dictionary of memory variables that contains duplicate values. This code will check for repeated variables in the `data` dictionary returned by the `load_memory_variables` method of each sub-memory. If a repeated variable is found, an error will be raised.

<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the same people again.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->
Importance of VectorStoreToolkit, vectorstore_agent and VectorStoreInfo in document based domain specific question answering
Labels: name: auto:question

### Issue you'd like to raise.

I am trying to query the documents using the below stack

**Langchain+ embedding tool + vectore store + LLM model**

There are many tools and techniques for this in langchain including **load_qa_chain, RetrievalQA,VectorstoreIndexCreator,ConversationalRetrievalChain**. Those are already giving the good results(Not optimal)

But found one more technique including **VectorStoreInfo,VectorStoreToolkit and vectorstore_agent**

What is the advantage/Importance of this pipeline which using **VectorStoreInfo,VectorStoreToolkit and vectorstore_agent** over the other which doesnt follow this pipeline(use any of **load_qa_chain, RetrievalQA,VectorstoreIndexCreator,ConversationalRetrievalChain**)

### Suggestion:

_No response_
feat:add DBGPTRetriever
Labels: name: auto:enhancement, name: auto:documentation

  - Description: 1.add DBGPTRetriever implement 2.update dbgpt.mdx and dbgpt.ipynb for how to use DBGPTRetriever.
  - Issue: None,
  - Dependencies: None,
  - Tag maintainer: @rlancemartin, @eyurtsev, have a look, thanks .

```
    def _get_relevant_documents(
            self, query: str, *, run_manager: CallbackManagerForRetrieverRun
    ) -> List[Document]:
        """Get documents relevant for a query."""
        try:
            from pilot import EmbeddingEngine
        except ImportError:
            raise ImportError(
                "You need to install `pip install db-gpt` to use this retriever."
            )
        embedding_engine = EmbeddingEngine(
            model_name=self.embedding_model_path,
            vector_store_config=self.vector_store_config,
        )
        docs = embedding_engine.similar_search(query, self.top_k)
        return docs
```
openai.error.InvalidRequestError: Resource not found
Labels: name: auto:bug

### System Info

Python 3.9.12
Langchain 0.0.229
OS Linux Mint 21.1

### Who can help?

_No response_

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [X] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

I am following this tutorial on structured output https://python.langchain.com/docs/modules/model_io/output_parsers/structured

I am passing my openai API key from config, I have made sure that it is being passed as I can see the output of the `chat_model` instance...

```python
from langchain.output_parsers import StructuredOutputParser, ResponseSchema
from langchain.prompts import PromptTemplate, ChatPromptTemplate, HumanMessagePromptTemplate
from langchain.chat_models import ChatOpenAI

from config import config

response_schemas = [
    ResponseSchema(name="answer", description="answer to the user's question"),
    ResponseSchema(name="source", description="source used to answer the user's question, should be a website.")
]
output_parser = StructuredOutputParser.from_response_schemas(response_schemas)

format_instructions = output_parser.get_format_instructions()
prompt = PromptTemplate(
    template="answer the users question as best as possible.\n{format_instructions}\n{question}",
    input_variables=["question"],
    partial_variables={"format_instructions": format_instructions}
)

chat_model = ChatOpenAI(temperature=0, openai_api_key=config.OPENAI_API_KEY)
print(chat_model)

prompt = ChatPromptTemplate(
    messages=[
        HumanMessagePromptTemplate.from_template("answer the users question as best as possible.\n{format_instructions}\n{question}")  
    ],
    input_variables=["question"],
    partial_variables={"format_instructions": format_instructions}
)

_input = prompt.format_prompt(question="what's the capital of france?")
print(_input.to_messages())
output = chat_model(_input.to_messages())
print(output)
print(output_parser.parse(output.content))
```

### Expected behavior

The expected output of the code should be

```json
{'answer': 'Paris', 'source': 'https://en.wikipedia.org/wiki/Paris'}
```
Support for sparse tables
Labels: name: auto:improvement

### Feature request

The sql agent should query in a manner that it gets **unique** values as sample data (metadata) instead of all values. Only then will it be able to understand which columns to query from. Otherwise, it might get confused between similar-sounding column names (ex: age, age_group)

### Motivation

Databases are typically very sparse (several columns are null). In such cases, the sql agent will perform poorly. The reason is it uses the InfoSQLDatabaseTool(sql_db_schema) to get all sample rows from the database. If the values are themselves null, then it doesn't get an accurate idea of what each column is supposed to contain. This would affect the query generation and the checking part too.

### Your contribution

I'm not so sure as of now.
Fast load conversationsummarymemory from existing summary
Labels: name: lgtm, name: auto:improvement, name: auto:documentation

  - Description: Adds an optional buffer arg to the memory's from_messages() method. If provided the existing memory will be loaded instead of regenerating a summary from the loaded messages.
 
Why? If we have past messages to load from, it is likely we also have an existing summary. This is particularly helpful in cases where the chat is ephemeral and/or is backed by serverless where the chat history is not stored but where the updated chat history is passed back and forth between a backend/frontend. 

Eg: Take a stateless qa backend implementation that loads messages on every request and generates a response — without this addition, each time the messages are loaded via from_messages, the summaries are recomputed even though they may have just been computed during the previous response. With this, the previously computed summary can be passed in and avoid:
  1) spending extra $$$ on tokens, and 
  2) increased response time by avoiding regenerating previously generated summary.

Tag maintainer: @hwchase17
Twitter handle: https://twitter.com/ShantanuNair
Generating answers from LLM's pretrianed knowledge  base, instead of from the embedded document.
Labels: name: auto:bug

### System Info

langchain==0.0.219
python 3.9

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [X] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

```
import os
import openai
import pinecone
from langchain.document_loaders import DirectoryLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.embeddings.openai import OpenAIEmbeddings
from langchain.vectorstores import Pinecone
from langchain.llms import OpenAI
from langchain.chains.question_answering import load_qa_chain
directory = '/content/data'

def load_docs(directory):
  loader = DirectoryLoader(directory)
  documents = loader.load()
  return documents

documents = load_docs(directory)

def split_docs(documents, chunk_size=1000, chunk_overlap=20):
  text_splitter = RecursiveCharacterTextSplitter(chunk_size=chunk_size, chunk_overlap=chunk_overlap)
  docs = text_splitter.split_documents(documents)
  return docs

docs = split_docs(documents)

embeddings = OpenAIEmbeddings(model_name="ada")
pinecone.init(
    api_key="pinecone api key",
    environment="env"
)
index_name = "langchain-demo"
index = Pinecone.from_documents(docs, embeddings, index_name=index_name)

model_name = "gpt-4"
llm = OpenAI(model_name=model_name)

chain = load_qa_chain(llm, chain_type="stuff")

def get_similiar_docs(query, k=2, score=False):
  if score:
    similar_docs = index.similarity_search_with_score(query, k=k)
  else:
    similar_docs = index.similarity_search(query, k=k)
  return similar_docs

def get_answer(query):
  similar_docs = get_similiar_docs(query)
  answer = chain.run(input_documents=similar_docs, question=query)
  return answer

```

In the above code, If I ask any question it is answered from an outer world other than the document corpus.


### Expected behavior

If I am asked any domain-specific query, it should answer based on the embedded document corpus only. I am not expecting any outer-domain answer.  
If the query is not related to the embedded document store, then it shouldn't answer anything, instead of searching and generating the answer from its own base pretrained knowledge.
Issue: ConfluenceLoader 403 Forbidden Failed to parse Connect Session Auth Token
Labels: name: auto:question

### Issue you'd like to raise.

I am trying to integrate confluence for open ai embedding and vector store using in memory doc-array. 

I am sure this must not be issue Langchain but thought of posting here .  Any pointer would really appreciated.

I created one free trial version account with atlassian.com and trying one POC with confluence pages.

from langchain.document_loaders import ConfluenceLoader . 

loader = ConfluenceLoader(url="https://yogeshdeshmukh.atlassian.net/wiki",

            token="XXXX")

        documentLoaders = documentLoaders + loader.load(space_key="YYYY", include_attachments=False, limit=10)

As per logs it calls 

[https://yogeshdeshmukh.atlassian.net:443](https://yogeshdeshmukh.atlassian.net/) "GET /wiki/rest/api/content?spaceKey=~YYYY&limit=10&status=current&expand=body.storage&type=page HTTP/1.1" 403 None

DEBUG:atlassian.rest_client:HTTP: GET rest/api/content -> 403 Forbidden

DEBUG:atlassian.rest_client:HTTP: Response text -> {"error": "Failed to parse Connect Session Auth Token"}

ERROR:atlassian.confluence:'message'

Traceback (most recent call last):

  File "/Users/ydeshmukh/Library/Python/3.9/lib/python/site-packages/atlassian/confluence.py", line 3122, in raise_for_status

    error_msg = j["message"]

Any idea do i need to provide some additional parameters . I tried with password but was failing later came to know password base  basic auth deprecated hence register for token but that also failing . 


### Suggestion:

_No response_
Reuse the existing collection if configured properly in Qdrant.from_texts
Labels: name: auto:improvement, name: auto:documentation

This PR changes the behavior of `Qdrant.from_texts` so the collection is reused if not requested to recreate it. Previously, calling `Qdrant.from_texts` or `Qdrant.from_documents` resulted in removing the old data which was confusing for many.

@rlancemartin, @eyurtsev