delayedkarma/langchain-issues huggingface dataset:
ChatMessageChunk concat error
Labels: name: auto:bug, name: area: models

### System Info

- LangChain: 0.0.279
- Python: 3.11.4
- Platform: Linux and MacOS

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [x] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

```python
from langchain.schema.messages import ChatMessageChunk

message = ChatMessageChunk(role="User", content="I am") + ChatMessageChunk(role="User", content=" indeed.")
```
Here is the error info:
```
---------------------------------------------------------------------------
ValidationError                           Traceback (most recent call last)
Cell In[2], line 1
----> 1 message = ChatMessageChunk(role="User", content="I am") + ChatMessageChunk(role="User", content=" indeed.")

File ~/pyenv/venv/lib/python3.11/site-packages/langchain/schema/messages.py:120, in BaseMessageChunk.__add__(self, other)
    115 def __add__(self, other: Any) -> BaseMessageChunk:  # type: ignore
    116     if isinstance(other, BaseMessageChunk):
    117         # If both are (subclasses of) BaseMessageChunk,
    118         # concat into a single BaseMessageChunk
--> 120         return self.__class__(
    121             content=self.content + other.content,
    122             additional_kwargs=self._merge_kwargs_dict(
    123                 self.additional_kwargs, other.additional_kwargs
    124             ),
    125         )
    126     else:
    127         raise TypeError(
    128             'unsupported operand type(s) for +: "'
    129             f"{self.__class__.__name__}"
    130             f'" and "{other.__class__.__name__}"'
    131         )

File ~/pyenv/venv/lib/python3.11/site-packages/langchain/load/serializable.py:74, in Serializable.__init__(self, **kwargs)
     73 def __init__(self, **kwargs: Any) -> None:
---> 74     super().__init__(**kwargs)
     75     self._lc_kwargs = kwargs

File ~/pyenv/venv/lib/python3.11/site-packages/pydantic/v1/main.py:341, in BaseModel.__init__(__pydantic_self__, **data)
    339 values, fields_set, validation_error = validate_model(__pydantic_self__.__class__, data)
    340 if validation_error:
--> 341     raise validation_error
    342 try:
    343     object_setattr(__pydantic_self__, '__dict__', values)

ValidationError: 1 validation error for ChatMessageChunk
role
  field required (type=value_error.missing
```

### Expected behavior

Expected output:
```python
ChatMessageChunk(content='I am indeed.', additional_kwargs={}, role='User')
```
RecursiveUrlLoader doesn't include root URL content
Labels: name: area: doc loader, name: auto:bug

### System Info

```
url = "https://www.wsj.com"
loader = RecursiveUrlLoader(url=url, max_depth=2, extractor=lambda x: Soup(x, "html.parser").text)
docs = loader.load()
```

Docs don't have any data related to `url`. The problem is related to DFS in the codebase. It doesn't handle the root case.



### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

To produce the issue executed the provided code snippet. 

### Expected behavior

RecursiveUrlLoader should hold the the given `url` data along with it's child.
Add a clear() method to VectorStoreRetrieverMemory with FAISS vector store
Labels: name: area: vector store, name: auto:improvement

 **- Description:** In constructing the VectorStoreRetrieverMemory with InMemoryDocstore() like  faiss vectorstore, it may also need to clear the chat_history memory like  ConversationBufferMemory. To clear the memory without reconstruct a new memory, I add a clear() method to VectorStoreRetrieverMemory make it reusable in the same session, and can use memory.clear() to empty the memory vectorstore. < Only fot the FAISS vectorstore>

  **- Issue:** the issue #5817  how to clear VectorStoreRetrieverMemory?
  
  **- Dependencies:** None
  **- Tag maintainer:** 
  **- Twitter handle:** @932196e6a7d042d

  **- Usecase: **

```
from langchain.memory import VectorStoreRetrieverMemory
from langchain.prompts import PromptTemplate
from langchain.docstore import InMemoryDocstore
from langchain.vectorstores import FAISS
import faiss
embedding_size = 1536 # Dimensions of the OpenAIEmbeddings
index = faiss.IndexFlatL2(embedding_size)
embedding_fn = OpenAIEmbeddings().embed_query
vectorstore = FAISS(embedding_fn, index, InMemoryDocstore({}), {})
retriever = vectorstore.as_retriever(search_kwargs=dict(k= 3)) #set the vector store args
memory = VectorStoreRetrieverMemory(retriever=retriever)

# When added to an agent, the memory object can save pertinent information from conversations or used tools
memory.save_context({"input": "My favorite food is pizza"}, {"output": "that's good to know"})
memory.save_context({"input": "My favorite sport is soccer"}, {"output": "..."})
memory.save_context({"input": "I don't the Celtics"}, {"output": "ok"}) 

# Query the relevance history
memory.load_memory_variables({"prompt": "How's the weather "})
## return {'history': "input: My favorite food is pizza\noutput: that's good to know"}


#Clear 
memory.clear()
memory.load_memory_variables({"prompt": "How's the weather "})
## return {"history":}
```


 @baskaryan
How can I append suffix to the prompt when I am using agents to control the output language
Labels: name: area: agent, name: auto:question, name: area: models

### Issue you'd like to raise.

I would like to output the answer in a specific language (e.g. chinese) when I am using agent. But when I tried to do that with the code below, it gives me an error  ```OutputParserException: Could not parse LLM output:```

```
# retrieval qa chain
qa = RetrievalQA.from_chain_type(
    llm=llm,
    chain_type="stuff",
    retriever=vector_store.as_retriever()
)


from langchain.agents import Tool

tools = [
    Tool(
        name='Knowledge Base',
        func=qa.run,
        description=(
            '<some description>'
        )
    )
]


from langchain.agents import initialize_agent

agent = initialize_agent(
    agent='chat-conversational-react-description',
    tools=tools,
    llm=llm,
    verbose=True,
    max_iterations=3,
    early_stopping_method='generate',
    memory=conversational_memory
)

prompt_prefix = f"""请用中文回答"""

agent.agent.llm_chain.prompt += prompt_prefix 

query = "<some question>"
agent(query)
```

Anyone knows how to add custom prompt to agent to enforce the output language?

### Suggestion:

_No response_
HuggingFaceBgeEmbeddings error
Labels: name: area: embeddings, name: auto:bug

### System Info

Windows 10

langchain 0.0.279



### Who can help?

@hwchase17 

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [X] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

from langchain.embeddings import HuggingFaceBgeEmbeddings

model_name = "BAAI/bge-small-en"
model_kwargs = {'device': 'cpu'}
encode_kwargs = {'normalize_embeddings': False}
hf = HuggingFaceBgeEmbeddings(
    model_name=model_name,
    model_kwargs=model_kwargs,
    encode_kwargs=encode_kwargs
)

### Expected behavior

![image](https://github.com/langchain-ai/langchain/assets/38323944/3255d740-75f1-461f-b9cf-941e1a2f9326)

from langchain.embeddings import HuggingFaceBgeEmbeddings

Couldn't find HuggingFaceBgeEmbeddings
Enhance Google search tool SerpApi response
Labels: name: auto:improvement

Enhance SerpApi response which potential to have more relevant output.

<img width="345" alt="Screenshot 2023-09-01 at 8 26 13 AM" src="https://github.com/langchain-ai/langchain/assets/10222402/80ff684d-e02e-4143-b218-5c1b102cbf75">

Query: What is the weather in Pomfret?

**Before:**

> I should look up the current weather conditions.
...
Final Answer: The current weather in Pomfret is 73°F with 1% chance of precipitation and winds at 10 mph.

**After:**

> I should look up the current weather conditions.
...
Final Answer: The current weather in Pomfret is 62°F, 1% precipitation, 61% humidity, and 4 mph wind.

---

Query: Top team in english premier league?

**Before:**

> I need to find out which team is currently at the top of the English Premier League
...
Final Answer: Liverpool FC is currently at the top of the English Premier League.

**After:**

> I need to find out which team is currently at the top of the English Premier League
...
Final Answer: Man City is currently at the top of the English Premier League.

---

Query: Any upcoming events in Paris?

**Before:**

> I should look for events in Paris
Action: Search
...
Final Answer: Upcoming events in Paris this month include Whit Sunday & Whit Monday (French National Holiday), Makeup in Paris, Paris Jazz Festival, Fete de la Musique, and Salon International de la Maison de.

**After:**

> I should look for events in Paris
Action: Search
...
Final Answer: Upcoming events in Paris include Elektric Park 2023, The Aces, and BEING AS AN OCEAN.
Refactor text splitters split and merge
Labels: name: auto:refactor, name: size:XXL

#9843

Replace this entire comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. These live is docs/extras directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17, @rlancemartin.
 -->
Uncomment create_vector_extension in __post_init__ of PGVector class
Labels: name: area: vector store, name: auto:bug

I have the 0-day experience with PGVector in mind. I can't use PGVector.create_vector_extension at all if the `vector` extension is not already created on postgres. If one initializes PGVector without the extension, it fails since __post_init__ tries to create tables and that fails without the extension.

Is there a reason why create_vector_extension is commented from __post_init__? If not then uncommenting it should fix this issue.

#9511 

twitter @skydowx
added llama-chat-content formatter support
Labels: name: auto:improvement, name: area: models

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - Description: added llama-chat model content formatted support, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: not required any new dependency,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. These live is docs/extras directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17, @rlancemartin.
 -->
feat(llms): add model_kwargs to hf tgi
Labels: name: auto:improvement, name: area: models

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. These live is docs/extras directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17, @rlancemartin.
 -->

@baskaryan
Following what we discussed in #9724 and your suggestion, I've added a `model_kwargs` parameter to hf tgi.
Add parser and loader for Azure document intelligence service.
Labels: name: lgtm, name: area: doc loader, name: auto:enhancement

Hi,

this PR contains loader / parser for Azure Document intelligence which is a ML-based service to ingest arbitrary PDFs / images, even if scanned. The loader generates Documents by pages of the original document. This is my first contribution to LangChain. 

Unfortunately I could not find the correct place for test cases. Happy to add one if you can point me to the location, but as this is a cloud-based service, a test would require network access and credentials - so might be of limited help.

Dependencies: The needed dependency was already part of pyproject.toml, no change.
Twitter: feel free to mention @LarsAC on the announcement
add additional model support to chatlitellm
Labels: name: auto:improvement, name: area: models

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. These live is docs/extras directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17, @rlancemartin.
 -->
enable serde retrieval qa with sources
Labels: name: lgtm, name: auto:improvement

#3983 mentions serialization/deserialization issues with both `RetrievalQA` & `RetrievalQAWithSourcesChain`.
`RetrievalQA` has already been fixed in #5818. 

Mimicing #5818, I added the logic for `RetrievalQAWithSourcesChain`.
adding suffixes
Labels: name: auto:improvement, name: area: models

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. These live is docs/extras directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17, @rlancemartin.
 -->

**Description**: Adding support for specifying the suffix to append to HumanMessage, AIMessage and SystemMessage. Certain models, such as Llama2-*-chat, expects a certain way of formatting the prompt, which includes encasing the message in a certain construct (`
 ... 
` for HumanMessage and `
...
` for SystemMessage which cannot __easily__ be achieved in the current state of langchain.

At this point in time I haven't added support for also specifying Beginning-of-String (BOS) and End-of-String (EOS) tokens, as this will likely have to be made at the prompt level and not the message level.

**Issue**:  Not aware of any existing ones and instead of creating one I jumped directly into creating a PR. Let me know if me creating an issue is desirable.

**Dependencies**: None

**Tag maintainer**:

**Twitter handle**:
ConversationalRetrievalChain doesn't work with Azure OpenAI
Labels: name: area: embeddings, name: area: vector store, name: auto:bug, name: area: models

### System Info

Latest

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [X] Embedding Models
- [X] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Use ConversationalRetrievalChain with an Azure OpenAI gpt model

This is the code:

llm = AzureOpenAI(engine=OPENAI_DEPLOYMENT_NAME, model=OPENAI_MODEL, temperature=0.0)
qa = ConversationalRetrievalChain.from_llm(llm, retriever)


I get the following error:

usr/local/lib/python3.10/dist-packages/pydantic/main.cpython-310-x86_64-linux-gnu.so in pydantic.main.BaseModel.__init__()

ValidationError: 1 validation error for LLMChain
llm
  Can't instantiate abstract class BaseLanguageModel with abstract methods agenerate_prompt, apredict, apredict_messages, generate_prompt, invoke, predict, predict_messages (type=type_error)


### Expected behavior

The code executes without any error
ModuleNotFoundError: No module named 'langchain_experimental.data_anonymizer'
Labels: name: auto:bug

### System Info

Langchain 0.0.279 / Langchain experimental 0.0.12/ Python 3.10

### Who can help?

_No response_

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Follow the guideline: https://python.langchain.com/docs/guides/privacy/presidio_data_anonymization

I was able to make it work by downloading the source of the langchain experimental and copy & pasting the missing folder into the local lib folder.
However, this won't be a good option when building a docker image. Why isn't it part of the experimental library 0.0.12?

### Expected behavior

Example codes will work.
SageMaker Endpoints with Streaming Capability
Labels: name: auto:enhancement, name: area: models

### Feature request

With the new announced [support for Streaming in SageMaker Endpoints](https://aws.amazon.com/blogs/machine-learning/elevating-the-generative-ai-experience-introducing-streaming-support-in-amazon-sagemaker-hosting/), Langchain can add a Streaming capability to the `SagemakerEndpoint` class.

We can leverage the code that already exists as part of the blog post and extend where required by using the [boto3](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sagemaker-runtime/client/invoke_endpoint_with_response_stream.html) API

### Motivation

Decreasing latency is important for developers. Streaming is currently available in `OpenAI`, `ChatOpenAI`, `ChatAnthropic`, `Hugging Face Text Generation Inference`, and `Replicate`. Having `SageMaker` opens the possibilities to a wide range of developers that leverage this AWS Service.

### Your contribution

I can work on this feature to start the process with guidance from the core development team and the community.
Added embeddings support for ollama
Labels: name: area: embeddings, name: auto:enhancement

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - Description: Added support for Ollama embeddings
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: N/A
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: @herrjemand

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. These live is docs/extras directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17, @rlancemartin.
 -->

cc  https://github.com/jmorganca/ollama/issues/436
Updated Neptune graph to use boto
Labels: name: auto:improvement

## Description
This PR updates the `NeptuneGraph` class to start using the boto API for connecting to the Neptune service. With boto integration, the graph class now supports authenticating requests using Sigv4; this is encapsulated with the boto API, and users only have to ensure they have the correct AWS credentials setup in their workspace to work with the graph class.

This PR also introduces a conditional prompt that uses a simpler prompt when using the `Anthropic` model provider. A simpler prompt have seemed to work better for generating cypher queries in our testing.

**Note**: This version will require boto3 version 1.28.38 or greater to work.