delayedkarma/langchain-issues huggingface dataset:
20230922 fix wrong import when launch langchain server with docker-compose since langsmith-0.0.26
Labels: name: auto:bug

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
MatchingEngine does not store metadata in embeddings
Labels: name: area: embeddings, name: area: vector store, name: auto:bug

### System Info

Langchain version : 0.0.297

### Who can help?

@hwchase17 
@agola11 

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

The _add_texts_ function accepts metadata as an input parameter, even though it is not utilized in the embedding creation process.

### Expected behavior

Metadata should be added.

SImilar to what is done for PGVECTOR.
Support weight only quantization with intel-extension-for-transformers
Labels: name: auto:enhancement, name: area: models

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Support weight only quantization with intel-extension-for-transformers.
[Intel® Extension for Transformers](https://github.com/intel/intel-extension-for-transformers) is an innovative toolkit to accelerate Transformer-based models on Intel platforms, in particular effective on 4th Intel Xeon Scalable processor Sapphire Rapids (codenamed [Sapphire Rapids](https://www.intel.com/content/www/us/en/products/docs/processors/xeon-accelerated/4th-gen-xeon-scalable-processors.html)). The toolkit provides the below key features:
 * Seamless user experience of model compressions on Transformer-based models by extending [Hugging Face transformers](https://github.com/huggingface/transformers) APIs and leveraging [Intel® Neural Compressor](https://github.com/intel/neural-compressor)
 * Advanced software optimizations and unique compression-aware runtime.
 * Optimized Transformer-based model packages.
 * [NeuralChat](https://github.com/intel/intel-extension-for-transformers/blob/main/intel_extension_for_transformers/neural_chat), a customizable chatbot framework to create your own chatbot within minutes by leveraging a rich set of plugins and SOTA optimizations.
 * [Inference](https://github.com/intel/intel-extension-for-transformers/blob/main/intel_extension_for_transformers/llm/runtime/graph) of Large Language Model (LLM) in pure C/C++ with weight-only quantization kernels.
 
This PR is an integration of weight only quantization feature with intel-extension-for-transformers.
 * Unit test is in lib/langchain/tests/integration_tests/llm/test_weight_only_quantization.py
Implement Alibaba Tongyi chat model apis. 
Labels: name: auto:enhancement, name: area: models

Hi there
This PR is aim to implement chat model for Alibaba Tongyi LLM model. It contains work below:
1.Implement ChatTongyi chat model in langchain.chat_models.tongyi. Note this is different with tongyi llm model to another PR https://github.com/langchain-ai/langchain/pull/10878.
For detail it implements _generate() and _stream() function in ChatTongyi.
2. Add some examples in chat/tongyi.ipynb. 
3. Add integration test in chat_models/test_tongyi.py 

Note async completion for the Text API is not yet supported.
Dependencies: dashscope. It will be installed manually cause it is not need by everyone.

Happy for feedback on any aspect of this PR @hwchase17 @baskaryan.
typing errors in OllamaEmbeddings
Labels: name: area: embeddings, name: auto:bug

### System Info

`LangChain==0.0.298`, Python==3.11.5

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [X] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

```python
from langchain.embeddings import OllamaEmbeddings

_ = OllamaEmbeddings(model="llama2")
```

Running `mypy==1.5.1` on this file:

```none
a.py:3:5: error: Missing named argument "mirostat" for "OllamaEmbeddings"  [call-arg]
    _ = OllamaEmbeddings(model="llama2")
        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
a.py:3:5: error: Missing named argument "mirostat_eta" for "OllamaEmbeddings"  [call-arg]
    _ = OllamaEmbeddings(model="llama2")
        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
a.py:3:5: error: Missing named argument "mirostat_tau" for "OllamaEmbeddings"  [call-arg]
    _ = OllamaEmbeddings(model="llama2")
        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
a.py:3:5: error: Missing named argument "num_ctx" for "OllamaEmbeddings"  [call-arg]
    _ = OllamaEmbeddings(model="llama2")
        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
a.py:3:5: error: Missing named argument "num_gpu" for "OllamaEmbeddings"  [call-arg]
    _ = OllamaEmbeddings(model="llama2")
        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
a.py:3:5: error: Missing named argument "num_thread" for "OllamaEmbeddings"  [call-arg]
    _ = OllamaEmbeddings(model="llama2")
        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
a.py:3:5: error: Missing named argument "repeat_last_n" for "OllamaEmbeddings"  [call-arg]
    _ = OllamaEmbeddings(model="llama2")
        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
a.py:3:5: error: Missing named argument "repeat_penalty" for "OllamaEmbeddings"  [call-arg]
    _ = OllamaEmbeddings(model="llama2")
        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
a.py:3:5: error: Missing named argument "temperature" for "OllamaEmbeddings"  [call-arg]
    _ = OllamaEmbeddings(model="llama2")
        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
a.py:3:5: error: Missing named argument "stop" for "OllamaEmbeddings"  [call-arg]
    _ = OllamaEmbeddings(model="llama2")
        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
a.py:3:5: error: Missing named argument "tfs_z" for "OllamaEmbeddings"  [call-arg]
    _ = OllamaEmbeddings(model="llama2")
        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
a.py:3:5: error: Missing named argument "top_k" for "OllamaEmbeddings"  [call-arg]
    _ = OllamaEmbeddings(model="llama2")
        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
a.py:3:5: error: Missing named argument "top_p" for "OllamaEmbeddings"  [call-arg]
    _ = OllamaEmbeddings(model="llama2")
        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
```

It can be fixed by using `= None` as the default for the `Optional` fields

### Expected behavior

`mypy` to pass, as there are no type errors present
Issue: unreasonable default template in document retrieval task
Labels: name: auto:improvement, name: area: models

### Issue you'd like to raise.

Current default prompt template named CHAT_PROMPT in `chains/question_answering/stuff_prompt.py` contains a `SystemMessage` and `HumanMessage` combination, In which `SystemMessage` just looks like this:
```python
"""Use the following pieces of context to answer the users question. 
If you don't know the answer, just say that you don't know, don't try to make up an answer.
----------------
{context}"""
```

As far as I know, `SystemMessage` usually intends to be a series of rules which LLM should obey in conversation, such as personality, function, usage etc. It's should stay immutable through whole conversation with users, but there is a significant and continuous transformation in real world 'cause relevant documents being retrieved has been located in the end of prompt. Is there any reason to do so?

### Suggestion:

We can put prompt in the `HumanMessage` as demonstrated below

```python
system_template = """Use the following pieces of context to answer the users question. 
If you don't know the answer, just say that you don't know, don't try to make up an answer.
----------------
"""
messages = [
    SystemMessagePromptTemplate.from_template(system_template),
    HumanMessagePromptTemplate.from_template("{context}\n\n{question}"),
]
CHAT_PROMPT = ChatPromptTemplate.from_messages(messages)
```
"Minimax" object has no field "_client"
Labels: name: auto:bug, name: area: models

### System Info

when I run this command `python3 ./libs/langchain/tests/integration_tests/llms/test_minimax.py`
It always reports this error `ValueError: "Minimax" object has no field "_client"`

After I replaced "_client" with "client", it works.

version is `langchain==0.0.297`


### Who can help?

_No response_

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

python3 ./libs/langchain/tests/integration_tests/llms/test_minimax.py

### Expected behavior

Don't throw erros
Crashes when trying to stream Chat Vertex AI
Labels: name: auto:bug, name: area: models

### System Info

Langchain v0.0.298
Python 3.11.4
Pop!OS 22.04

### Who can help?

@hwchase17 @agol

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

When trying to stream `chat-bison`:

```
  File "/home/hunmonk/.pyenv/versions/lwe/lib/python3.11/site-packages/langchain/chat_models/base.py", line 204, in stream
    for chunk in self._stream(
  File "/home/hunmonk/.pyenv/versions/lwe/lib/python3.11/site-packages/langchain/chat_models/vertexai.py", line 195, in _stream
    responses = chat.send_message_streaming(question.content, **params)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'ChatSession' object has no attribute 'send_message_streaming'
```
As far as I can tell, there is no `send_message_streaming()` method declared *anywhere* in Langchain.

When tryign to stream `codechat-bison`:

```
  File "/home/hunmonk/git/llm-workflow-engine/lwe/backends/api/request.py", line 320, in iterate_streaming_response
    for chunk in self.llm.stream(messages):
  File "/home/hunmonk/.pyenv/versions/lwe/lib/python3.11/site-packages/langchain/chat_models/base.py", line 215, in stream
    raise e
  File "/home/hunmonk/.pyenv/versions/lwe/lib/python3.11/site-packages/langchain/chat_models/base.py", line 204, in stream
    for chunk in self._stream(
  File "/home/hunmonk/.pyenv/versions/lwe/lib/python3.11/site-packages/langchain/chat_models/vertexai.py", line 194, in _stream
    chat = self._start_chat(history, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/hunmonk/.pyenv/versions/lwe/lib/python3.11/site-packages/langchain/chat_models/vertexai.py", line 210, in _start_chat
    return self.client.start_chat(message_history=history.history, **params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: CodeChatModel.start_chat() got an unexpected keyword argument 'message_history'
```
That looks like a Pydantic validation error in the class itself?

### Expected behavior

Streaming these models should work.
Add retry backoff to clarifai calls
Labels: name: area: embeddings, name: auto:improvement

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** Add retry backoff when ClarifAI requests fail
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Optimization by PROmpting (OPRO)
Labels: name: auto:enhancement, name: area: models

### Feature request

[LARGE LANGUAGE MODELS AS OPTIMIZERS](https://arxiv.org/pdf/2309.03409.pdf) is promising method to optimize prompts, to find the best prompts.
Not sure the LangChain is the best tool to apply this method but Hey, it is LangChain :exclamation: 


### Motivation

For workflows that use a stable prompts, this method could significantly improve results. 
So, there is definitely a valid use case for it.

### Your contribution

Not sure I can do it. Sorry.
LocalAI embeddings shouldn't require OpenAI
Labels: name: area: embeddings, name: auto:improvement

### System Info

macOS Ventura 13.5.2, M1

### Who can help?

@mudler 

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [X] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

https://github.com/langchain-ai/langchain/blob/v0.0.298/libs/langchain/langchain/embeddings/localai.py#L197

### Expected behavior

Why does LocalAI embeddings require OpenAI?  I think LocalAI's embeddings has no need for OpenAI, it has a whole embeddings suite: https://localai.io/features/embeddings/

I think it should be directly usable with its [`/embeddings` endpoint](https://github.com/go-skynet/LocalAI/blob/v1.25.0/api/api.go#L190)
`_identifying_params` broken in `ChatAnthropic` class
Labels: name: auto:bug, name: area: models

### System Info

v0.0.298

### Who can help?

@hwchase17 @ago

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Create a `ChatAnthropic` class instance, note that both `_identifying_params` and `dict()` do not output the proper attributes, even though `_default_params` does.

I'd make a PR to fix this, but upon examining the code, it's not clear to me what is causing the bug.

```python
from langchain.chat_models import ChatAnthropic
chat = ChatAnthropic()
chat._default_params
{'max_tokens_to_sample': 256, 'model': 'claude-2'}
chat._identifying_params
{}
chat.dict()
{'_type': 'anthropic-chat'}
```

### Expected behavior

`_identifying_params`and `dict()` should output the correct params.
add async support for llamacpp.py
Labels: name: auto:improvement, name: area: models

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** Added support for llamacpp being used in chains with "a" methods, like apredict, agenerate, etc. Also now it supports `langchain.callbacks.AsyncIteratorCallbackHandler` 
  - **Issue:** https://github.com/langchain-ai/langchain/issues/9865
  - **Dependencies:** -,
  - **Tag maintainer:** @baskaryan
  - **Twitter handle:** -

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Docs: Using SupabaseVectorStore with existing documents 
Labels: name: area: vector store, name: auto:documentation

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
## Description
Adds additional docs on how to use `SupabaseVectorStore` with existing data in your DB (vs inserting new documents each time).
finetuned models on 3.5-turbo use base tokenizer
Labels: name: auto:improvement, name: area: models

**Description:**
The memory summary buffer calls `get_num_tokens_from_messages` from `openai.py`, which relies on some logic using the model name to calculate the number of tokens. This logic is incomplete, in the case of models finetuned through openai's API, which carry a naming pattern along the lines of: `ft:gpt-3.5-turbo-0613:{your_org}::{snapshot_id}`. This PR alters the model-based logic in this function so we can use finetuned models with the summary buffer memory.
 
 **Issue:** 
#10900 

**Tag maintainer:** 
@agola11
@hwchase17
Improve structured chat output parser regex
Labels: name: area: agent, name: auto:improvement

- **Description:** This PR improves the regex in the output parser for the `StructuredChatAgent`. Previously, it failed to match valid json if there was a space after `json` and before the new line, i.e., the follow string won't be matched by the previous regex:
````
```json 
{
  "action": "Tool",
  "action_input": "Tool input."
}
```
````


- **Issue:** N/A,
- **Dependencies:** N/A,
- **Tag maintainer:** don't know,
- **Twitter handle:** solitarypenman
Fix Runnable.transform() for false-y inputs
Labels: name: auto:bug

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
added Cassandra caches to the llm_caching notebook doc
Labels: name: lgtm, name: auto:documentation, name: area: models

This adds a section on usage of `CassandraCache` and `CassandraSemanticCache` to the doc notebook about caching LLMs, as suggested in [this comment](https://github.com/langchain-ai/langchain/pull/9772/#issuecomment-1710544100) on a previous merged PR.

I also spotted what looks like a mismatch between different executions and propose a fix (line 98).

Being the result of several runs, the cell execution numbers are scrambled somewhat, so I volunteer to refine this PR by (manually) re-numbering the cells to restore the appearance of a single, smooth running (for the sake of orderly execution :)