delayedkarma/langchain-issues huggingface dataset:
Issue: How to return retrieval texts?
Labels: name: auto:question, name: area: models

### Issue you'd like to raise.

Friends, how can I  return the retrieval texts(context)? My script is as follows:
```python
from langchain.document_loaders import PyPDFDirectoryLoader
from langchain.embeddings import OpenAIEmbeddings
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.vectorstores import FAISS
from langchain.schema import Document
from langchain.chat_models.openai import ChatOpenAI
from langchain.prompts import ChatPromptTemplate
from langchain.schema.output_parser import StrOutputParser
from langchain.schema.runnable import  RunnablePassthrough

pdf_path = '/home/data/pdf'
pdf_loader = PyPDFDirectoryLoader(pdf_path)
docs = pdf_loader.load()

splitter = RecursiveCharacterTextSplitter(chunk_size=256,chunk_overlap=0)
split_docs = splitter.split_documents(docs)

docsearch = FAISS.from_documents(split_docs,OpenAIEmbeddings())
retriever = docsearch.as_retriever()

template = """Answer the question based only on the following context:
{context}

Question: {question}
"""
prompt = ChatPromptTemplate.from_template(template)

model = ChatOpenAI(temperature=0.2)

# RAG pipeline
chain = (
    {"context": retriever, "question": RunnablePassthrough()}
    | prompt
    | model
    | StrOutputParser()
)
print(chain.invoke("How to maintain the seat belts in the Jingke model?"))
``` 

### Suggestion:

_No response_
create_csv_agent UnicodeDecodeError('utf-8' )
Labels: name: area: agent, name: auto:bug

### System Info

python

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

from langchain import OpenAI
from langchain.chains import AnalyzeDocumentChain
from langchain.chains.summarize import load_summarize_chain
from langchain_experimental.agents.agent_toolkits import create_csv_agent
from langchain.chains.question_answering import load_qa_chain
from langchain.chat_models import ChatOpenAI
from langchain.agents import AgentType

model = ChatOpenAI(model="gpt-4") # gpt-3.5-turbo, gpt-4

agent = create_csv_agent(model,"APT.csv",verbose=True)

agent.run("how many rows are there?")

### Expected behavior

UnicodeDecodeError                        Traceback (most recent call last)
[<ipython-input-25-0a5f8e8f5933>](https://localhost:8080/#) in <cell line: 11>()
      9 model = ChatOpenAI(model="gpt-4") # gpt-3.5-turbo, gpt-4
     10 
---> 11 agent = create_csv_agent(model,"APT.csv",verbose=True)
     12 
     13 agent.run("how many rows are there?")

11 frames
/usr/local/lib/python3.10/dist-packages/pandas/_libs/parsers.pyx in pandas._libs.parsers.raise_parser_error()

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb1 in position 0: invalid start byte
Use List instead of list
Labels: name: lgtm, name: auto:nit

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->

Unify List usages in libs/langchain/langchain/text_splitter.py, only one place it's `list`, all other ocurrences are `List`
langchain.vectorstores.Chroma support for EmbeddingFunction.__call__ update of ChromaDB
Labels: name: area: embeddings, name: area: vector store, name: auto:bug, name: integration: chroma

### System Info

This was working fine for my previous configuration,
langchain v0.0.225
chromadb v0.4.7

But now neither this is working, nor the latest version  of both
langchain v0.0.336
chromadb v0.4.17

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [X] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

I have the packages installed

Running these pieces of code
```
from langchain.document_loaders import TextLoader
from langchain.indexes import VectorstoreIndexCreator

loader = TextLoader(file_path)
index = VectorstoreIndexCreator().from_loaders([loader])  # this is where I am getting the error
```

OR 

```
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.vectorstores import Chroma
from langchain.embeddings.openai import OpenAIEmbeddings

text_splitter = RecursiveCharacterTextSplitter()
splits = text_splitter.split_documents(docs)
embedding = OpenAIEmbeddings()
vectordb = Chroma.from_documents(  # this is where I am getting the error
            documents=splits,
            embedding=embedding,
        )
```

Here is the error
```
Expected EmbeddingFunction.__call__ to have the following signature: odict_keys(['self', 'input']), got odict_keys(['self', 'args', 'kwargs'])\nPlease see https://docs.trychroma.com/embeddings for details of the EmbeddingFunction interface.\nPlease note the recent change to the EmbeddingFunction interface: https://docs.trychroma.com/migration#migration-to-0416---november-7-2023 \n
```

### Expected behavior

Earlier a chromadb instance would be created, and I would be able to query it with my prompts. That is the expected behaviour.
Request new feature for Robotic Application
Labels: name: area: memory, name: auto:question, name: auto:enhancement, name: area: models

### Feature request

Hi Guys,

Here I made a new fork to [https://github.com/MetaSLAM/CyberChain](https://github.com/MetaSLAM/CyberChain), where I would like to combine the powerful Langchain ability and GPT4 into the real-world robotic challenges. 

My question raised here is mainly about:

1. How can I setup the Chat GPT 4 version into Langchain, where I would like to levevarge the powerful visual inference of GPT4;
2. I there any suggestion for the memory system, because the robot may travel through a large-scale environment for lifelong navigation, I would like to construct a memory system within langchain to enhance its behaviour in the long-term operation.

Many thanks for your hard work, and Langchain is difinitely an impressive work.

Max

### Motivation

Combine real-world robotic application with the LangChain framework.

### Your contribution

I will provide my research outcomes under our MetaSLAM organization (https://github.com/MetaSLAM), hope this can benefit both robotics and AI communitry, targeting for the general AI system.
model init ValueError
Labels: name: auto:bug, name: area: models

### System Info

langchain version: 0.0.336
mac
python3.8

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

```py
import langchain
from langchain.chat_models.minimax import MiniMaxChat
print(langchain.__version__)
chat = MiniMaxChat(minimax_api_host="test_host", minimax_api_key="test_api_key", minimax_group_id="test_group_id")
assert chat._client
assert chat._client.host == "test_host"
assert chat._client.group_id == "test_group_id"
assert chat._client.api_key == "test_api_key"
```

output:

```sh
0.0.336
Traceback (most recent call last):
  File "/Users/hulk/code/py/workhome/test_pydantic/test_langchain.py", line 4, in <module>
    chat = MiniMaxChat(minimax_api_host="test_host", minimax_api_key="test_api_key", minimax_group_id="test_group_id")
  File "/Users/hulk/miniforge3/envs/py38/lib/python3.8/site-packages/langchain/llms/minimax.py", line 121, in __init__
    self._client = _MinimaxEndpointClient(
  File "pydantic/main.py", line 357, in pydantic.main.BaseModel.__setattr__
ValueError: "MiniMaxChat" object has no field "_client"
```

### Expected behavior

Instantiation Success
[PGVector] The scores returned by 'similarity_search_with_score' are NOT proportional to the similarity
Labels: name: area: vector store, name: auto:bug

### System Info

langchain==0.0.266

### Who can help?

@hwchase17 
@eyurtsev 


### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

## Behavior
When I call the `vector_store.similarity_search_with_score` function:
- Expected: The returned scores will be proportional to the similarity. This means the higher score, the higher similarity.
- Actual: The scores are proportional to the the distance.


### Problem
- When I call `as_retriever` function with the `score_threshold`, the behavior is wrong. Because when `score_threshold` is declared, it will filter documents that have score greater than or equal to `score_threshold` value. So the top documents that found from pgvector will be filter out while it's the most similar in fact.

### Expected behavior

The returned scores from PGVector queries are proportional to the similarity.
In other words, the higher score, the higher similarity.
Issue: problem with conversationchain take multiple inputs
Labels: name: auto:bug, name: area: models

### Issue you'd like to raise.

Hi, I'm trying to make a chatbot using conversationchain. The prompt takes three variables, “history” from memory, user input: "input", and another variable say variable3. But it has an error “Got unexpected prompt input variables. The prompt expects ['history', 'input', 'variable3'], but got ['history'] as inputs from memory, and input as the normal input key. ”

This error doesn't occur if I'm using llmchain. So how can I prevent it if I want to use conversationchain?

Thanks

### Suggestion:

_No response_
Custom Tool Output in a Chain
Labels: name: area: agent, name: auto:enhancement

### Feature request

Although there are many output parsers in langchain, how to custom the output in a chain agent can not find any solutions right now--yet it may be sometimes necessary. Let's say a chain agent x, the tools are Tool1, Tool2 and Tool3, if:

the output of Tool2 should be customized, and no longer be processed by GPT again,
the outputs of Tool1 and Tool3 should be normal, and be processed by GPT again,

in this case, no solution could be found: because the output parser is based on the agent other than any specific tools.


Should this feature be satisfied?

### Motivation

When multiple tools in an agent chain, and some of the tools should be output customized.

### Your contribution

no
`get_openai_callback()` does not count tokens when LCEL chain used with `.stream()` method
Labels: name: auto:bug, name: area: models

### System Info

```
langchain==0.0.335
python==3.10
```

### Who can help?

@agola11 

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [X] Callbacks/Tracing
- [ ] Async

### Reproduction

From the LCEL interface [docs](https://python.langchain.com/docs/expression_language/interface) we have the following snippet:
```python
from langchain.chat_models import ChatOpenAI
from langchain.prompts import ChatPromptTemplate

model = ChatOpenAI()
prompt = ChatPromptTemplate.from_template("tell me a joke about {topic}")
chain = prompt | model

for s in chain.stream({"topic": "bears"}):
    print(s.content, end="", flush=True)
```

From the token usage tracking [docs](https://python.langchain.com/docs/modules/model_io/chat/token_usage_tracking) we have the snippet
```python
from langchain.callbacks import get_openai_callback
from langchain.chat_models import ChatOpenAI

llm = ChatOpenAI(model_name="gpt-4")

with get_openai_callback() as cb:
    result = llm.invoke("Tell me a joke")
    print(cb)
```
that yields the following output:
```
    Tokens Used: 24
        Prompt Tokens: 11
        Completion Tokens: 13
    Successful Requests: 1
    Total Cost (USD): $0.0011099999999999999
```

I am trying to combine the two concepts in the following snippet
```python
from langchain.prompts import ChatPromptTemplate
from langchain.callbacks import get_openai_callback
from langchain.chat_models import ChatOpenAI

llm = ChatOpenAI(model_name="gpt-4")
prompt = ChatPromptTemplate.from_template("tell me a joke about {topic}")
chain = prompt | llm

with get_openai_callback() as cb:
    for s in chain.stream({"topic": "bears"}):
        print(s.content, end="", flush=True)
    print(cb)
```
but get the following result:
```
    Tokens Used: 0
        Prompt Tokens: 0
        Completion Tokens: 0
    Successful Requests: 0
    Total Cost (USD): $0
```

Is token counting (and pricing) while streaming not supported at the moment?

### Expected behavior

The following but with the actual values for tokens and cost.
```
    Tokens Used: 0
        Prompt Tokens: 0
        Completion Tokens: 0
    Successful Requests: 0
    Total Cost (USD): $0
```
updated `Activeloop DeepMemory` notebook
Labels: name: area: doc loader, name: auto:documentation

- Fixed the title of the notebook. It created an ugly ToC element as `Activeloop DeepLake's DeepMemory + LangChain + ragas or how to get +27% on RAG recall.`
- Added Activeloop description
- improved consistency in text
- fixed ToC (it was using HTML tagas that break left-side in-page ToC). Now in-page ToC works
Bagatur/callbacks refactor
Labels: name: area: agent, name: auto:refactor

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
FIX: Limit Azure OpenAI embeddings chunk size
Labels: name: area: embeddings, name: auto:bug

Hi! 
This short PR aims at:
* Fixing `OpenAIEmbeddings`' check on `chunk_size` when used with Azure OpenAI (thus with openai < 1.0). Azure OpenAI embeddings support at most 16 chunks per batch, I believe we are supposed to take the min between the passed value/default value and 16, not the max - which, I suppose, was introduced by accident while refactoring the previous version of this check from this other PR of mine: #10707 
* Porting this fix to the newest class (`AzureOpenAIEmbeddings`) for openai >= 1.0

This fixes #13539 (closed but the issue persists).  

@baskaryan @hwchase17
full template docs
Labels: name: auto:documentation

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
img update
Labels: name: auto:documentation

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
bump 336, exp 44
Labels: name: auto:nit

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
(vectorstore)/PGVectorAsync
Labels: name: area: vector store, name: auto:enhancement

- feat(vectorstore): :sparkles: Create new integration with PGVector async
- test(vectorstore): :white_check_mark: Mark test as async and create db context manager to handle fresh start on tests
- fix(vectorstore): :pencil2: Typo on async calling
- feat(vectorstore): :bug: Set expire on commit to false
- feat(vectorstore): :sparkles: Implement asimilarity_search_with_relevance_scores
- feat(vectorstore): :memo: Update documentation


Eugene:

- Caught up with master (still need to resolve issues)
Runnable with message history
Labels: name: auto:improvement, name: area: models

![Screenshot 2023-11-15 at 1 12 20 PM](https://github.com/langchain-ai/langchain/assets/22008038/2b6a7e38-e934-4b35-af57-0b24e2554e1f)

https://dev.smith.langchain.com/public/c1c06afb-2e4f-4b53-bf37-6926003e7fa9/r
TextGen is not raising Exception when response status code is not 200
Labels: name: auto:bug, name: area: models

### System Info

Langchain 0.0.335

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

When status code is not 200, [TextGen ](https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/llms/textgen.py#L223)only prints the code and returns an empty string.

This is a problem if I want to use TextGen `with_retry()` and I want to retry non 200 responses such as 5xx.
- If I edit the textgen.py source code myself and raise an Exception [here](https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/llms/textgen.py#L223), then my `with_retry()` works as desired.

I tried to handle this by raising an empty string error in my Output Parser, but the TextGen `with_retry()` is not triggering.

### Expected behavior

I'd like TextGen to raise an Exception when the status code is not 200.

Perhaps consider using the `HTTPError` from `requests` package.