delayedkarma/langchain-issues huggingface dataset:
packaging: fix format for python version
Labels: name: auto:improvement, name: size:XS

Hello,

could you adjust the version format? I am running into issues where it's not understood by some tools.

The python version format is only major.minor. It's one of the edge cases of python packaging.

You can see examples in setuptools, both the required python version for the package and the required python version for dependencies. It's not documented well but the format is only meant to be two digits.
https://setuptools.pypa.io/en/latest/userguide/declarative_config.html
https://setuptools.pypa.io/en/latest/userguide/pyproject_config.html

```
# setup.cfg
[options]
zip_safe = False
include_package_data = True
packages = find:
python_requires = >=3.7
install_requires =
    requests
    importlib-metadata; python_version<"3.8"
```
feat: add streaming param to ChatMistralAI constructor
Labels: name: auto:improvement, name: area: models, name: size:S, name: partner

- **Description:** A new boolean parameter `streaming` has been added to the constructor of the `ChatMistralAI` class. This parameter allows users to specify whether they wish to use the streaming functionality at the time of instantiating a `ChatMistralAI` object.
- **Tag maintainer:** @baskaryan
- **Twitter handle:** @LMS_David_RS
Dmaturana/randomlengthfewshotselector
Labels: name: area: doc loader, name: auto:improvement, name: size:L

  - **Description:** The length example selector always retrieves the same examples, going from top to bottom. In the case you have a lot of examples, there will not be any variation. I just copied and modified this length selector to do a random length base examples selector, so for each prompt, the examples selected will be random 
  
  - **Dependencies:** Added the random import, not dependencies changed
  
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
Issue with DynamoDBChatMessageHistory Not Respecting max_token_limit in ConversationTokenBufferMemory
Labels: name: area: memory, name: auto:bug, name: integration: aws

### System Info

Langchain Version: langchain 0.0.350, langchain-community 0.0.3, langchain-core 0.1.1
Python Version: 3.10.6
Operating System: macOs
Additional Libraries: boto 2.49.0, boto3 1.34.1

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [X] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Steps to Reproduce:
- Create an instance of `DynamoDBChatMessageHistory` with a specified table name and session ID.
- Initialize `ConversationTokenBufferMemory` with a `max_token_limit`.
- Attach the memory to a `ConversationChain`.
- Call predict on the `ConversationChain` with some input.


Code sample:
```
import boto3
from langchain.llms import Bedrock
from langchain.memory.chat_message_histories import DynamoDBChatMessageHistory
from langchain.memory import ConversationTokenBufferMemory

session = boto3.Session(
    aws_access_key_id=os.environ.get('AWS_ACCESS_KEY_ID'),
    aws_secret_access_key=os.environ.get('AWS_SECRET_ACCESS_KEY'),
    aws_session_token=os.environ.get('AWS_SESSION_TOKEN'),  
    region_name='us-east-1'
)

dynamodb = session.resource('dynamodb')
chat_sessions_table = dynamodb.Table('SessionTable')

boto3_bedrock = session.client(service_name="bedrock-runtime")

max_tokens_to_sample = 100
temperature = 0
modelId = "anthropic.claude-instant-v1"
top_k = 250
top_p = 0.999

model_kwargs = {
    "temperature": temperature,
    "max_tokens_to_sample": max_tokens_to_sample,
    "top_k": top_k,
    "top_p": top_p
}

llm = Bedrock(
    client=boto3_bedrock,
    model_id=modelId,
    region_name='us-east-1',
    model_kwargs=model_kwargs,
    streaming=True,callbacks=[StreamingStdOutCallbackHandler()]
    )

message_history = DynamoDBChatMessageHistory(table_name="SessionTable", session_id="10", boto3_session=session)

memory = ConversationTokenBufferMemory(
    llm=llm,  # Use the Bedrock instance
    max_token_limit=100,  
    return_messages=True,
    chat_memory=message_history,
    ai_prefix="A",
    human_prefix="H"
)

#add the memory to the Chain
conversation = ConversationChain(
    llm=llm, verbose=True, memory=memory
)

conversation.predict(input="Hello!")

memory.load_memory_variables({})
```

### Expected behavior

Expected Behavior:
- The `DynamoDBChatMessageHistory` should respect the `max_token_limit` set in `ConversationTokenBufferMemory`, limiting the token count accordingly.

Actual Behavior:
The `DynamoDBChatMessageHistory` does not limit the token count as per the `max_token_limit` set in `ConversationTokenBufferMemory` and keeps saving all the items in memory on the DynamoDB table.
Feature Request - ChatGPT OpenAPI NLATool Authentication Implementation Logic
Labels: name: auto:enhancement, name: area: models

### Feature request

 
**Issue Title:** Enhance NLATool Authentication in ChatGPT OpenAPI

**Description:**
I have identified a feature gap in the current implementation of ChatGPT OpenAPI when using NLATool as a proxy for authentication. The existing logic does not fully meet downstream requirements, and I intend to propose a modification to address this issue.

**Proposed Modification:**
I suggest adding a new attribute within the NLATool implementation. During initialization, this attribute should be passed to `NLAToolkit.from_llm_and_ai_plugin`. The subsequent call chain is as follows: `from_llm_and_spec -> _get_http_operation_tools -> NLATool.from_llm_and_method`. The responsibility of `NLATool.from_llm_and_method` is to construct the NLATool component, which includes an underlying `OpenAPIEndpointChain` base package.

The challenge lies in the fact that the `OpenAPIEndpointChain` base package currently lacks support for authentication. To address this, it is essential to load the created attribute into the `OpenAPIEndpointChain`. During the execution of the `_call` method, the authentication logic should be executed.

**Implementation Steps:**
1. Modify the initialization and execution code of the `OpenAPIEndpointChain` class to support authentication.
2. Ensure that the newly added attribute is properly integrated into the `OpenAPIEndpointChain` during its initialization.
3. Implement the authentication logic in the `_call` method of the `OpenAPIEndpointChain`.


Thank you for your consideration.
 

### Motivation

The current implementation of ChatGPT OpenAPI using NLATool as a proxy for authentication falls short of meeting downstream requirements. By enhancing the NLATool authentication logic, we aim to improve the overall functionality and responsiveness of the system, ensuring it aligns more effectively with user needs. This modification seeks to bridge the existing feature gap and enhance the usability and versatility of the ChatGPT OpenAPI.

### Your contribution

**Expected Challenges:**
While I have not yet started the implementation, I anticipate challenges during the process. One notable challenge is that the `langchain` module does not currently define the core for the authentication class. Consequently, addressing this issue may require changes across multiple modules. A pull request spanning multiple modules may encounter challenges during the review process.

**Request for Feedback:**
Before I commence with the implementation, I would appreciate your insights and guidance on the proposed modification. Your feedback on potential challenges and recommendations for an effective implementation would be invaluable.
getting the error "TypeError: Agent.plan() got multiple values for argument 'intermediate_steps'" with agent_executor
Labels: name: area: agent, name: auto:bug

### System Info

langchain:                            0.0.351
langchain-community:        0.0.4
langchain-core:                   0.1.1
langchain-experimental:     0.0.47
python:                                3.10.4

### Who can help?

@hwchase17 , @agola11 , @eyurtsev 

### Information

- [X] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [X] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [X] Agents / Agent Executors
- [X] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

1. Create an agent and which is working as expected.
2. create an agent_executor by using above agent.
3. If i try to use agent_execuor, getting error "TypeError: Agent.plan() got multiple values for argument 'intermediate_steps'"

**Below is the code:**

Create an agent:-

```
agent = initialize_agent(llm=llm,
                         tools=tools,
                         agent = AgentType.CONVERSATIONAL_REACT_DESCRIPTION,
                         verbose=True,
                         agent_kwargs=agent_kwargs,
                         output_parser = output_parser,
                         output_key = "result",
                         handle_parsing_errors = True,
                         max_iterations=3,
                         early_stopping_method="generate", 
                         memory = memory,                         
                        )
```

Create an agent_executor:-

```
agent_executor = AgentExecutor(agent=agent,
                               tools=tools,
                               verbose=True,
                               memory = memory,                               
                                )

```

calling the agent_executor
`result = agent_executor.invoke({"input":"Tell me about yourself", "format_instructions": response_format})["output"]`

Getting below error:-

```
Entering new AgentExecutor chain...


> Entering new AgentExecutor chain...
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[18], line 1
----> 1 result = agent_executor.invoke({"input":"Tell me about yourself", "format_instructions": response_format})["output"]
      2 print(f"result: {result}")

TypeError: Agent.plan() got multiple values for argument 'intermediate_steps'



```

What I have observed from the above error is that the chain is executing multiple times, hence the 'Entering new AgentExecutor chain...' message is displaying twice. This could be the cause of the issue.

### Expected behavior

Should return proper output with thought and action
Update arxiv.py with get_summaries_as_docs inside of Arxivloader
Labels: name: area: doc loader, name: auto:improvement, name: size:XS

Added the call function get_summaries_as_docs inside of Arxivloader

  - **Description:** Added a function that returns the documents from get_summaries_as_docs, as the call signature is present in the parent file but never used from Arxivloader, this can be used from Arxivloader itself just like .load() as both the signatures are same. 
  - **Issue:** Reduces time to load papers as no pdf is processed only metadata is pulled from Arxiv allowing users for faster load times on bulk loads. Users can then choose one or more paper and use ID directly with .load() to load pdf thereby loading all the contents of the paper.
Issue: Getting an error while trying to run LLMChain with Custom LLM
Labels: name: auto:question, name: area: models


I am trying to run the LLMChain using `llm_chain = LLMChain(llm=llm, prompt=prompt)`, where `llm` is a custom LLM defined based on https://python.langchain.com/docs/modules/model_io/llms/custom_llm, 

While trying to run this I am getting the following error: `Can't instantiate abstract class BaseLanguageModel with abstract methods agenerate_prompt, apredict, apredict_messages, generate_prompt, predict, predict_messages (type=type_error)`

Can someone help me with this? 

### Suggestion:

_No response_
ImportError: cannot import name 'AzureOpenAIEmbeddings' from 'langchain.embeddings' (/opt/conda/lib/python3.10/site-packages/langchain/embeddings/__init__.py)
Labels: name: area: embeddings, name: auto:bug

### System Info

unable to resolve below issue

![image](https://github.com/langchain-ai/langchain/assets/144101750/a7b75fce-bb32-4b13-ab3e-e122455e3213)


### Who can help?

_No response_

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [X] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

![image](https://github.com/langchain-ai/langchain/assets/144101750/40a9ed30-0761-46b0-8bcc-acb9cceb1734)


### Expected behavior

ImportError: cannot import name 'AzureOpenAIEmbeddings' from 'langchain.embeddings' (/opt/conda/lib/python3.10/site-packages/langchain/embeddings/__init__.py)
Add support Vertex AI Gemini uses a public image URL
Labels: name: lgtm, name: auto:improvement, name: area: models, name: size:M

## What

Since `langchain_google_genai.ChatGoogleGenerativeAI` supported A public image URL, we add to support it in `langchain.chat_models.ChatVertexAI` as well.

### Example

```py
from langchain.chat_models.vertexai import ChatVertexAI
from langchain_core.messages import HumanMessage

llm = ChatVertexAI(model_name="gemini-pro-vision")
image_message = {
    "type": "image_url",
    "image_url": {
        "url": "https://python.langchain.com/assets/images/cell-18-output-1-0c7fb8b94ff032d51bfe1880d8370104.png",
    },
}
text_message = {
    "type": "text",
    "text": "What is shown in this image?",
}
message = HumanMessage(content=[text_message, image_message])

output = llm([message])
print(output.content)
```

## Refs

- https://python.langchain.com/docs/integrations/llms/google_vertex_ai_palm
- https://python.langchain.com/docs/integrations/chat/google_generative_ai
The calculated score is wrong when using DistanceStrategy.MAX_INNER_PRODUCT (Faiss)
Labels: name: area: embeddings, name: auto:bug

### System Info

langchain: 0.0.338
python: 3.9



### Who can help?

@hwchase17 
@eyurtsev 

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [X] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

In the following code:
``` python
if self.distance_strategy == DistanceStrategy.MAX_INNER_PRODUCT:  
    return self._max_inner_product_relevance_score_fn  
elif self.distance_strategy == DistanceStrategy.EUCLIDEAN_DISTANCE:  
    # Default behavior is to use Euclidean distance for relevancy  
    return self._euclidean_relevance_score_fn  
elif self.distance_strategy == DistanceStrategy.COSINE:  
    return self._cosine_relevance_score_fn
```
When I use MAX_INNER_PRODUCT, the score calculation method is `_max_inner_product_relevance_score_fn`:
``` python
    def _max_inner_product_relevance_score_fn(distance: float) -> float:
        """Normalize the distance to a score on a scale of [0, 1]."""
        if distance > 0:
            return 1.0 - distance

        return -1.0 * distance
```
However, if I use MAX_INNER_PRODUCT, the index must be FlatIP:
``` python
if distance_strategy == DistanceStrategy.MAX_INNER_PRODUCT:
    index = faiss.IndexFlatIP(len(embeddings[0]))
else:
    # Default to L2, currently other metric types not initialized.
    index = faiss.IndexFlatL2(len(embeddings[0]))
```
Thus, the distance represents the cosine similarity, meaning the distance should be equivalent to similarity. However, in the method `_max_inner_product_relevance_score_fn`, a larger distance results in a lower score. 

Is this a bug?


### Expected behavior

I think the distance sholud be  equivalent to similarity。
Issue: Getting 'An output parsing error occurred' error even after passing 'handle_parsing_errors=True' to the agent
Labels: name: area: agent, name: auto:bug

### Issue you'd like to raise.

I am using langchain with gpt 4 model. Im using the create pandas dataframe agent for my use case. For 60 % of the time i run the code, im getting below error-

An output parsing error occurred. In order to pass this error back to the agent and have it try again, pass `handle_parsing_errors=True` to the AgentExecutor. This is the error: Could not parse LLM output: `Thought: To answer the question about what the "EffectiveDate" column represents, I need to use common sense based on the column name and the data provided. For the "exists" part, I need to check if there are any standardization issues in the "EffectiveDate" column. I will look at the data provided and think about the possible issues listed.

Now i have already passed the argument- `handle_parsing_errors=True' while creating the agent, but still it gives me the above error, suggesting me to pass this argument.
I have also tried giving other values to the handle_parsing_errors like - passing custom error message or function, still im stuck with this error most of the time.


### Suggestion:

_No response_
openai migrate
Labels: name: auto:question, name: area: models, name: integration: chroma

### System Info


loader1 = CSVLoader(file_path='/home/calvin/下载/test.csv')
Doc = loader1.load()
text_splitter = CharacterTextSplitter(chunk_size=100,chunk_overlap=0)
texts = text_splitter.split_documents(Doc)
embeddings = OpenAIEmbeddings()
db = Chroma.from_documents(texts, embeddings)
retriever = db.as_retriever()
qa = RetrievalQA.from_chain_type(llm=OpenAI(mode="gpt-3.5-turbo"), chain_type="stuff", retriever=retriever)
query = "1501475820"
print(qa.run(query))

I run this code,but i can not use chat-gpt-3.5-turbo，so i try to opanAI MIGRATE，but i exit it ,then i found 
![Screenshot_20231220_160320](https://github.com/langchain-ai/langchain/assets/108516348/cc3574ce-5fe8-4f10-bace-070ae4d41727)

then  always  tell me 
![Screenshot_20231220_160734](https://github.com/langchain-ai/langchain/assets/108516348/4f7b2e29-fb59-4c65-819b-a85f991b4f66)

@hwcha



### Who can help?

from langchain.chains import RetrievalQA

### Information

- [X] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

loader1 = CSVLoader(file_path='/home/calvin/下载/test.csv')
Doc = loader1.load()
text_splitter = CharacterTextSplitter(chunk_size=100,chunk_overlap=0)
texts = text_splitter.split_documents(Doc)
embeddings = OpenAIEmbeddings()
db = Chroma.from_documents(texts, embeddings)
retriever = db.as_retriever()
qa = RetrievalQA.from_chain_type(llm=OpenAI(mode="gpt-3.5-turbo"), chain_type="stuff", retriever=retriever)
query = "1501475820"
print(qa.run(query))


### Expected behavior

i want use gpt-3.5-turbo to query
Issue: how to deploy gpt-4-turbo through langchain
Labels: name: auto:question, name: area: models

### Issue you'd like to raise.

Hi, I can't find from how to deploy gpt-4-turbo in langchain. 
Could anyone please tell me throuhg which module gpt-4-turbo can be deployed? 
Seems that the langchain.llm has already been removed from new version of langchain. 

### Suggestion:

_No response_
ImportError: cannot import name 'LLMContentHandler' from 'langchain.llms.sagemaker_endpoint' occurring with 0.0.351
Labels: name: auto:bug, name: area: models, name: integration: aws

### System Info

Langchain == 0.0.351
Python == 3.10.6
Running in AWS sagemaker notebook, issue occurred on multiple kernels.  

Code worked perfectly yesterday, error occurred upon starting up this morning (12/19/23)

Code worked again upon reverting to 0.0.349

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

pip install -U langchain
from langchain.llms.sagemaker_endpoint import LLMContentHandler

### Expected behavior

Expected behavior is that the import works
How does AgentExecutor make LLM on_llm_new_token most streaming output instead of AgentExecutorIterator?
Labels: name: auto:question, name: area: models

How does AgentExecutor make LLM on_llm_new_token most streaming output instead of AgentExecutorIterator?
The current effect is that it will stream out AgentExecutorIterator
Desired effect: streams LLM on_llm_new_token
update notebook documentation for custom tool 
Labels: name: lgtm, name: area: doc loader, name: auto:documentation, name: size:L



  - **Description:** Documentation update. The custom tool notebook documentation is updated to revome the warning caused by directly instantiating of the LLMMathChain with an llm which is is deprecated. The from_llm class method is used instead. LLM output results gets updated as well.
  - **Issue:** no applicable
  - **Dependencies:** No dependencies
  - **Tag maintainer:** @baskaryan
  - **Twitter handle:** @ybouakkaz
Add support for Vertex ai multimodal embeddings
Labels: name: area: embeddings, name: auto:enhancement, name: size:L

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://python.langchain.com/docs/contributing/

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
ConvoOutputParser: Handle multiline Action Input + potential hallucin…
Labels: name: area: agent, name: auto:improvement, name: size:M

…ated Observation

  - **Description:** Handle multiline Action Inputs.
  - **Issue:** Fixes cases where our agent made multiline Action Input (specifically contained python code)
  - **Dependencies:** --
  - **Tag maintainer:** @hwchase17
  - **Twitter handle:** All the credits to @robocorp

Note: I did not fully get where the tests should go so that is my best effort.