delayedkarma/langchain-issues huggingface dataset:
Improve CSV reader which can't call .strip() on NoneType 
Labels: name: area: doc loader, name: auto:bug

Improve CSV reader which can't call .strip() on NoneType if there are less cells in the row compared to the header

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** 
I have a CSV file as followed

```
headerA,headerB,headerC
v1A,v1B,v1C,
v2A,v2B
v3A,v3B,v3C
```
In this case, row 2 is missing a value, which results in reading a None type. The strip() method can not be called on None, hence raising. In this PR I am making the change to only call strip if the value if not None.

  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
how to use reranker model with langchain in retrievalQA case?
Labels: name: area: embeddings, name: auto:question, name: area: models

### System Info

version 0.0.331

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [X] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

omit

### Expected behavior

I would like to use  BAAI/bge-reranker-large  model as the reranker to rerank the  intial retrieval results, 
how to use this model with langchain ?
I have seen the demo with Cohere ranker, 
but how to use ohter reranker models?
Does langchain support other reranker models except for CohereReranker?
Issue: AgentExecutor typings should accept Runnable for the agent property (to support LCEL agent)
Labels: name: area: agent, name: auto:bug

### Issue you'd like to raise.

Using LCEL as suggested on the [docs](https://python.langchain.com/docs/modules/agents/), combined with `AgentExecutor`, generates a typing error when passing the runnable agent to the `AgentExecutor` constructor. This is because `AgentExecutor` defines its `agent` property as of type `BaseSingleActionAgent | BaseMultiActionAgent`:

https://github.com/langchain-ai/langchain/blob/55aeff6777431dc24e48f018e39aa418f95a6489/libs/langchain/langchain/agents/agent.py#L728-L731


So:
```python
agent = {
    ...
} | prompt | llm | OpenAIFunctionsAgentOutputParser()

# agent would be of type Runnable[Unknown, Unknown]
# but the typing on AgentExecutor only takes a BaseSingleActionAgent 
# or a BaseMultiActionAgent as a valid agent
agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True)
#                                      ↳  "Runnable[Unknown, Unknown]" is incompatible with "BaseSingleActionAgent"
#                                      ↳  "Runnable[Unknown, Unknown]" is incompatible with "BaseMultiActionAgent"

```

### Suggestion:

`AgentExecutor` should accept a `Runnable` for its agent property
[Feature Request] AnthropicFunctionsAgent
Labels: name: area: agent, name: auto:enhancement

### Feature request

Hi, I'd like to request a feature for a `AnthropicFunctionsAgent` built on top of `AnthropicFunctions` and ideally compatible with `create_conversational_retrieval_agent`.

### Motivation

Everyone working with Anthropic models could use an Agents class!

### Your contribution

Can't currently.
Running Ollama asynchronously generates a warning:  RuntimeWarning: coroutine 'AsyncCallbackManagerForLLMRun.on_llm_new_token' was never awaited (langchain/llms/ollama.py:164)
Labels: name: auto:bug, name: area: models

### System Info

python==3.11
langchain==0.0.326
ollama==v0.1.8

### Who can help?

@agola11

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [X] Callbacks/Tracing
- [X] Async

### Reproduction

Steps to reproduce the behavior:

```
llm = Ollama(model='llama2')
r = await llm.agenerate(['write a limerick about babies'])
print('\n'.join([t[0].text for t in r.generations]))
```
Generates the following output: 
```
[/path/to/python3.11/site-packages/langchain/llms/ollama.py:164](https://file+.vscode-resource.vscode-cdn.net/Users/gburns/miniconda3/envs/alhazen/lib/python3.11/site-packages/langchain/llms/ollama.py:164): RuntimeWarning: coroutine 'AsyncCallbackManagerForLLMRun.on_llm_new_token' was never awaited
  run_manager.on_llm_new_token(
RuntimeWarning: Enable tracemalloc to get the object allocation traceback

 Sure! Here is a limerick about babies:
 everybody loves babies, they're so sweet;
their coos and snuggles can't be beat.
They suck their thumbs and play with toes,
and bring joy to all who sees.
```



### Expected behavior

The generation should occur without the warning. 

The error is due to the `_OllamaCommon._stream_with_aggregation(` function not being able to distinguish between being called in a blocking or an async context. 

The reason this is important is that sometimes Ollama gets stuck in a generation (taking a long time to complete) and I would like to be able to call a timeout on the underlying process. The following code can do this, but we get the warning as previously described (note, this requires that ollama be running in the background).

```   
def _callback(fut: asyncio.Future):
        if fut.cancelled() or not fut.done():
            print("Timed out! - Terminating server")
            fut.cancel()
            
async def run_llm(llm, prompt, timeout=300):
        # create task
        task = asyncio.create_task(llm.agenerate([prompt]))
        task.add_done_callback(_callback)
        # try to await the task
        try:
            r = await asyncio.wait_for(task, timeout=timeout)
        except asyncio.TimeoutError as ex:
            print(ex)
        if r is not None:
            return '\n'.join([t[0].text for t in r.generations])
        else:
            return ''
            
text = await llm.agenerate(['write a limerick about babies'])
print(text)
 ```
PyPI versions issue e.g. langchain==0.0.331 is not newer than langchain==0.0.79 
Labels: name: auto:bug

### System Info

Unix and Windows

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction
```
pip install langchain==0.0.79
pip install langchain>=0.0.331
```

### Expected behavior

I would expect to install the latest version 0.0.331 but its not. Seems pip undestands that 0.0.79 is a higher version than 0.0.331?
Issue: STRUCTURED_CHAT_ZERO_SHOT_REACT_DESCRIPTION sometimes not return the "Final Answer"

### Issue you'd like to raise.

The agent(STRUCTURED_CHAT_ZERO_SHOT_REACT_DESCRIPTION) sometimes does not respond the "Final Answer" instead of the AI's thought.

### Actual result:
The human is asking for the default xxxxxx. The tool has provided the information. I will now relay this information to the human.

### Expected result:
{
  "action": "Final Answer",
  "action_input": "The information comes from the tool."
}

### System info:
AzureChatOpenAI / gpt-4

This is a sometimes issue. I have tried with the smith playground, it can happen as well. Any thoughts about it?
Seeing an error ModuleNotFoundError: No module named 'langchain.globals'
Labels: name: auto:bug

### System Info

LangChain version pinned to "^0.0.278" (using Poetry)
Python 3.11.5

Other modules from langchain (such as langchain.cache and langchain.chains) are imported within the same file in the application code and are able to be found. Only the `langchain.globals` module is not being recognized

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Import langchain.globals

Error happens on application startup

### Expected behavior

Expected to be able to find the module, as langchain's other module's are able to be found by the same application code
ChromaDb EmbeddingFunction definition updated
Labels: name: area: embeddings, name: area: vector store, name: auto:bug, name: integration: chroma

### System Info

Using Google Colab Free version with T4 GPU. 
chromadb==0.4.16



### Who can help?

@agola11 @hwchase17 

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [X] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

As per the latest Chromadb migration logs ([link](https://docs.trychroma.com/migration#migration-to-0416---november-7-2023)) `EmbeddingFunction` defnition has been updated and it affects all the custom made embedding function.   
What this means is the `langchain.embeddings.HuggingFaceBgeEmbeddings` is inconsistent with this new definition and throws the following error:  
```py
ValueError: Expected EmbeddingFunction.__call__ to have the following signature: odict_keys(['self', 'input']), got odict_keys(['self', 'args', 'kwargs'])
Please see https://docs.trychroma.com/embeddings for details of the EmbeddingFunction interface.
Please note the recent change to the EmbeddingFunction interface: https://docs.trychroma.com/migration#migration-to-0416---november-7-2023 
```
The above code can be reproduced by inserting documents into Chromadb embedded using `HuggingFaceBgeEmbeddings` like so:  
```py  
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.vectorstores import Chroma
from langchain.embeddings import HuggingFaceBgeEmbeddings
from transformers import AutoTokenizer

embedding_function = HuggingFaceBgeEmbeddings(
    model_name="BAAI/bge-base-en-v1.5",
    model_kwargs={'device': 'cuda'},
    encode_kwargs={'normalize_embeddings': True},
    query_instruction="Represent this sentence for searching relevant passages: "
)  
tokenizer = AutoTokenizer.from_pretrained('BAAI/bge-base-en-v1.5')
text_splitter = RecursiveCharacterTextSplitter.from_huggingface_tokenizer(
    tokenizer, chunk_size=100, chunk_overlap=0
)
text = 'Some text that needs to be embedded.'
print(len(embedding_function.embed_query(text))) # works so far
splits = text_splitter.create_documents([text])
db = Chroma.from_documents(splits, embedding_function, persist_directory="./chroma_db")
```

I am not sure, but the answer might lie in correcting the `HuggingFaceBgeEmbeddings` class : [link](https://github.com/langchain-ai/langchain/blob/1f27104626fc71a5199df965011810426dd2eede/libs/langchain/langchain/embeddings/huggingface.py#L188) ?

### Expected behavior

The expected behaviour would have made a valid `db` object upon running the code  
```py 
db = Chroma.from_documents(splits, embedding_function, persist_directory="./chroma_db")
Issue: UnstructuredPDFLoader doesn't support Unstructured functionalities

### Issue you'd like to raise.

When initializing the langchain UntructuredPDFLoader e.g. as follows
` loader = UnstructuredPDFLoader(downloaded_file, mode='elements')`

This method calls the following function (see langchain/document_loaders/pdf.py):

` class UnstructuredPDFLoader(UnstructuredFileLoader): `

   `def _get_elements(self) -> List: `

        from unstructured.partition.pdf import partition_pdf
        return partition_pdf(filename=self.file_path, **self.unstructured_kwargs) 

The function `partition_pdf()` from Unstructured allows one to decide between passing either a file_path to a file in storage, or alternatively a ByteStream pointing to a file in memory but it does not allow one to pass both. Langchain forces users to pass the parameter ` file_path `and thus one cannot use the option of using a stream to load a file (as Unstructured doesn't expect both a file_path and a stream).


### Suggestion:

Remove the part which forces one to pass a ` file_path ` to UnstructuredPDFLoader initializiation. With this change, users can decide to pass a Stream in the `unstructured_kwargs ` field and thus use the loader.

To test this I rewrote the _get_elements function as follows and like this it works to pass a stream:
` 
def _get_elements(self) -> List: `
           
            ` from unstructured.partition.pdf import partition_pdf
            return partition_pdf(**self.unstructured_kwargs) `
Issue with user_agent error when creating a CohereReRank object in LangChain 
Labels: name: auto:bug, name: area: models

### System Info

last version of langchain and cohere

### Who can help?

@@agola11
I am encountering an error related to the user_agent when attempting to create a CohererRank object in the LangChain p. I have verified that I am passing a valid cohere_api_key and can successfully use the rerank function from Cohere directly. However, when trying to utilize it through the LangChain project, I encounter this specific issue related to the user_agent.

[/usr/local/lib/python3.10/dist-packages/langchain/retrievers/document_compressors/cohere_rerank.py](https://localhost:8080/#) in validate_environment(cls, values)
     53             values, "cohere_api_key", "COHERE_API_KEY"
     54         )
---> 55         client_name = values["user_agent"]
     56         values["client"] = cohere.Client(cohere_api_key, client_name=client_name)
     57         return values

KeyError: 'user_agent'

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction


import os
import getpass

os.environ["COHERE_API_KEY"] = getpass.getpass("Cohere API Key:")
from langchain.llms import OpenAI
from langchain.retrievers import ContextualCompressionRetriever
from langchain.retrievers.document_compressors import CohereRerank
compressor = CohereRerank()


### Expected behavior

i want to create an object of CohereReRank.
Refine Weaviate docs and add RAG example
Labels: name: lgtm, name: area: vector store, name: auto:documentation

  - **Description:** Refine Weaviate tutorial and add an example for Retrieval-Augmented Generation (RAG)
  - **Issue:** (not applicable),
  - **Dependencies:** none
  - **Tag maintainer:** @baskaryan <!--
If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
  - **Twitter handle:** @helloiamleonie
The prompt word format misleads the output content of the large model
Labels: name: auto:bug, name: area: models

### System Info

langchain   0.0.331

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [X] Prompts / Prompt Templates / Prompt Selectors
- [X] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

response_schemas = [
            ResponseSchema(type='list', name='disease', description='disease name')
]

**prompt**
The output should be a markdown code snippet formatted in the following schema, including the leading and trailing "```json" and "```":

```json
{
        "disease": list  // disease name
}
```

**output**
{
        "disease": "感冒"  // disease name
}

This output cannot be parsed by StructuredOutputParser，Because it contains notes。

### Expected behavior

Expect to generate prompt words：

**prompt**
The output should be a markdown code snippet formatted in the following schema, including the leading and trailing "```json" and "```"，and the field "disease" meas "disease name".

```json
{
        "disease": list
}
```

Expected output：

{
        "disease": "感冒"
}
OpenAI instance of Runnable expected
Labels: name: auto:bug, name: auto:question, name: area: models

### System Info

# Hi there,
I have started learning about Langchain today. I was creating my first langchain prompt template but something doesn't seem to work.

Here is my code in main.py:
```python
from openai import OpenAI
from dotenv import load_dotenv
from langchain.prompts import PromptTemplate
from langchain.chains import LLMChain
import os

# Load the .env file
load_dotenv()

# Create an instance of the OpenAI class
def generate_pet_name(animal_type="dog"):
    
    prompt_template = PromptTemplate(
        input_variables=['animal_type'],
        template="I have a {animal_type} as my pet. Suggest me a name for it.",
    )

    name_chain = LLMChain(
        llm=OpenAI(),
        prompt=prompt_template,
    )

    response = name_chain({'animal_type': animal_type})
    print(response)


if __name__ == "__main__":
    generate_pet_name(animal_type="dog")
```

While I think the code is okay and I have followed the GitHub get started it doesn't seem to work and throwing me this error:
```console
Traceback (most recent call last):
  File "/Users/ss/Workspace/Ai/AMP/main.py", line 28, in <module>
    generate_pet_name(animal_type="dog")
  File "/Users/ss/Workspace/Ai/AMP/main.py", line 18, in generate_pet_name
    name_chain = LLMChain(
                 ^^^^^^^^^
  File "/Users/ss/Workspace/Ai/AMP/.venv/lib/python3.11/site-packages/langchain/load/serializable.py", line 97, in __init__
    super().__init__(**kwargs)
  File "/Users/ss/Workspace/Ai/AMP/.venv/lib/python3.11/site-packages/pydantic/v1/main.py", line 341, in __init__
    raise validation_error
pydantic.v1.error_wrappers.ValidationError: 2 validation errors for LLMChain
llm
  instance of Runnable expected (type=type_error.arbitrary_type; expected_arbitrary_type=Runnable)
llm
  instance of Runnable expected (type=type_error.arbitrary_type; expected_arbitrary_type=Runnable)
  ```
  
  
  Please Help me as I am not that proficient in Python.

### Who can help?

@agola11 

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Just copy the code given there and run it. You will see the error.

### Expected behavior

Run the code and get a response without error
Fixed a typo in smart_llm prompt
Labels: name: lgtm, name: auto:nit, name: area: models

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Bug after the openai updated in Embedding
Labels: name: area: embeddings, name: auto:bug, name: integration: chroma

### System Info

AWS Sagemaker DataScience3.0 Image.

### Who can help?

_No response_

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [X] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

here is my code, it worked before Nov 7th.

`Chroma.from_documents(documents=document, embedding=embeddings,)`

Then i get this error:
`ValueError: Expected EmbeddingFunction.__call__ to have the following signature: odict_keys(['self', 'input']), got odict_keys(['self', 'args', 'kwargs'])
Please see https://docs.trychroma.com/embeddings for details of the EmbeddingFunction interface.
Please note the recent change to the EmbeddingFunction interface: https://docs.trychroma.com/migration#migration-to-0416---november-7-2023 `


### Expected behavior

Is anyone know how to fix this?