delayedkarma/langchain-issues huggingface dataset:
improve llamacpp embeddings
Labels: name: area: embeddings, name: auto:improvement

  - **Description:**
  Improve llamacpp embedding class by adding the `device` parameter so it can be passed to the model and used with `gpu`, `cpu` or Apple metal (`mps`).
  Improve performance by making use of the bulk client api to compute embeddings in batches.
  
  - **Dependencies:** none
  - **Tag maintainer:** 
@hwchase17
add docs for where_document argument and add to missing methods
Labels: name: area: vector store, name: auto:documentation, name: integration: chroma

**Description:**

Chroma vector store was missing documentation and  tests for the `where_document` argument. 

In addition, `similarity_search` method did not accept `where_document` as an argument. This PR fixes these issues.
  
**Issue:** 

No issue raised for this but relates to #10082

**Dependencies:** :
 None
Update `OpenAIEmbeddings` to work with OpenAI SDK updates
Labels: name: area: embeddings, name: auto:bug, name: area: models

### System Info

LC: `v0.0.331rc1`

### Who can help?

The latest LC release candidate does not support the new embeddings API of the OpenAI SDK. 

```
AttributeError: module 'openai' has no attribute 'Embedding'
```

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Install the latest release candidate and use OpenAIEmbeddings

### Expected behavior

Update to work with latest OpenAI SDK.
chore(openllm): simplified interface with new API
Labels: name: auto:improvement, name: area: models, name: size:L

Depends on https://github.com/bentoml/OpenLLM/pull/518 to be released for 0.3.15

twitter handle: @aarnphm_

cc @baskaryan this is ready from OpenLLM.

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
AttributeError: module 'openai' has no attribute 'Completion'
Labels: name: auto:bug, name: area: models

### System Info

Downloading langchain-0.0.331-py3-none-any.whl (2.0 MB)
Downloading openai-1.1.1-py3-none-any.whl (217 kB)


### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Run the following code in colab:

```
!pip install langchain
!pip install openai

from langchain.llms import OpenAI

OpenAI().predict("hoge")
```

You'll get:

```
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
[<ipython-input-5-0eec0f4f0523>](https://localhost:8080/#) in <cell line: 4>()
      2 from langchain.llms import OpenAI
      3 
----> 4 OpenAI().predict("hoge")

3 frames
[/usr/local/lib/python3.10/dist-packages/langchain/llms/openai.py](https://localhost:8080/#) in validate_environment(cls, values)
    264             import openai
    265 
--> 266             values["client"] = openai.Completion
    267         except ImportError:
    268             raise ImportError(

AttributeError: module 'openai' has no attribute 'Completion'
```

### Expected behavior

This was working until yesterday. It's likely due to the openai dependency update.
Add "Astra DB" vector store integration
Labels: name: lgtm, name: area: vector store, name: auto:enhancement, name: template

# Astra DB Vector store integration

  - **Description:** This PR adds a `VectorStore` implementation for DataStax Astra DB using its HTTP API
  - **Issue:** (no related issue)
  - **Dependencies:** A new required dependency is `astrapy` (`>=0.5.3`) which was added to pyptoject.toml, optional, as per guidelines
  - **Tag maintainer:** I recently mentioned to @baskaryan this integration was coming
  - **Twitter handle:** `@rsprrs` if you want to mention me

This PR introduces the `AstraDB` vector store class, extensive integration test coverage, a reworking of the documentation which conflates Cassandra and Astra DB on a single "provider" page and a new, completely reworked vector-store example notebook (common to the Cassandra store, since parts of the flow is shared by the two APIs). I also took care in ensuring docs (and redirects therein) are behaving correctly.

All style, linting, typechecks and tests pass as far as the `AstraDB` integration is concerned.

I could build the documentation and check it all right (but ran into trouble with the `api_docs_build` makefile target which I could not verify: `Error: Unable to import module 'plan_and_execute.agent_executor' with error: No module named 'langchain_experimental'` was the first of many similar errors)

Thank you for a review!
Stefano
Issue: QA with a large csv dataset
Labels: name: area: agent, name: auto:bug

### Issue you'd like to raise.

Hi , I was working on QA using a large csv dataset.
I am using a local llm model (llama2) along with create_csv_agent. Following is my code snippet.

`from langchain.llms import OpenAI
from langchain.chat_models import ChatOpenAI
from langchain.agents.agent_types import AgentType
from langchain_experimental.agents.agent_toolkits import create_csv_agent


agent = create_csv_agent(
    local_llm,
    "MLdata.csv",
    verbose=True,
    agent_type=AgentType.ZERO_SHOT_REACT_DESCRIPTION,handle_parsing_errors=True
)
print(agent.run("Please provide me the 10 records with VAX_TYPE COVID19."))
`

First of all the agent is only displaying 5 rows instead of 10. 
Secondly when I asked about "count the total number of rows in the dataset". It also generated a wrong output (generated output 5).

How to fix this issue? 

### Suggestion:

_No response_
Recent langchain version not matching openai v1.0.0+ release spec for client
Labels: name: auto:bug, name: area: models

### System Info

Langchain: v0.0.331
openai: v1.1.0
platform: Mac M2
python: 3.11.2

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

1. Import OpenAI from llms
2. Instantiate it

```
>>> from langchain.llms import OpenAI
>>> llm = OpenAI()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/lifehedging/.pyenv/versions/myenv/lib/python3.11/site-packages/langchain/load/serializable.py", line 97, in __init__
    super().__init__(**kwargs)
  File "/Users/lifehedging/.pyenv/versions/myenv/lib/python3.11/site-packages/pydantic/v1/main.py", line 339, in __init__
    values, fields_set, validation_error = validate_model(__pydantic_self__.__class__, data)
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lifehedging/.pyenv/versions/myenv/lib/python3.11/site-packages/pydantic/v1/main.py", line 1102, in validate_model
    values = validator(cls_, values)
             ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/lifehedging/.pyenv/versions/myenv/lib/python3.11/site-packages/langchain/llms/openai.py", line 266, in validate_environment
    values["client"] = openai.Completion
                       ^^^^^^^^^^^^^^^^^
AttributeError: module 'openai' has no attribute 'Completion'. Did you mean: 'completions'?
```


### Expected behavior

It correctly instantiates provided an API key is present in the environment
Issue: rewrite the _DEFAULT_ENTITY_EXTRACTION_TEMPLATE in JSON format 
Labels: name: area: memory, name: auto:question, name: auto:improvement

### Issue you'd like to raise.

Hi, It's me again. I'm trying to create my own ConversationSummaryMemory and make  _DEFAULT_ENTITY_EXTRACTION_TEMPLATE in JSON format. Is there a way to do so?

### Suggestion:

_No response_
A ton of Bugs after OpenAI API update today
Labels: name: area: embeddings, name: auto:bug, name: area: models

### System Info

all last versions as of 6/11/2023

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Run anything with OpeanAI ChatCompletions and Embeddings 

### Expected behavior

File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/langchain/embeddings/openai.py", line 284, in validate_environment
    values["client"] = openai.Embedding
                       ^^^^^^^^^^^^^^^^
AttributeError: module 'openai' has no attribute 'Embedding'. Did you mean: 'embeddings'?

File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/langchain/load/serializable.py", line 97, in __init__
    super().__init__(**kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pydantic/v1/main.py", line 341, in __init__
    raise validation_error
pydantic.v1.error_wrappers.ValidationError: 1 validation error for ChatOpenAI
__root__
  `openai` has no `ChatCompletion` attribute, this is likely due to an old version of the openai package. Try upgrading it with `pip install --upgrade openai`. (type=value_error)
Using RunnablePassthrough with Deeplake gives RuntimeError: std::get: wrong index for variant
Labels: name: area: embeddings, name: area: vector store, name: auto:bug, name: area: models

### System Info

Langchain 0.0.331 latest
Openai v0.28.1
Python v3.11.6
Deeplake 3.8.4 latest

### Who can help?
@eyurtsev
@hwchase17

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Once data has been loaded to Deeplake db


```
from langchain.embeddings import OpenAIEmbeddings
from langchain.schema.output_parser import StrOutputParser
from langchain.schema.runnable import RunnablePassthrough
from langchain.vectorstores import DeepLake
from langchain.chat_models import ChatOpenAI
from langchain.prompts import ChatPromptTemplate

model = ChatOpenAI()

db = DeepLake(dataset_path="./my_deeplake/", read_only=True, embedding=OpenAIEmbeddings())

retriever = db.as_retriever()
retriever.search_kwargs['distance_metric'] = 'cos'
retriever.search_kwargs['fetch_k'] = 10
retriever.search_kwargs['maximal_marginal_relevance'] = True
retriever.search_kwargs['k'] = 5

template = """Answer the question based only on the following context:
{context}

Question: {question}
"""
prompt = ChatPromptTemplate.from_template(template)

retrieval_chain = (
    {"context": retriever, "question": RunnablePassthrough()}
    | prompt
    | model
    | StrOutputParser()
)


retrieval_chain.invoke("where did harrison work?")
```
Error

RuntimeError: std::get: wrong index for variant

<img width="1489" alt="image" src="https://github.com/langchain-ai/langchain/assets/150083258/9dbcad3a-759b-492e-86f6-3cd1dc67ce91">



### Expected behavior

Expected behavior is it should retrieve documents from vectordb
Module 'openai' has no attribute 'Embedding'
Labels: name: area: embeddings, name: area: vector store, name: auto:bug, name: area: models

### System Info

LangChain: 0.0.331
OpenAI: 1.1.0
Python:  3.11.2 

### Who can help?

@hwchase17 @agola11 

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Full Message:

> ...langchain/embeddings/openai.py", line 284, in validate_environment
>     values["client"] = openai.Embedding
>                        ^^^^^^^^^^^^^^^^
> AttributeError: module 'openai' has no attribute 'Embedding'. Did you mean: 'embeddings'?

Code line caught this error:

`index = VectorstoreIndexCreator(vectorstore_kwargs={"persist_directory":"persist"}).from_loaders([loader])`

### Expected behavior

Creates Vector Store Index
Issue: "response_format" parameter on the new GPT-4-turbo model
Labels: name: auto:question, name: auto:improvement, name: area: models

### "response_format" parameter on the new GPT-4-turbo model

I can sucessfully call the new GPT-4-turbo model by using:

`const model = new ChatOpenAI({modelName:"gpt-4-1106-preview"})`

But I can't add the "response_format" parameter to set its response explicitly to be a json, as stated in:

https://platform.openai.com/docs/guides/text-generation/json-mode

### Suggestion:

Is there a way to pass the parameter to the model or this should be added to the ChatOpenAI code?
community: Added Lantern as VectorStore
Labels: name: lgtm, name: area: vector store, name: auto:enhancement, name: size:XXL

Support [Lantern](https://github.com/lanterndata/lantern) as a new VectorStore type.

- Added Lantern as VectorStore.
 It will support 3 distance functions `l2 squared`, `cosine` and `hamming` and will use `HNSW` index.
- Added tests
- Added example notebook
chore: change chatCompletion function  to latest api change
Labels: name: auto:improvement, name: area: models

- **Description:** Updated the method of obtaining chat completions to be compatible with the latest OpenAI API. OpenAI has deprecated `chatCompletion` in their recent release. This update replaces the deprecated method with the current `client.chat.completions` method, ensuring compatibility with the latest API standards.

- **Twitter handle:** @nish306 

I have made sure the PR is passing linting and testing before submitting. I have run `make format`, `make lint`, and `make test` to check this locally.
openai has no ChatCompletion attribute, this is likely due to an old version of the openai package. 
Labels: name: auto:bug, name: area: models

### System Info

Langchain v0.0.323
Openai v1.0.1 (latest)
Python v3.11.6

### Who can help?

@hwchase17 @agola11 

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

```
from langchain.chat_models import ChatOpenAI

llm = ChatOpenAI(
    model_name="gpt-4",
    request_timeout=120,
)

```

openai has no ChatCompletion attribute, this is likely due to an old version of the openai package. Try upgrading it with `pip install --upgrade openai`. (type=value_error)

### Expected behavior

There is an issue with ChatOpenAI that I believe may be related to the newest openai python package update.
Bagatur/oai v1 scratch
Labels: name: auto:improvement, name: area: models

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Issue: Agent runs on loop, "Observation: Invalid Format: Missing 'Action:' after 'Thought:"
Labels: name: area: agent, name: auto:bug

### Issue you'd like to raise.

Whenever my agent does not require a tool, (tool has already been used, or query does not require a tool), it continues output.

```
datetimetool = Tool(
    name="datetimetool",
    func=lambda x: datetime.now().strftime('%A %d %B %Y, %I:%M%p'), 
    description="Retrieve and return the current date and/or time. \
        Input should be an empty string.",
)

tools = [datetimetool]

PREFIX = '''
You are a truthful, helpful agent.
'''

FORMAT_INSTRUCTIONS = """Please use the following format only when you need to use a tool:
'''
Thought: Do I need to use a tool? Yes
Action: the action to take, should be one of [{tool_names}]
Action Input: the input to the action
Observation: the result of the action
'''

When you have gathered all the information regarding the user's query,\
    use the following format to answer the query and do not repeat yourself.

'''
Thought: Do I need to use a tool? No
AI: [print answer and stop output]
'''
"""

SUFFIX = '''

Begin!


Instructions: {input}
{agent_scratchpad}
'''

agent = initialize_agent(
    tools=tools,
    llm=llm,
    agent="zero-shot-react-description",
    verbose=True,
    return_intermediate_steps=True,
    handle_parsing_errors=True,
    agent_kwargs={
        'prefix': PREFIX, 
        'format_instructions': FORMAT_INSTRUCTIONS,
        'suffix': SUFFIX,
        'input variables': ["chat_history", "input", "agent_scratchpad", "tool_names"],
    }    
)

query = "How are you?" # Or: "What's the time?"

res = agent({'input': query})
print(res['output'])
```

### Suggestion:

_No response_
OpenAIEmbeddings() does not work because of these bugs
Labels: name: area: embeddings, name: area: vector store, name: auto:bug

### System Info

Python Version: 3.11
LangChain Version: 0.0.331
OpenAI Version: 1.0.0

### Who can help?

@hwchase17, @agola11, @eyurtsev

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [X] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

The following error has been caused due to the recent change in version of OpenAI to 1.0.0
**Use OpenAI==0.28.1 to fix this error**
With the code:
`embeddings = OpenAIEmbeddings()`

The error produced is:
`AttributeError: module 'openai' has no attribute 'Embedding'. Did you mean: 'embeddings'?`

I went through the `langchain/embeddings/openai.py` file and then changed `value["client"] = openai.Embedding` to `value["client"] = openai.embeddings`, but then I receive this new error:

`AttributeError: module 'openai' has no attribute 'error'` in the same file (`langchain/embeddings/openai.py`)


### Expected behavior

There should be no error when calling this function.
BaseCombineDocumentsChain's "input_documents" parameter not overridden by child classes
Labels: name: auto:bug, name: area: models

### System Info

Langchain v0.0.287
Windows 10
Python 3.9

### Who can help?

@eyurtsev

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

```
template = "Here is some context:\n" \
           "{my_documents}\n" \
           "With the help of the context above, please answer the following query:\n" \
           f"{query}"

chain = LLMChain(llm=llm, prompt=PromptTemplate.from_template(template))

combined_chain = StuffDocumentsChain(llm_chain=chain, document_variable_name="my_documents")

result = combined_chain.run(my_documents=documents)
```

### Expected behavior

The code above triggers the following error:

> ValueError: Missing some input keys: {'input_documents'}

Though, the document variable has been explicitly named as "my_documents" and not "input_documents". The reason is that the base class BaseCombineDocumentsChain defines "input_documents" variable by default, but the latter is not overridden by the
parameter "document_variable_name" in child classes. By consistency, the "document_variable_name" should refer both to the variable name specified in the prompt and to the named parameter in the run method of the chain.
Fix for anyscale chat model api key
Labels: name: lgtm, name: auto:bug, name: area: models

* ChatAnyscale was missing coercion to SecretStr for anyscale api key
* The model inherits from ChatOpenAI so it should not force the openai api key to be secret str until openai model has the same changes

https://github.com/langchain-ai/langchain/issues/12841