delayedkarma/langchain-issues huggingface dataset:
Fix VectorStoreMemory when used with CombinedMemory
Labels: name: area: memory, name: auto:bug

Fix VectorStoreMemory mixing the other memory input when used in CombinedMemory

<!--
Thank you for contributing to LangChain! Your PR will appear in our release under the title you set. Please make sure it highlights your valuable contribution.

Replace this with a description of the change, the issue it fixes (if applicable), and relevant context. List any dependencies required for this change.

After you're done, someone will review your PR. They may suggest improvements. If no one reviews your PR within a few days, feel free to @-mention the same people again, as notifications can get lost.

Finally, we'd love to show appreciation for your contribution - if you'd like us to shout you out on Twitter, please also include your handle!
-->

<!-- Remove if not applicable -->



#### Before submitting

<!-- If you're adding a new integration, please include:

1. a test for the integration - favor unit tests that does not rely on network access.
2. an example notebook showing its use


See contribution guidelines for more information on how to write tests, lint
etc:

https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
-->

Fixes Issue when using multiple memory
When we used multiple memory, VectorStoreMemory together with ConversationBufferWindowMemory, the VectorStoreMemory history is mixed up with ConversationBufferWindowMemory

```python
vectorstore_memory = VectorStoreRetrieverMemory(input_key="input", memory_key="related_conv", retriever=retriever)
conv_memory = ConversationBufferWindowMemory(input_key="input", memory_key="previous_conv", k=1)
memory = CombinedMemory(memories=[conv_memory, vectorstore_memory])
```

`related_conv` contains `input + previous_conv + response`
#### Who can review?
Tag maintainers/contributors who might be interested:
@dev2049

<!-- For a quicker response, figure out the right person to tag with @

  @hwchase17 - project lead

  Tracing / Callbacks
  - @agola11

  Async
  - @agola11

  DataLoaders
  - @eyurtsev

  Models
  - @hwchase17
  - @agola11

  Agents / Tools / Toolkits
  - @hwchase17

  VectorStores / Retrievers / Memory
  - @dev2049

 -->
[Feature][VectorStore] Support StarRocks as vector db
Labels: name: enhancement, name: area: vector store

<!--
Thank you for contributing to LangChain! Your PR will appear in our release under the title you set. Please make sure it highlights your valuable contribution.

Replace this with a description of the change, the issue it fixes (if applicable), and relevant context. List any dependencies required for this change.

After you're done, someone will review your PR. They may suggest improvements. If no one reviews your PR within a few days, feel free to @-mention the same people again, as notifications can get lost.

Finally, we'd love to show appreciation for your contribution - if you'd like us to shout you out on Twitter, please also include your handle!
-->

<!-- Remove if not applicable -->

Fixes # (issue)

#### Before submitting

<!-- If you're adding a new integration, please include:

1. a test for the integration - favor unit tests that does not rely on network access.
2. an example notebook showing its use


See contribution guidelines for more information on how to write tests, lint
etc:

https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
-->

Here are some examples to use StarRocks as vectordb

```
from langchain.vectorstores import StarRocks
from langchain.vectorstores.starrocks import StarRocksSettings

embeddings = OpenAIEmbeddings()

# conifgure starrocks settings
settings = StarRocksSettings()
settings.port = 41003
settings.host = '127.0.0.1'
settings.username = 'root'
settings.password = ''
settings.database = 'zya'

# to fill new embeddings
docsearch = StarRocks.from_documents(split_docs, embeddings, config = settings)   


# or to use already-built embeddings in database.
docsearch = StarRocks(embeddings, settings)
```

#### Who can review?

Tag maintainers/contributors who might be interested:

@dev2049 

<!-- For a quicker response, figure out the right person to tag with @

  @hwchase17 - project lead

  Tracing / Callbacks
  - @agola11

  Async
  - @agola11

  DataLoaders
  - @eyurtsev

  Models
  - @hwchase17
  - @agola11

  Agents / Tools / Toolkits
  - @hwchase17

  VectorStores / Retrievers / Memory
  - @dev2049

 -->
Issue: Update OpenAI model token mapping to reflect new API update 2023-06-13

### Issue you'd like to raise.

The blog post here
https://openai.com/blog/function-calling-and-other-api-updates

specifies
>  - new 16k context version of gpt-3.5-turbo (vs the standard 4k version)

The `langchain/llms/openai.py` `model_token_mapping` should be changed to reflect this.


### Suggestion:

Add `gpt-3.5-turbo-16k` property to `model_token_mapping` with value 16k
Add mmr support for opensearch

### Fixes  [(MMR Support for OpenSearch)](https://github.com/hwchase17/langchain/issues/6108)

Added Maximal Marginal Relevance Search to Opensearch 

### Before submitting

Updated [Opensearch Docs](https://python.langchain.com/en/latest/modules/indexes/vectorstores/examples/opensearch.html) with MMR example

### Who can review?

Hi @dev2049 appreciate if you could review. 

Twitter: https://twitter.com/oneryalcin
Chooch AI Integration
Labels: name: enhancement, name: llms, name: needs documentation


Fixes # 
-  Added chooch.ai LLM wrapper integration.
-  Added delete_collection_file and similarity_search_with_score_chooch to chroma.py file.  You can discard these.  It useful for robust operations.

Example usage:

api_url = "http://localhost:8082"
from langchain.llms import ChoochAI
llm = ChoochAI(api_url=api_url)

This is used with our ImageChat 3 MultiModal Self Hosted deployment.  We will have a Cloud version ready soon too.


#### Who can review?

Tag maintainers/contributors who might be interested:

@hwchase17,  @dev2049, @agola11
feat: interfaces for async embeddings, implement async openai
Labels: name: enhancement

This change adds support to the base `Embeddings` class for two methods, `aembed_query` and `aembed_documents`, those two methods supporting async equivalents of `embed_query` and
`embed_documents` respectively. This ever so slightly rounds out async support within langchain, with an initial implementation of this functionality being implemented for openai.

Implements #6109
Add new openai 0613 model costs

Added costs for gpt-4-32k-0613, gpt-4-0613, gpt-3.5-turbo-16k, gpt-3.5-turbo-0613, and gpt-3.5-turbo-16k-0613 to openai_info callback based on this [OpenAI post](https://openai.com/blog/function-calling-and-other-api-updates)

@agola11
Async request support for Embeddings, with initial support for OpenAI

### Feature request

The proposal here is pretty simple, we add two methods to the `Embeddings` base class, `aembed_documents` and `aembed_query`, allowing for async versions of the equivalent synchronous methods. The first implementation of this would be for OpenAI, since that's a popular embedding API.

### Motivation

async is supported within other aspects of langchain, and embeddings are one location where support isn't presently there. For a specific example, in a service my company is currently working on converting to be async, this support would improve the throughput of that support noticeably.

### Your contribution

I have a PR in the works that I'll be putting up shortly.
MMR Support for OpenSearch

### Feature request

Currently few vectorstores like Qdrant has support for MMR (Maximal Marginal Relevance)
Opensearch does not have it. 

### Motivation

Since we use OpenSearch as our Vectorstore and we want variance in our results for best entropy I'd like to have MMR implemented for `OpenSearchVectorStore`

### Your contribution

My PR: https://github.com/hwchase17/langchain/pull/6116
update web_base.py to have verify option
Labels: name: lgtm

We propose an enhancement to the web-based loader initialize method by introducing a "verify" option. This enhancement addresses the issue of SSL verification errors encountered on certain web pages. By providing users with the option to set the verify parameter to False, we offer greater flexibility and control.
<!--
Thank you for contributing to LangChain! Your PR will appear in our release under the title you set. Please make sure it highlights your valuable contribution.

Replace this with a description of the change, the issue it fixes (if applicable), and relevant context. List any dependencies required for this change.

After you're done, someone will review your PR. They may suggest improvements. If no one reviews your PR within a few days, feel free to @-mention the same people again, as notifications can get lost.

Finally, we'd love to show appreciation for your contribution - if you'd like us to shout you out on Twitter, please also include your handle!
-->

### Fixes #6079 

#### Who can review?
@eyurtsev @hwchase17
Update web_base.py

We propose an enhancement to the web-based loader initialize method by introducing a "verify" option. This enhancement addresses the issue of SSL verification errors encountered in certain web pages. By providing users with the option to set the verify parameter to False, we offer greater flexibility and control.

<!--
Thank you for contributing to LangChain! Your PR will appear in our release under the title you set. Please make sure it highlights your valuable contribution.

Replace this with a description of the change, the issue it fixes (if applicable), and relevant context. List any dependencies required for this change.

After you're done, someone will review your PR. They may suggest improvements. If no one reviews your PR within a few days, feel free to @-mention the same people again, as notifications can get lost.

Finally, we'd love to show appreciation for your contribution - if you'd like us to shout you out on Twitter, please also include your handle!
-->

<!-- If you're adding a new integration, please include:

1. a test for the integration - favor unit tests that does not rely on network access.
2. an example notebook showing its use


See contribution guidelines for more information on how to write tests, lint
etc:

https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
-->

#### Who can review?
@eyurtsev
Support and make use of function calling and other OpenAI updates on 2023-06-13

### Feature request

OpenAI released several major updates today (2023-06-13) that likely have major implications for what is possible. At the very least, it will make things more reliable.

Here's a shortlist from [the blog post](https://openai.com/blog/function-calling-and-other-api-updates):

- Dramatically improved function calling support + JSON consistency
- `gpt-3.5-turbo` with a 16K context window (🤯)
- Token cost changes for completions and embeddings
- Upcoming deprecation for March-versioned models

### Motivation

The release of OpenAI's blog post found here:

https://openai.com/blog/function-calling-and-other-api-updates


I'm adding this issue mostly to flag and track this OpenAI release and kick off a forum for discussion.

### Your contribution

Can potentially add PRs but haven't contributed here previously.
This new feature return_intermediate_steps=True is not working as intended, can you please update

https://github.com/hwchase17/langchain/blame/ee3d0513addae0680a30afa384431a032244e76b/langchain/chains/graph_qa/cypher.py#L40

@tomasonjo This new feature **return_intermediate_steps=True** is not working as intended, can you please update

There is some issue in line number 258 of "langchain\chains\base.py"
 **return self(args[0], callbacks=callbacks)[self.output_keys[0]]** 
the above line always makes it to return only the result and not the intermediate steps.

As a temporary solution I modified line no: 130 of "langchain\chains\graph_qa\cypher.py" to return **{"result":chain_result}** which works.

Thanks in advance!
OpenAI Cost calculation could have a bug!

### System Info

I've run a prompt that said `1 + 1 = ?` with my agent, I've used `get_openai_callback` to show some metric (see the image):

![test](https://i.imgur.com/YS2pBh7.png)

Used LLM model is `GPT-3.5-turbo`

On OpenAI website we have for `GPT-3.5-turbo` model a  `0.002/1k tokens`, that means my test/prompt should cost:
2206 ÷ 1000 x 0.002 = 0.004412

The weird thing based on my test is the cost, it shows `$0.04412`  and not `$0.004412`

It could be a bug, Any ideas?

Here is the code:
```python
with get_openai_callback() as cb:
    response = agent.run(prompt)

    # Show OpenAI Cost
    print(f"Total Tokens: {cb.total_tokens}")
    print(f"Prompt Tokens: {cb.prompt_tokens}")
    print(f"Completion Tokens: {cb.completion_tokens}")
    print(f"Total Cost (USD): ${cb.total_cost}")
```

Can anyone please explain what's going on?
Thanks in advance.

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [X] Callbacks/Tracing
- [ ] Async

@agola11
@hwchase17
A change in the output format of documents loaded with CSVLoader: (+ weird redundancy in the output)

### System Info

from this format (example):
[Document(page_content='Team: **Athletics**'', metadata={'source': '**my source1**', 'row': **0**}, lookup_index=0), Document(page_content='Team: **Rangers**', lookup_str='', metadata={'source': '**my source2**', 'row': **1**}, lookup_index=0), 
Document(page_content='Team: **Yankees**', lookup_str='', metadata={'source': '**my source3**', 'row': **2**},lookup_index=0)]

To this:
[Document(**lc_kwargs**={page_content='Team: **Athletics**'', metadata={'source': '**my source1**', 'row': **0**}, lookup_index=0),
page_content='Team: **Athletics**'', metadata={'source': '**my source1**', 'row': **0**}, lookup_index=0),

Document(**lc_kwargs**={page_content='Team: **Rangers**', lookup_str='', metadata={'source': '**my source2**', 'row': **1**}, lookup_index=0),page_content='Team: **Rangers**', lookup_str='', metadata={'source': '**my source2**', 'row': **1**}, lookup_index=0), 

Document(**lc_kwargs**={page_content='Team: **Yankees**', lookup_str='', metadata={'source': '**my source3**', 'row': **2**},lookup_index=0),
page_content='Team: **Yankees**', lookup_str='', metadata={'source': '**my source3**', 'row': **2**},lookup_index=0)]


### Who can help?

@hwchase17 @eyurtsev

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [X] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Check out my colab notebook for better understanding:
https://colab.research.google.com/drive/1w8ZTAkapRev8KHI9w4IAk37T9GfFGWW2?usp=drive_link#scrollTo=tl502z9-RRJC

### Expected behavior

expected output (as you can see in the official doc: [https://python.langchain.com/en/latest/modules/indexes/document_loaders/examples/csv.html] ) : 

[Document(page_content='Team: Nationals\n"Payroll (millions)": 81.34\n"Wins": 98', lookup_str='', metadata={'source': './example_data/mlb_teams_2012.csv', 'row': 0}, lookup_index=0), 
Document(page_content='Team: Reds\n"Payroll (millions)": 82.20\n"Wins": 97', lookup_str='', metadata={'source': './example_data/mlb_teams_2012.csv', 'row': 1}, lookup_index=0)]
Run event handler inline if it's not an async function

When we try to run a callback as part of an async call, we can potentially get an async function.
- If that's the case, we should call await on it to make sure it's finished before we move on. 
- If it's not an async function, is a plain old function, and we should just call it.

This makes the setup slightly cleaner because it's not creating a new context several times for one callback.

@hwchase17, @agola11. Twitter username is @mariokostelac.
when comparison.value is int the "valueText" should change to "valueInt"

### Feature request

when comparison.value is int the "valueText" should change to "valueInt"

### Motivation

when comparison.value is int the "valueText" should change to "valueInt"

### Your contribution

self_query.weaviate
```python
  def visit_comparison(self, comparison: Comparison) -> Dict:
    if isinstance(comparison.value,(int,float)):
        return {
          "path": [comparison.attribute],
          "operator": self._format_func(comparison.comparator),
          "valueInt": comparison.value,
      }
    else:
        return {
          "path": [comparison.attribute],
          "operator": self._format_func(comparison.comparator),
          "valueText": comparison.value,
      }
``
'ValueError: "SQLiteEntityStore" object has no field "conn"' error for tutorial "Entity Memory with SQLite storage"

### System Info

LangChain version: 0.0.198
Platform: Ubuntu 20.04 LTS
Python version: 3.10.4

### Who can help?

_No response_

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

1. Reproduce tutorial ["Entity Memory with SQLite storage"](https://python.langchain.com/en/latest/modules/memory/examples/entity_memory_with_sqlite.html)
2. While executing the following code: ```entity_store = SQLiteEntityStore()```,
get the error: ValueError: "SQLiteEntityStore" object has no field "conn"

### Expected behavior

SQLiteEntityStore() must be executed correctly.
Non English Language Support in GoogleSearchAPIWrapper

### Feature request

While, I could pass gl and hl parameters in SerperAPI, one cannot do it for the GoogleSearchAPIWrapper even though the CSE API supports it. 

### Motivation

This should be a priority addition to make the library more inclusive. I have tried passing it with serper and it does a great job at language likes German, Hindi, French, Spanish.

### Your contribution

I tried to see the base utility, but couldn't figure out a way to add the gl, hl parameters in the base utility file.
Guardrails output parser: Pass LLM api for reasking

<!--
Thank you for contributing to LangChain! Your PR will appear in our release under the title you set. Please make sure it highlights your valuable contribution.

Replace this with a description of the change, the issue it fixes (if applicable), and relevant context. List any dependencies required for this change.

After you're done, someone will review your PR. They may suggest improvements. If no one reviews your PR within a few days, feel free to @-mention the same people again, as notifications can get lost.

Finally, we'd love to show appreciation for your contribution - if you'd like us to shout you out on Twitter, please also include your handle!
-->

<!-- Remove if not applicable -->

Fixes https://github.com/ShreyaR/guardrails/issues/155 

Enables guardrails reasking by specifying an LLM api in the output parser.
Issue: If DynamoDB table does not exist conversation message history fails with "local variable 'response' referenced before assignment"

### System Info

If the dyanamodb table does not exist when retrieving conversation history then a generic "local variable 'response' referenced before assignment" error is returned. This is due to the exception handling at https://github.com/hwchase17/langchain/blob/master/langchain/memory/chat_message_histories/dynamodb.py#L50 does not determine if the table does not exist

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

1. Create a dynamodb chat message history, initialised with a DynamoDB table name that doesn't exist
2. Interact with the message history

### Expected behavior

An error should be logged stating that the dynamodb table was not found
Conversational Retriever Chain - condense_question_prompt parameter is not being considered.

### System Info

Langchain 0.0.198,
Python 3.10
AWS Sagemaker environment

### Who can help?

@agola11, @hwchase17

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

`from langchain.chains import ConversationalRetrievalChain
import json
from langchain.prompts.prompt import PromptTemplate
prompt_template =  """Answer based on context

Context: {context}

Question: {question}"""
TEST_PROMPT = PromptTemplate(
    template=prompt_template, input_variables=["context", "question"]
)

question = 'How do I bake cake?'

chain = ConversationalRetrievalChain.from_llm(llm=llm, condense_question_prompt=TEST_PROMPT, retriever=retriever, return_source_documents=True, verbose=True)
chat_history = []

chain({ "chat_history": chat_history, "question": question})

`

### Expected behavior

Expected behavior is that it should take the given TEST_PROMPT, while sending the PROMPT to LLM, which is not doing in the original behavior
Improve AnalyticDB Vector Store implementation without affecting user 

Hi there:

As I implement the AnalyticDB VectorStore use two table to store the document before. It seems just use one table is a better way. So this commit is try to improve AnalyticDB VectorStore implementation without affecting user behavior:

**1. Streamline the `post_init `behavior by creating a single table with vector indexing.
2. Update the `add_texts` API for document insertion.
3. Optimize `similarity_search_with_score_by_vector` to retrieve results directly from the table.
4. Implement `_similarity_search_with_relevance_scores`.
5. Add `embedding_dimension` parameter to support different dimension embedding functions.**

Users can continue using the API as before. 
Test cases added before is enough to meet this commit.