delayedkarma/langchain-issues huggingface dataset:
updated cohere.py to support input_type for new embedding models
Labels: name: area: embeddings, name: auto:improvement

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** Added input_type field to Cohere API calls for embedding. Newer models require this field be specified for embedding with the value "search_document". Older models still work.
  - **Issue:** Allows any model v2.0 > to work with Langchain.
  - **Dependencies:** 
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** @jay_martz

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
YoutubeLoader translation not working
Labels: name: area: doc loader, name: auto:bug

### System Info

langchain == 0.0.336
python == 3.9.6

### Who can help?

@hwchase17 @eyu

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [X] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Code:

video_id = YoutubeLoader.extract_video_id("https://www.youtube.com/watch?v=DWUzWvv3xz4")
loader = YoutubeLoader(video_id, add_video_info=True,
                                   language=["en","hi","ru"],
                                   translation = "en")

loader.load()

Output:

[Document(page_content='में रुके शोध सजावट हाउ टू लेट सफल टीम मेंबर्स नो दैट वास नॉट फीयर कॉन्टेक्ट्स रिस्पोंड फॉर एग्जांपल एक्टिव एंड स्ट्रैंथ एंड assistant 3000 1000 कॉन्टैक्ट एंड वे नेवर रिग्रेट थे रिस्पांस यू वांट यू ऑटोमेटेकली गेट नोटिफिकेशन टुडे ओके खुफिया इज द नोटिफिकेशन पाइथन इन नोटिफिकेशन सीनेटर नोटिफिकेशन प्यार सेलिब्रेट विन रिस्पोंस वर्षीय बेटे ईमेल एस वेल एजुकेटेड व्हाट्सएप नोटिफिकेशन इफ यू वांट एनी फीयर अदर टीम मेंबर टू ऑल्सो गेट नोटिफिकेशन व्हेन यू व्हेन यू रिसीवर रिस्पांस फ्रॉम अननोन फीयर कांटेक्ट सुधर रिस्पांस सिस्टम लिफ्ट से ज़ू और यह टीम मेंबर कैन रिस्पोंड इम्युनिटी द न्यू कैंट व ईमेल ऐड्रेस आफ थे पर्सन वरीय स्वीडन से लेफ्ट से अब दूर एक पाठ 7 टारगेट्स ऑयल सुबह रायपुर दिस ईमेल एड्रेस नो अब्दुल विल गेट एनी नोटिफिकेशन फ्रॉम assistant साक्षी व्हेनेवर एनी बडी दिस पॉइंट स्ट्रेन ईमेल अब्दुल विल अलसो गेट डर्टी में अगर सब्जेक्ट लाइन रिस्पांस रिसीवड ए [संगीत]', metadata={'source': 'DWUzWvv3xz4', 'title': 'How to Notify your team member Immediately when a Lead Responds', 'description': 'Unknown', 'view_count': 56, 'thumbnail_url': 'https://i.ytimg.com/vi/DWUzWvv3xz4/hq720.jpg', 'publish_date': '2021-10-04 00:00:00', 'length': 87, 'author': '7Targets AI Sales Assistant'})]

### Expected behavior

The output video transcript should be in English.
fix: VLLMOpenAI -- create() got an unexpected keyword argument 'api_key'
Labels: name: lgtm, name: auto:bug, name: area: models

The issue was occurring because of an `OpenAI` update in `Completions`. It's not accepting the `api_key` and `api_base` arguments.

The fix is we check for the openai version and if it's v1 or greater then remove these keys from args before passing them to `Compilation.create(...)` when sending from `VLLMOpenAI`

Fixed: #13507 

@eyu
@efriis 
@hwchase17
Vectorstore relevance fn fix
Labels: name: area: vector store, name: auto:improvement, name: integration: elasticsearch, name: size:M

**Description:**

- The changes fixes the normalization in the relevance score functions.
- Some vector databases use `cosine` to indicate `cosine similarity` while some use it to indicate `cosine distance`.
  - SemaDB uses `cosine` metric to indicate `cosine distance`. https://rapidapi.com/semafind-semadb/api/semadb/details
  - Elasticsearch uses `cosine` metric to indicate `cosine similarity`. https://www.elastic.co/guide/en/elasticsearch/reference/current/dense-vector.html#dense-vector-params
  - Neo4j uses `cosine` metric to indicate `cosine similarity`. https://neo4j.com/docs/cypher-manual/current/indexes-for-vector-search/#indexes-vector-create
  - Momento uses `cosine_similarity` metric to indicate `cosine similarity`. https://github.com/momentohq/client-sdk-python/blob/main/src/momento/requests/vector_index/similarity_metric.py
  - pgvector uses `cosine distance`. https://github.com/pgvector/pgvector#pgvector
  - Pinecone uses `cosine` metric to indicate `cosine similarity`. https://docs.pinecone.io/docs/indexes#distance-metrics
  - FAISS does not directly support `cosine similarity` nor `cosine distance`. https://github.com/facebookresearch/faiss/blob/main/faiss/MetricType.h#L44
  - TimescaleVector is built on top of pgvector, hence uses `cosine distance`.
  - Chroma uses `cosine` metric to indicate `cosine distance`. https://docs.trychroma.com/usage-guide#changing-the-distance-function
  - Redis uses `cosine` metric to indicate `cosine distance`. https://redis.io/docs/interact/search-and-query/advanced-concepts/vectors/

**Issue:**
https://github.com/langchain-ai/langchain/issues/11587
https://github.com/langchain-ai/langchain/issues/13437
https://github.com/langchain-ai/langchain/issues/7384
https://github.com/langchain-ai/langchain/issues/11386
https://github.com/langchain-ai/langchain/issues/12697

Twitter handle: dangoldbj

@baskaryan, @eyurtsev, @hwchase17
SQLiteCache - store only the hash of the prompt as key instead of the entire prompt
Labels: name: area: memory, name: auto:improvement

### Feature request

I see the current SQLiteCache is storing the entire prompt message in the SQLite db.
It would be more efficient to just hash the prompt and store this as the key for cache lookup. 

### Motivation

My prompt messages are often lengthy and I want to optimize the storage requirements of the Cache. 
Keeping an md5 hash as the lookup key would make the lookup also faster rather than doing a string search. 

### Your contribution

If this feature makes sense, I can work on this and raise a PR. Let me know.
Fix `MarkdownHeaderTextSplitter` not recognizing tilde-fenced code blocks
Labels: name: lgtm, name: auto:bug

 - **Description:** Previously `MarkdownHeaderTextSplitter` did not consider tilde-fenced code blocks (https://spec.commonmark.org/0.30/#fenced-code-blocks). This PR fixes that.
   ````md
   # Bug caused by previous implementation:
   ~~~py
   foo()
   # This is a comment that would be considered header
   bar()
   ~~~
   ````
 - **Tag maintainer:** @baskaryan
Text splitters inhéritance
Labels: name: area: vector store, name: auto:bug, name: auto:question

### System Info

I am facing an issue using `MarkdownHeaderTextSplitter` class and, after looking at the code, I noticed that the problem might be present in several Text Splitters and I did not find any issue on this, so I create a new one.

I am trying to use `MarkdownHeaderTextSplitter` regarding the `TextSplitter` interface by calling the method `transform_document`.

However, the [`MarkdownHeaderTextSplitter` does not inherit from `TextSplitter`](https://github.com/langchain-ai/langchain/blob/35e04f204ba3e69356a4f8f557ea88f46d2fa389/libs/langchain/langchain/text_splitter.py#L331) and I wondered if it was a justified implementation or just an oversight.

It seems that the [`HTMLHeaderTextSplitter` is in that case too](https://github.com/langchain-ai/langchain/blob/35e04f204ba3e69356a4f8f557ea88f46d2fa389/libs/langchain/langchain/text_splitter.py#L496).

Can you give me some insight on how to use theses classes if the behavior is normal ?

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

This code is a good way to reproduce what I am trying to do

```python
from langchain.text_splitter import MarkdownHeaderTextSplitter
from langchain.document_loaders import TextLoader

loader = TextLoader("test.md")
document = loader.load()

transformer = MarkdownHeaderTextSplitter([
    ("#", "Header 1"),
    ("##", "Header 2"),
    ("###", "Header 3"),
])

tr_documents = transformer.transform_documents(document)
```

### Expected behavior

I want this to return a list of documents (`langchain.docstore.document.Document`) splitted in the same way `MarkdownHeaderTextSplitter.split_text` does on the content of a markdown document [as presented in the documentation](https://python.langchain.com/docs/modules/data_connection/document_transformers/text_splitters/markdown_header_metadata).
Documentation: Reduce largest static image file size
Labels: name: auto:documentation

 - **Description:** Reduce image asset file size used in documentation by running them via lossless image optimization ([tinypng](https://www.npmjs.com/package/tinypng-cli) was used in this case). Images wider than 1916px (the maximum width of an image displayed in documentation) where downsized.
  - **Issue:** No issue is created for this, but the large image file assets caused slow documentation load times
  - **Dependencies:** No dependencies affected
Sudden failure to initialize
Labels: name: area: embeddings, name: auto:bug, name: area: models

### System Info

I was using 0.0.182-rc.1 and I tried upgrading to the latest, 0.0.192
But I'm still getting the error.
Please note: everything was working fine before, I have made no changes.
Did openai change something?
Not sure what is going on here but it looks like its from openai side. If so how do I fix this? Do I wait for a langchain update?

Error:
/node_modules/openai
/src/error.ts:66
      return new BadRequestError(status, error, message, headers);
             ^
Error: 400 '$.input' is invalid. Please check the API reference: https://p
latform.openai.com/docs/api-reference.
    at Function.generate (/home/hedgehog/Europ/profiling-github/profiling/
server/node_modules/openai/src/error.ts:66:14)
    at OpenAI.makeStatusError (/home/hedgehog/Europ/profiling-github/profi
ling/server/node_modules/openai/src/core.ts:358:21)
    at OpenAI.makeRequest (/home/hedgehog/Europ/profiling-github/profiling
/server/node_modules/openai/src/core.ts:416:24)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at /home/hedgehog/Europ/profiling-github/profiling/server/node_modules
/langchain/dist/embeddings/openai.cjs:223:29
    at RetryOperation._fn (/home/hedgehog/Europ/profiling-github/profiling
/server/node_modules/p-retry/index.js:50:12)

@agola11 

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [X] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Not sure, it broke suddenly without any changes

### Expected behavior

It would initialize properly and execute all the onModuleLoad functions in the Nest js application with the embeddings ready for use.
Found Potential Bug! (langchain > llm > base.py)
Labels: name: auto:bug, name: area: models

### System Info

Location: langchain/lllms/base.py
I think there's a hidden error in a 'generate' method. (line 554)

Inside of the second 'if' statement, it calls the index of callbacks that may cause the potential error.
`Callbacks` is an object of `CallbackManager`, which is not iterable, so it can not be called as `callbacks[0]`. (see the source code below)

Thanks for @Seuleeee




### Who can help?

@hwchase17 @agola11 

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [X] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [X] Callbacks/Tracing
- [ ] Async

### Reproduction

langchain/lllms/base.py

```
      if (
            isinstance(callbacks, list)
            and callbacks
            and (
                isinstance(callbacks[0], (list, BaseCallbackManager))
                or callbacks[0] is None
            )
        ):
```


Error message

```
Traceback (most recent call last):
  File "/tmp/ipykernel_713/4197562787.py", line 10, in test_rag
    result=conv_chain.run(query)
  File "/usr/local/lib/python3.8/dist-packages/langchain/chains/base.py", line 505, in run
    return self(args[0], callbacks=callbacks, tags=tags, metadata=metadata)[
  File "/usr/local/lib/python3.8/dist-packages/langchain/chains/base.py", line 310, in __call__
    raise e
  File "/usr/local/lib/python3.8/dist-packages/langchain/chains/base.py", line 304, in __call__
    self._call(inputs, run_manager=run_manager)
  File "/usr/local/lib/python3.8/dist-packages/langchain/chains/conversational_retrieval/base.py", line 159, in _call
    answer = self.combine_docs_chain.run(
  File "/usr/local/lib/python3.8/dist-packages/langchain/chains/base.py", line 510, in run
    return self(kwargs, callbacks=callbacks, tags=tags, metadata=metadata)[
  File "/usr/local/lib/python3.8/dist-packages/langchain/chains/base.py", line 310, in __call__
    raise e
  File "/usr/local/lib/python3.8/dist-packages/langchain/chains/base.py", line 304, in __call__
    self._call(inputs, run_manager=run_manager)
...
  File "/usr/local/lib/python3.8/dist-packages/langchain/llms/base.py", line 617, in generate
    isinstance(callbacks[0], (list, BaseCallbackManager)
TypeError: 'CallbackManager' object is not subscriptable
```

### Expected behavior

Revise the code that has a potential error.
Tell me how can I contribute to fix this code. (test code ...etc)
implement memory for RetrievalQA.from_chain_type()
Labels: name: area: memory, name: auto:enhancement, name: area: models

### Feature request

can you add memory for RetrievalQA.from_chain_type(). I haven't seen any implementation of memory for this kind of RAG chain. It would be nice to have memory and ask questions in context.

### Motivation

I just can't get any memory to work with RetrievalQA.from_chain_type().

### Your contribution

Not right now... I don't have all required knowledge about LLMs
FIX: doc-string - langchain.vectorstores.dashvector.DashVector
Labels: name: lgtm, name: area: vector store, name: auto:documentation

- **Description:** There are several mistakes in the sample code in the doc-string of `DashVector` class, and this pull request aims to correct them.
  The correction code has been tested against latest version (at the time of creation of this pull request) of: `langchain==0.0.336` `dashvector==1.0.6` .
- **Issue:** No issue is created for this.
- **Dependencies:** No dependency is required for this change,
<!--  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below), -->
- **Twitter handle:** `zeyanglin`

<!-- Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
-->
Add rag-opensearch template
Labels: name: auto:enhancement, name: template, name: size:XXL

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->

Adding rag-opensearch template.
Why does Redis vector store misses some keys in redistext search. although key and related details exists in index.Issue
Labels: name: area: vector store, name: auto:bug, name: auto:improvement, name: integration: redis

### Issue you'd like to raise.

**Description:**
The Redistext search feature in Redis Vector Store functions well with a small number of indexed documents (20-30). However, when the quantity of indexed documents exceeds 400-500, the performance degrades noticeably. Some keys are missed during the Redistext search, and Redis Similarity search retrieves incorrect keys.

**Steps to Reproduce:**

1. Store 400-500 documents in an Index of Redis vector store database.
2. Conduct Redistext search and observe that it is not able to find some of the stored keys.
3. Use Redis Similarity search and notice retrieval of incorrect keys, for some number of keys that are already stored in REdis vector store database.

**Additional Details:**

The issue is not present with smaller datasets (20-30 indexed documents) .
All problematic keys are confirmed to be stored in the Redis Vector Store database with double-checked hash values.

**Expected Behavior:**
Redistext search and Redis Similarity search should provide accurate results even with larger datasets.

**Environment:**

- Official Redis stack docker image (redis/redis-stack-server:latest)
- Docker version 24.0.2

**Any guidance or solutions to improve search performance would be highly appreciated.**

Thank you.

### Suggestion:

**Optimize Redistext Search and Redis Similarity Search for Larger Datasets**

**Proposed Solution:**
Given the observed performance degradation with Redistext search and Redis Similarity search when handling larger datasets (400-500 indexed documents), I suggest reassessing the Redis index similarity search and Redistext search functions. The objective is to ensure accurate results even when the dataset is extensive.
faiss中的错误
Labels: name: area: vector store, name: auto:bug

### System Info

When using fail, similarity_search_with_score function, the parameters filter and score_threshold together, the results will be problematic

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

1.use similarity_search_with_score
2.parameter：filter and score_threshold

### Expected behavior

result is error
Adding messages to history doesn't work with custom message types
Labels: name: area: memory, name: area: agent, name: auto:bug, name: area: models

### System Info

langchain 0.0.320
MacOS 13.14.1
Python 3.9.18

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [X] Memory
- [X] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

I'm using `DynamoDBChatMessageHistory` for storing chat messages. However, I do not use `Human` or `AI` as my chat prefixes. Using pre-defined methods `add_user_message` and `add_ai_message` won't work for me. I extended the `BaseMessage` class to create a new message type. I get this error when trying to read messages from history.

```
 raise ValueError(f"Got unexpected message type: {_type}")
ValueError: Got unexpected message type: User
```

Here's my code:

```
import uuid

from typing_extensions import Literal
from langchain.memory.chat_message_histories import DynamoDBChatMessageHistory
from langchain.schema.messages import BaseMessage


class UserMessage(BaseMessage):
    type: Literal["User"] = "User"


class BotMessage(BaseMessage):
    type: Literal["Bot"] = "Bot"


session_id = str(uuid.uuid4())
history = DynamoDBChatMessageHistory(
    table_name="tableName",
    session_id=session_id,
    primary_key_name='sessionId'
)

# history.add_user_message("hi!")  # works fine
# history.add_ai_message("whats up?")  # works fine

history.add_message(UserMessage(content="Hello, I'm the user!"))  # saves the message to the table with the correct message type
history.add_message(BotMessage(content="Hello, I'm the bot!!"))  # doesn't run due to the above-encountered error

print(history.messages)  # throws a ValueError
```

### Expected behavior

The error should not be thrown so that other execution steps are completed.
Neptune graph updates
Labels: name: lgtm, name: auto:improvement, name: area: models

## Description
This PR adds an option to allow unsigned requests to the Neptune database when using the `NeptuneGraph` class.

```python
graph = NeptuneGraph(
    host='<my-cluster>',
    port=8182,
    sign=False
)
```

Also, added is an option in the `NeptuneOpenCypherQAChain` to provide additional domain instructions to the graph query generation prompt. This will be injected in the prompt as-is, so you should include any provider specific tags, for example `<instructions>` or `<INSTR>`.

```python
chain = NeptuneOpenCypherQAChain.from_llm(
    llm=llm,
    graph=graph,
    extra_instructions="""
    Follow these instructions to build the query:
    1. Countries contain airports, not the other way around
    2. Use the airport code for identifying airports
    """
)
```
renamed `google_vertex_ai_vector_search` notebook
Labels: name: lgtm, name: area: vector store, name: auto:documentation, name: size:S

The `integrations/vectorstores/matchingengine.ipynb` example has the "Google Vertex AI Vector Search" title. This place this Title in the wrong order in the ToC (it is sorted by the file name).
- Renamed  `integrations/vectorstores/matchingengine.ipynb` into  `integrations/vectorstores/google_vertex_ai_vector_search.ipynb`.
- Updated a correspondent comment in docstring
- Rerouted old URL to a new URL