delayedkarma/langchain-issues huggingface dataset:
add vikingdb and embedding
Labels: name: lgtm, name: area: embeddings, name: auto:enhancement, name: size:XL

<!-- Thank you for contributing to LangChain!

Please title your PR "<package>: <description>", where <package> is whichever of langchain, community, core, experimental, etc. is being modified.

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes if applicable,
  - **Dependencies:** any dependencies required for this change,
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` from the root of the package you've modified to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://python.langchain.com/docs/contributing/

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/docs/integrations` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
add vikingdb and embedding
Labels: name: area: embeddings, name: auto:enhancement, name: size:L

<!-- Thank you for contributing to LangChain!

Please title your PR "<package>: <description>", where <package> is whichever of langchain, community, core, experimental, etc. is being modified.

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes if applicable,
  - **Dependencies:** any dependencies required for this change,
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` from the root of the package you've modified to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://python.langchain.com/docs/contributing/

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/docs/integrations` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
refactored `BaseChatMemory`, `ChatMessageHistory`, `get_prompt_input_key`
Labels: name: area: memory, name: auto:refactor, name: size:L

Moved `BaseChatMemory`, `ChatMessageHistory`, `get_prompt_input_key` into `langchain_core`
`BaseChatMemory`, `ChatMessageHistory` are base classes. The `get_prompt_input_key` was moved because of dependency.
Also fixed an error in imports in `langchain_experimental/autonomous_agents/autogpt/memory.py`
community: Semanticscholar tool to search 200M+ scientific articles
Labels: name: area: agent, name: lgtm, name: auto:enhancement, name: size:L

  - **Description:** Tool now supports querying over 200 million scientific articles, vastly expanding its reach beyond the 2 million articles accessible through Arxiv. This update significantly broadens access to the entire scope of scientific literature.
  - **Dependencies:** semantischolar https://github.com/danielnsilva/semanticscholar
  - **Twitter handle:** @shauryr
community: Refactor OpenAIWhisperParserLocal
Labels: name: lgtm, name: area: doc loader, name: auto:refactor, name: size:M

This PR addresses an issue in OpenAIWhisperParserLocal where requesting CUDA without availability leads to an AttributeError #15143 

Changes:

   - Refactored Logic for CUDA Availability: The initialization now includes a check for CUDA availability. If CUDA is not available, the code falls back to using the CPU. This ensures seamless operation without manual intervention.
   - Parameterizing Batch Size and Chunk Size: The batch_size and chunk_size are now configurable parameters, offering greater flexibility and optimization options based on the specific requirements of the use case.
feat: semantic scholar agent tools

<!-- Thank you for contributing to LangChain!

Please title your PR "<package>: <description>", where <package> is whichever of langchain, community, core, experimental, etc. is being modified.

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes if applicable,
  - **Dependencies:** any dependencies required for this change,
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` from the root of the package you've modified to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://python.langchain.com/docs/contributing/

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/docs/integrations` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
docs: Update dependencies installation cell in steam toolkit
Labels: name: area: doc loader, name: auto:documentation, name: size:XS

**Description:** `decouple` is not the correct package, it's `python-decouple`, and the notebook cell doesn't compile.

<!-- Thank you for contributing to LangChain!

Please title your PR "<package>: <description>", where <package> is whichever of langchain, community, core, experimental, etc. is being modified.

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes if applicable,
  - **Dependencies:** any dependencies required for this change,
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` from the root of the package you've modified to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://python.langchain.com/docs/contributing/

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/docs/integrations` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
OpenAIWhisperParserLocal fails when specifying cuda device but cuda is not available
Labels: name: area: doc loader, name: auto:bug

### System Info

langchain==0.0.352
langchain-community==0.0.6
langchain-core==0.1.3


### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [X] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

```
from langchain.document_loaders.parsers.audio import  OpenAIWhisperParserLocal
whisper = OpenAIWhisperParserLocal(device="cuda")
```

This fails when cuda is requested but is not available and generates the following error: 
`AttributeError: 'OpenAIWhisperParserLocal' object has no attribute 'lang_model'`

This is cause by the following logic: https://github.com/langchain-ai/langchain/blob/a2d30428237695f076060dec881bae0258123775/libs/community/langchain_community/document_loaders/parsers/audio.py#L176C18-L176C21

### Expected behavior

Provide a more clear error or fall back to CPU.
HuggingFaceHub api can not pass trust_remote_code argument
Labels: name: auto:bug, name: area: models

### System Info

wsl
conda 23.7.4 python 3.8.11

### Who can help?

_No response_

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

```
repo_id = "Qwen/Qwen-1_8B-Chat"
llm = HuggingFaceHub(
    repo_id=repo_id, model_kwargs={"max_length": 128, "temperature": 0.5}
)
llm_chain = LLMChain(prompt=prompt, llm=llm)
print(llm_chain.run(question))
```
output:
``` 
ValueError: Error raised by inference API: The repository for Qwen/Qwen-1_8B-Chat contains custom code which must be executed to correctly load the model. You can inspect the repository content at https://hf.co/Qwen/Qwen-1_8B-Chat.
Please pass the argument `trust_remote_code=True` to allow custom code to be run.
```
similar issue  #6080
docs: updated wrong output in `Upstash Redis Cache` section of LLM Ca…
Labels: name: lgtm, name: area: doc loader, name: auto:documentation, name: integration: redis, name: size:XS

…ching documentation

<!-- Thank you for contributing to LangChain!

Please title your PR "<package>: <description>", where <package> is whichever of langchain, community, core, experimental, etc. is being modified.

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes if applicable,
  - **Dependencies:** any dependencies required for this change,
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` from the root of the package you've modified to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://python.langchain.com/docs/contributing/

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/docs/integrations` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->


  - **Description:** Fixed the wrong output and code block comment in `Upstash Redis` Cache section of LLM Caching documentation, 
  - **Issue:** #15139 ,
  - **Dependencies:** N/A,
  - **Twitter handle:** [@vardhaman722](https://twitter.com/vardhaman722)
DOC: Wrong output in `Upstash Redis Cache` section of LLM Caching documentation 
Labels: name: area: doc loader, name: auto:documentation, name: integration: redis

### Issue with current documentation:

In the current documentations the output of `Upstash Redis Cache` section in LLM Caching documentation seems wrong. The second run after caching is done has wrong output and wrong code and comments written in the code block.

### Idea or request for content:

Update the code block with appropriate comment and matching output to remove the confusion.
Sagemaker Endpoint not working streaming
Labels: name: auto:bug, name: area: models, name: integration: aws

### System Info

LangChain : 0.0.352
Python : 3.11.5



### Who can help?

@hwchase17 

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

1. use streaming

```
llm_sm_ep = SagemakerEndpoint(
    endpoint_name=endpoint_name,
    client=client,
    content_handler=content_handler,
    model_kwargs=model_param,
    endpoint_kwargs=endpoint_param,
    streaming=True,
)
```

### Expected behavior

When I used TGI model, `invoke_endpoint_with_response_stream` of response does't have `outputs`. Instead it return `token` like below full response.

```
data:{"token":{"id":601,"text":" time","logprob":-0.10015869,"special":false},"generated_text":null,"details":null}
```
fix: add stronger contrast or labeling between notebook input and out…
Labels: name: auto:documentation, name: size:S

<!-- Thank you for contributing to LangChain!

Please title your PR "<package>: <description>", where <package> is whichever of langchain, community, core, experimental, etc. is being modified.

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes if applicable,
  - **Dependencies:** any dependencies required for this change,
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` from the root of the package you've modified to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://python.langchain.com/docs/contributing/

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/docs/integrations` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->

  - **Description:** Enhanced the contrast or add labels to distinguish between input and output blocks in the LangChain documentation by modifying the CSS styles for the code blocks in the `custom.css` file referenced in the `docusaurus.config.js` file, 
  - **Issue:** #14532 ,
  - **Dependencies:** N/A,
  - **Twitter handle:** [@vardhaman722](https://twitter.com/vardhaman722)
community: support UAE sentence embeddings
Labels: name: lgtm, name: area: embeddings, name: auto:enhancement, name: size:M


  - **Description:** support loading the current SOTA sentence embeddings [WhereIsAI/UAE](https://huggingface.co/WhereIsAI/UAE-Large-V1) in langchain.
  - **Dependencies:** [angle_emb](https://github.com/SeanLee97/AnglE)
  - **Twitter handle:** @xmlee97
RetryWithErrorOutputParser does not work with LLMChain because it does not implement the `parse` function
Labels: name: auto:improvement, name: area: models

### Feature request

Currently if one wants to use the RetryWithErrorOutputParser - we need to do the parsing manually instead of generating a chain that does it for us (including all the nice chain functions: batch, ainvoke, etc)

There are 2 issues:
1. The RetryWithOutputParser requires the prompt to be given to it as input so that it can do it's magic. It does this by implementing a `parse_with_prompt` function. Unfortunately this function is not plumbed all the way into the `BaseParser` so that when this is invoked as part of a regular chain it gives the `NotImplementedError: This OutputParser can only be called by the parse_with_prompt method.` exception.
2. The default output of the chatmodels is to return just the output or `AIMessages`. However in this case we need both the prompt and the output. 

### Motivation

Currently we need to run the output parsing for the retry parsing manually. This tends to look something like this:
```
        chain = chat_prompt | self.chat_model
        output_batch = chain.batch(messages_batch,
                                   config={"max_concurrency": 10,
                                           "callbacks": [tracing_callback_handler]})

        prompts_list = tracing_callback_handler.prompts
        result_list = tracing_callback_handler.results
        parsed_output_batch = []
        for idx, output in enumerate(output_batch):
            parsed_output = retry_parser.parse_with_prompt(output.content, prompts_list[idx])
            parsed_output_batch.append(parsed_output)
```

In the above code the `tracing_callback_handler` is a custom callback handler that persists the prompt and results - which we end up using to give the retry_parser the prompt.

This is cumbersome and it would be awesome if this would just work with the chain itself like so
```
        chain = chat_prompt | self.chat_model | retry_parser
       output_batch = chain.batch(messages_batch,
                                   config={"max_concurrency": 10,
                                           "callbacks": [tracing_callback_handler]})

```

### Your contribution

If someone can validate that my understanding of the problem is correct - I can go ahead and create a PR for this.
Adding Prompt template to ConversationalRetrievalChain.from_llm
Labels: name: auto:question, name: area: models


# How Adding a prompt template to conversational retrieval chain giving the code:


`template= """Use the following pieces of context to answer the question at the end. 
If you don't know the answer, 
just say that you don't know.
{context}
Question: {question}
Helpful Answer:"""
QA_CHAIN_PROMPT = PromptTemplate.from_template(template)

from langchain.memory import ConversationBufferMemory
memory = ConversationBufferMemory(
    memory_key="chat_history",
    return_messages=True
)

from langchain.chains import ConversationalRetrievalChain
qa = ConversationalRetrievalChain.from_llm(
    llm,
    retriever=db.as_retriever(),
    memory=memory,
    chain_type_kwargs={"prompt": QA_CHAIN_PROMPT}
)
`

`
ValidationError: 1 validation error for ConversationalRetrievalChain
chain_type_kwargs
  extra fields not permitted (type=value_error.extra)` 

How do I add the prompt template to the chain efficiently?

### Suggestion:

How do I add the prompt template to the chain efficiently? Please, I need help with this.
chain.invoke is no longer taking a json as input
Labels: name: auto:bug, name: area: models

### System Info

windows

### Who can help?

@hwchase17
@agola11

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

The below code fails

import os
from operator import itemgetter

from dotenv import load_dotenv
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain_community.document_loaders import DirectoryLoader
from langchain_community.embeddings import OpenAIEmbeddings
from langchain_community.vectorstores.faiss import FAISS
from langchain_core.output_parsers import StrOutputParser
from langchain_core.runnables import RunnableParallel, RunnablePassthrough
from langchain.chat_models import ChatOpenAI
from langchain.prompts import ChatPromptTemplate

load_dotenv()

OPENAI_API_KEY = os.environ.get('OPENAI_API_KEY')

loader = DirectoryLoader("/Users/joyeed/langchain_examples/langchain_examples/data/", glob='**/*.md')
documents = loader.load()

text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=0)
text = text_splitter.split_documents(documents)

embeddings = OpenAIEmbeddings(openai_api_key=OPENAI_API_KEY)
vectorstore = FAISS.from_documents(text, embeddings)

retriever = vectorstore.as_retriever()

prompt_template = ChatPromptTemplate.from_template(
    """
    Write 2 {platform} posts about {topic}?
    """
)

model = ChatOpenAI(openai_api_key=OPENAI_API_KEY)

# Compose the chain for generating posts
chain = (
    {"topic": RunnablePassthrough(), "platform": RunnablePassthrough(), "context": retriever} 
    | prompt_template 
    | model
    | StrOutputParser()
)

# Invoke the chain to generate a post
output = chain.invoke({"topic": "baseball", "platform": "twitter"})

# Print the generated post
print(output)

I think it is failing because the invoke is expecting a string as an input, but earlier we were able to pass key/value pairs. It is failing in tiktoken/core.py in the below code, it expects a text here.

            if match := _special_token_regex(disallowed_special).search(text):
                raise_disallowed_special_token(match.group())





### Expected behavior

invoke should allow accepting JSON inputs
Template issue: Neo4J environmental variables in .env file not found
Labels: name: area: vector store, name: auto:bug

### System Info

langchain version: 0.0.352, Windows 10, Python 3.11.6,


### Who can help?

@eyurtsev 

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

I'm testing a couple of apps created from Langchain templates and I use dotenv and a .env file in the app's root folder ( default in docs: "my-app" )
I'm not able to work with Neo4J data added into it, when trying to run eg.:
**neo4j-advanced-rag-app\packages\neo4j-advanced-rag\ingest.py**
( My .env is in the neo4j-advanced-rag-app folder )
This is strange, because eg. Langsmith related env vars can be used, so I think the issue is not related to all env vars in .env file!

The last terminal error is:
```
Traceback (most recent call last):
  File "d:\Projects\AI_testing\LangChain_test\Python_231026\neo4j-advanced-rag-app\packages\neo4j-advanced-rag\ingest.py", line 16, in <module>
    graph = Neo4jGraph()
            ^^^^^^^^^^^^
  File "D:\Projects\AI_testing\LangChain_test\Python_231026\langchain-venv\Lib\site-packages\langchain_community\graphs\neo4j_graph.py", line 65, in __init__     
    url = get_from_env("url", "NEO4J_URI", url)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\Projects\AI_testing\LangChain_test\Python_231026\langchain-venv\Lib\site-packages\langchain_core\utils\env.py", line 41, in get_from_env
    raise ValueError(
ValueError: Did not find url, please add an environment variable `NEO4J_URI` which contains it, or pass  `url` as a named parameter.
```

### Expected behavior

I expect the app created from a template can use all the env vars in the .env file, which is placed into app root folder.