delayedkarma/langchain-issues huggingface dataset:
feat: allow users to pass local embeddings to Weaviate Hybrid Search …
Labels: name: area: embeddings, name: auto:improvement, name: size:XS

…Retriever #5539

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
DOCS: fixed error in the docstring of RunnablePassthrough class
Labels: name: lgtm, name: auto:documentation, name: size:XS

This pull request addresses an issue found in the example code within the docstring of `libs/core/langchain_core/runnables/passthrough.py`

The original code snippet caused a `NameError` due to the missing import of `RunnableLambda`. The error was as follows:
```
     12     return "completion"
     13 
---> 14 chain = RunnableLambda(fake_llm) | {
     15     'original': RunnablePassthrough(), # Original LLM output
     16     'parsed': lambda text: text[::-1] # Parsing logic

NameError: name 'RunnableLambda' is not defined
```
To resolve this, I have modified the example code to include the necessary import statement for `RunnableLambda`. Additionally, I have adjusted the indentation in the code snippet to ensure consistency and readability.

The modified code now successfully defines and utilizes `RunnableLambda`, ensuring that users referencing the docstring will have a functional and clear example to follow.

There are no related GitHub issues for this particular change.

Modified Code:
```python
from langchain_core.runnables import RunnablePassthrough, RunnableParallel
from langchain_core.runnables import RunnableLambda

runnable = RunnableParallel(
    origin=RunnablePassthrough(),
    modified=lambda x: x+1
)

runnable.invoke(1) # {'origin': 1, 'modified': 2}

def fake_llm(prompt: str) -> str: # Fake LLM for the example
    return "completion"

chain = RunnableLambda(fake_llm) | {
    'original': RunnablePassthrough(), # Original LLM output
    'parsed': lambda text: text[::-1] # Parsing logic
}

chain.invoke('hello') # {'original': 'completion', 'parsed': 'noitelpmoc'}
```
Does only openai support SystemMessage?
Labels: name: auto:question, name: area: models

### System Info

None

### Who can help?

_No response_

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

My chat model is ERNIE-Bot, running the following code reported an error：
![image](https://github.com/langchain-ai/langchain/assets/45581281/8aee32db-9ca8-4fb3-aec4-4617219c375d)
But after I removed SystemMessage it works fine. So I want to know, do all models support SystemMessage?
![image](https://github.com/langchain-ai/langchain/assets/45581281/df82fd89-944b-4e51-bdc4-1fc121106731)


### Expected behavior

None
Update llamacpp.ipynb
Labels: name: lgtm, name: auto:documentation, name: size:XS

specifed -> specified

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
The experimental package needs to be compatible with the usage of importing agents
Labels: name: area: agent, name: lgtm, name: auto:bug, name: size:S


  - **Description:** The experimental package needs to be compatible with the usage of importing agents

For example, if i use `from langchain.agents import create_pandas_dataframe_agent`, running the program will prompt the following information:

```
Traceback (most recent call last):
   File "/Users/dongwm/test/main.py", line 1, in <module>
     from langchain.agents import create_pandas_dataframe_agent
   File "/Users/dongwm/test/venv/lib/python3.11/site-packages/langchain/agents/__init__.py", line 87, in __getattr__
     raise ImportError(
ImportError: create_pandas_dataframe_agent has been moved to langchain experimental. See https://github.com/langchain-ai/langchain/discussions/11680 for more information.
Please update your import statement from: `langchain.agents.create_pandas_dataframe_agent` to `langchain_experimental.agents.create_pandas_dataframe_agent`.
```

But when I changed to `from langchain_experimental.agents import create_pandas_dataframe_agent`, it was actually wrong:

```python
Traceback (most recent call last):
  File "/Users/dongwm/test/main.py", line 2, in <module>
    from langchain_experimental.agents import create_pandas_dataframe_agent
ImportError: cannot import name 'create_pandas_dataframe_agent' from 'langchain_experimental.agents' (/Users/dongwm/test/venv/lib/python3.11/site-packages/langchain_experimental/agents/__init__.py)
```

I should use `from langchain_experimental.agents.agent_toolkits import create_pandas_dataframe_agent`. In order to solve the problem and make it compatible, I added additional import code to the langchain_experimental package. Now it can be like this Used `from langchain_experimental.agents import create_pandas_dataframe_agent`

  - **Twitter handle:** [lin_bob57617](https://twitter.com/lin_bob57617)
Unable to get page count. Is poppler installed and in PATH
Labels: name: area: vector store, name: auto:bug, name: area: models

### System Info

im running it on google Collab

### Who can help?

trying the example of mult-modal rag - I tried everything no matter what if still getting this error
please tell if if there is any alternative way or how can we install it? @bas


### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

just run it on collab & we will not able to get output from partitions

https://github.com/langchain-ai/langchain/blob/master/cookbook/Multi_modal_RAG.ipynb

![image](https://github.com/langchain-ai/langchain/assets/62583018/2f10bb44-82e9-4dc9-9c66-e84b5a18c6e1)


### Expected behavior

it should work normal without error on collab
Astra DB, chat message history
Labels: name: area: memory, name: auto:enhancement, name: size:L

This PR adds a chat message history component that uses Astra DB for persistence through the JSON API.
The `astrapy` package is required for this class to work.

I have added tests and a small notebook, and updated the relevant references in the other docs pages.

(@rlancemartin this is the counterpart of the Cassandra equivalent class you so helpfully reviewed back at the end of June)

Thank you!
Astra DB, LLM cache classes (exact-match and semantic cache)
Labels: name: lgtm, name: auto:enhancement, name: area: models, name: size:XL

This PR provides idiomatic implementations for the exact-match and the semantic LLM caches using Astra DB as backend through the database's HTTP JSON API. These caches require the `astrapy` library as dependency.

Comes with integration tests and example usage in the `llm_cache.ipynb` in the docs.

@baskaryan this is the Astra DB counterpart for the Cassandra classes you merged some time ago, tagging you for your familiarity with the topic. Thank you!
lang chain Azure vector search not working neither on its direct fields nor on its metadata fields 
Labels: name: area: vector store, name: auto:bug

Please help , I am blocked on it from many days .

**I am trying to filter question of answer from pdf doc based on uploaded email filter . Only if email match happen answer the question from that document else just empty .
I tried following code its not working rather even in case of OTHER email id also it  give answer from my email uploaded document which is wrong , seems like filtering not working .**
        
First i tried by putting email as metadata it did not work then i added as independent email field its not working there either .

I am instantiating vector store like below , it has email field in it . I am able to create and successfully able to upload pdf doc with all the fields  including email is populated . I checked created index and email field is filterable , searchable .

- Creating azure instance as below :
`
`self.vector_store = AzureSearch(azure_search_endpoint=endpoint, azure_search_key=admin_key, index_name=index_name, embedding_function=embedding_function,fields=fields)`

- After doc uploaded as shown below it has email in it .

```

"@odata.context": "https://ops-documents.search.windows.net/indexes('index-new-5')/$metadata#docs(*)",
  "value": [
    {
      "@search.score": 1,
      "id": "NjQ1YWViNWQtNDJkNy00NTcxLTlkMTktMDIzZTc0NTZlNDhm",
      "content": "ICAO TRIP 2023 –  .",
"metadata": "{\"source\": \"7-pl\", \"page\": 7, \"file_id\": \"65612fd773a9aa51a0939c96\", \"upload_document_name\": \"SITA Lab Furhat Backoffice IMM officer Dialog - v1.pdf\", \"email\": \"savita.raghuvanshi@st.com\", \"company\": \"sita\"}",
     "email": "savita.raghuvanshi@st.com"
```
 

- 
- Using following way to search :       

       
```
my_retriever = self.vector_store.as_retriever(search_kwargs={'filter': {  'email': email }})
qa = RetrievalQA.from_chain_type(llm=self.llm,chain_type="stuff",retriever=my_retriever,chain_type_kwargs={"prompt": self.PROMPT},
            return_source_documents=True,
        )
        results = qa({"query": question})
```

```
Installations using as : 
**I am using OPENAI_API_VERSION="2023-10-01-preview" with gpt4 model** 
azure-common==1.1.28
azure-core==1.29.5
azure-identity==1.15.0
azure-search==1.0.0b2
azure-search-documents==11.4.0b8
langchain==0.0.326
```

Kindly let me know if anything needed from me . Thanks so much for your help .
Multitenant adddocs
Labels: name: lgtm, name: area: vector store, name: auto:improvement, name: size:XS

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** Added a line to pass the tenant parameter to add_data_object
  - **Issue:** An extra line added from the fix for #9956
  - **Dependencies:** n/a
  - **Tag maintainer:** @baskaryan 

Tested locally, works as expected with the line change.
Langchain Database Agent with local LLM
Labels: name: area: agent, name: auto:bug, name: auto:question, name: area: models

I am using local LLM with langchain: openhermes-2.5-mistral-7b.Q8_0.gguf

When using database agent this is how I am initiating things:

`db = SQLDatabase.from_uri(sql_uri)

model_path = "./openhermes-2.5-mistral-7b.Q8_0.gguf"

n_gpu_layers = 1 # Change this value based on your model and your GPU VRAM pool.
n_batch = 512 # Should be between 1 and n_ctx, consider the amount of VRAM in your GPU.
n_ctx=50000
callback_manager = CallbackManager([StreamingStdOutCallbackHandler()])

llm = LlamaCpp(
    model_path=model_path,
    #temperature=0,
    n_gpu_layers=n_gpu_layers, n_batch=n_batch, 
    n_ctx=n_ctx,
    callback_manager=callback_manager, 
    verbose=True,
)

#toolkit = CustomSQLDatabaseToolkit(db=db, llm=llm)
toolkit = SQLDatabaseToolkit(db=db, llm=llm)
toolkit.get_tools()

PREFIX = '''You are a SQL expert. You have access to a Microsoft SQL Server database.
            Identify which tables can be used to answer the user's question and write and execute a SQL query accordingly.
         '''

FORMAT_INSTRUCTIONS = """RESPONSE FORMAT INSTRUCTIONS
----------------------------

When responding please, please output a response in this format:

thought: Reason about what action to take next, and whether to use a tool.
action: The tool to use. Must be one of: {tool_names}
action_input: The input to the tool

For example:

thought: I need to get all tables from database
action: sql_db_list_tables
action_input: Empty string
"""

agent_executor = create_sql_agent(
    llm=llm,
    toolkit=toolkit,
    agent_type=AgentType.ZERO_SHOT_REACT_DESCRIPTION,
    verbose=True,
    handle_parsing_errors=True,
    agent_kwargs={
                  'prefix': PREFIX, 
                  'format_instructions': FORMAT_INSTRUCTIONS,
                  }
)




now = datetime.datetime.now()
print("Starting executor : ")
print(now.strftime("%Y-%m-%d %H:%M:%S"))

agent_executor.run("Who is oldest user")`

When Entering chain I usually get error "Could not parse LLM output" as despite the instructions Action Input part is not created by LLM.

> Entering new AgentExecutor chain...
Action: sql_db_list_tables
Traceback (most recent call last):
  File "/Users/dino/Codings/python/LLM_test1/.venv/lib/python3.9/site-packages/langchain/agents/agent.py", line 1032, in _take_next_step
    output = self.agent.plan(
  File "/Users/dino/Codings/python/LLM_test1/.venv/lib/python3.9/site-packages/langchain/agents/agent.py", line 636, in plan
    return self.output_parser.parse(full_output)
  File "/Users/dino/Codings/python/LLM_test1/.venv/lib/python3.9/site-packages/langchain/agents/mrkl/output_parser.py", line 70, in parse
    raise OutputParserException(
langchain.schema.output_parser.OutputParserException: Could not parse LLM output: `Action: sql_db_list_tables`

Any Idea how to fix this ?
getting an error with DallEAPIWrapper
Labels: name: auto:bug, name: area: models

### System Info

Hello! I got this error while trying to run code from [docs](https://python.langchain.com/docs/integrations/tools/dalle_image_generator).

I have Python 3.11.3, openai 1.3.5 and langchain 0.0.340 .

```
You tried to access openai.Image, but this is no longer supported in openai>=1.0.0 - see the README at https://github.com/openai/openai-python for the API.

You can run `openai migrate` to automatically upgrade your codebase to use the 1.0.0 interface. 

Alternatively, you can pin your installation to the old version, e.g. `pip install openai==0.28`

A detailed migration guide is available here: https://github.com/openai/openai-python/discussions/742
```

### Who can help?

_No response_

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Write code from [docs](https://python.langchain.com/docs/integrations/tools/dalle_image_generator) and run it using python3.

```python
from langchain.chains import LLMChain
from langchain.llms import OpenAI
from langchain.prompts import PromptTemplate
from langchain.utilities.dalle_image_generator import DallEAPIWrapper

llm = OpenAI(temperature=0.9)
prompt = PromptTemplate(
    input_variables=["image_desc"],
    template="Generate a detailed prompt to generate an image based on the following description: {image_desc}",
)
chain = LLMChain(llm=llm, prompt=prompt)
image_url = DallEAPIWrapper().run(chain.run("halloween night at a haunted museum"))

print(image_url)
```

### Expected behavior

get an image url
Is it possible to switch language models if the ms in the first model is large?
Labels: name: auto:question, name: auto:enhancement, name: area: models

### Issue you'd like to raise.

My ChatOpenAI usually takes a response time of 1000 ms. I want the model to switch to either GooglePalm or some other language model when the response time of ChatOpenAI is large. Is it possible?

### Suggestion:

_No response_
URL Lib error doesn't get resolved.
Labels: name: auto:bug, name: area: models

### Issue you'd like to raise.

I have the following piece of code:
openai_llm = ChatOpenAI(model_name='gpt-3.5-turbo-1106', streaming=True, callbacks=[StreamingStdOutCallbackHandler()],
                        temperature=0.5, max_retries=0)
                        
I still keep getting this error:
urllib3.util.retry:Converted retries value: 2 -> Retry(total=2, connect=None, read=None, redirect=None, status=None)

I tried everything. nothing is working at this point. I dont want the Retries to happen. I have fallback models but they aren't being utilized. 

### Suggestion:

_No response_
Update openai/create_llm_result function to consider kwargs
Labels: name: lgtm, name: auto:improvement, name: area: models, name: size:XS

Replace this entire comment with:
  - **Description:** updates `create_llm_result` function within `openai.py`  to consider latest `params`,
  - **Issue:** #8928
  - **Dependencies:** -,
  - **Tag maintainer:** -
  - **Twitter handle:** [burkomr](https://twitter.com/burkomr)

<!-- If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17. -->
How to mount the custom Retry class with 0 retries to improve Fallbacks?
Labels: name: area: doc loader, name: auto:question

### Issue you'd like to raise.

I've written this code to make sure that the max_retries count is 0. Even though I set that value in my LLM instance, It doesn't work. So I've decided to create a Custom WebBaseLoader. But I'm stuck on how to mount it and make sure that it works.

would appreciate any help! 

`class CustomWebBaseLoader(WebBaseLoader):
    def __init__(
            self,
            # ...
    ) -> None:
        # ...

        if session:
            self.session = session
        else:
            session = requests.Session()
            # ...

            # Set the retry configuration for the session
            retries = Retry(total=0, backoff_factor=1, status_forcelist=[500, 502, 503, 504])
            session.mount('http://', HTTPAdapter(max_retries=retries))
            session.mount('https://', HTTPAdapter(max_retries=retries))

            # ...
            self.session = session

    # ...
`

### Suggestion:

_No response_
Adds progress bar to GooglePalmEmbeddings
Labels: name: lgtm, name: area: embeddings, name: auto:improvement, name: size:S

  - **Description:** Adds a tqdm progress bar to GooglePalmEmbeddings when embedding a list.
  - **Issue:** #13637
  - **Dependencies:** TQDM as a main dependency (instead of extra)


Signed-off-by: ugm2 <unaigaraymaestre@gmail.com>
Urllib3 retry error fix with fallbacks.
Labels: name: auto:bug, name: area: models

### Issue you'd like to raise.

Hi, I was recently trying to implement Fallbacks into my system but I've realized that even though there are other models that can be hit if there is a failure, I've noticed that this isn't happening. I tried to Debug it and this is the error I see. I keep getting this error even though the max retries value is 0
openai_llm = ChatOpenAI(model_name='gpt-3.5-turbo-1106', streaming=True, callbacks=[StreamingStdOutCallbackHandler()],
temperature=0.5, max_retries=0)

Error:
DEBUG:urllib3.util.retry:Converted retries value: 2 -> Retry(total=2, connect=None, read=None, redirect=None, status=None)

The model takes forever to respond. is there a fix for this?

### Suggestion:

_No response_
Urllib Retry error help.
Labels: name: auto:bug, name: area: models

### Issue you'd like to raise.

Hi, I was recently trying to implement Fallbacks into my system but I've realized that even though there are other models that can be hit if there is a failure, I've noticed that this isn't happening. I tried to Debug it and this is the error I see. I keep getting this error even though the max retries value is 0
openai_llm = ChatOpenAI(model_name='gpt-3.5-turbo-1106', streaming=True, callbacks=[StreamingStdOutCallbackHandler()],
                        temperature=0.5, max_retries=0)

Error:
DEBUG:urllib3.util.retry:Converted retries value: 2 -> Retry(total=2, connect=None, read=None, redirect=None, status=None)

The model takes forever to respond. is there a fix for this?


### Suggestion:

_No response_
Issue: how to compute perplexity score during inference?
Labels: name: auto:question, name: area: models

### Issue you'd like to raise.

Hi, sorry if I missed it, but I couldn't find in the documentation, here in issues nor through google an answer to this question: how can one compute perplexity score (for each generated token and/or a mean perplexity score for the whole output) during inference?
In our use case, we use LLMs only for inference, but we have to be able to give some kind of confidence score along with the models' answers. We use various integration backends in our stack, HF transformers, vLLM, llama.cpp to name a few.
Any help would be greatly appreciated. Thanks!

### Suggestion:

_No response_
fix(docs): link to question answering
Labels: name: lgtm, name: auto:documentation, name: size:XS

first link in [overview](https://python.langchain.com/docs/use_cases/question_answering/code_understanding#overview)

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->