delayedkarma/langchain-issues huggingface dataset:
getiing an error with openai v1
Labels: name: auto:bug, name: area: models

### System Info

Getting the below error when try to use load_qa_chain

openai.lib._old_api.APIRemovedInV1: 

You tried to access openai.ChatCompletion, but this is no longer supported in openai>=1.0.0 - see the README at https://github.com/openai/openai-python for the API.

You can run `openai migrate` to automatically upgrade your codebase to use the 1.0.0 interface. 

Alternatively, you can pin your installation to the old version, e.g. `pip install openai==0.28`

A detailed migration guide is available here: https://github.com/openai/openai-python/discussions/742

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

run load_qa_chain

### Expected behavior

it should not error
Issue: Can't get response from chain.predict()
Labels: name: area: memory, name: auto:bug, name: area: models

### Issue you'd like to raise.

I'm using llama2 with langchain. To use gpu, I re-installed the llama-cpp-python with cublas options, but after that, I can not get response from langchain.LLMChain.predict() anymore.
The result is like:
```
User:hi


> Entering new LLMChain chain...
Prompt after formatting:

History: 
Question: hi
Answer: 


llama_print_timings:        load time =     883.81 ms
llama_print_timings:      sample time =       0.47 ms /     2 runs   (    0.24 ms per token,  4255.32 tokens per second)
llama_print_timings: prompt eval time =     883.77 ms /    14 tokens (   63.13 ms per token,    15.84 tokens per second)
llama_print_timings:        eval time =     228.71 ms /     1 runs   (  228.71 ms per token,     4.37 tokens per second)
llama_print_timings:       total time =    1117.20 ms

> Finished chain.
```
The following is how I define the llm and the chain:
```
llm = LlamaCpp(
        model_path = model_path,
        n_ctx=4096, 
        top_k=10, 
        top_p=0.9, 
        temperature=0.7, repeat_penalty=1.1, 
        verbose=True,
        callback_manager=callback_manager,
        n_gpu_layers=35,
        n_batch=100,
        stop=["Question:","Human:"]
    )

memory = ConversationBufferMemory(memory_key='chat_history')
llm_chain = LLMChain(
    prompt=prompt, 
    llm=llm,
    memory=memory,
   )
```
Actually when I used local llama.cpp and normally installed llama-cpp-python, everything went on well. The only change I made is the llama-cpp-python and all the code just remain the same. I wonder why will this happen.

### Suggestion:

_No response_
Issue: Get cost estimates of GPT-4-Vision

### Issue you'd like to raise.

Hi @baskaryan! Thanks for open-sourcing the [notebook](https://github.com/langchain-ai/langchain/blob/master/cookbook/openai_v1_cookbook.ipynb)  related to GPT-4-Vision. 

Is there a way, we can estimate the cost for the API calls similar to using `get_openai_callback`? 

### Suggestion:

We could get the cost using callbacks as:
```
from langchain.callbacks import get_openai_callback
from langchain.prompts import (
    ChatPromptTemplate,
    SystemMessagePromptTemplate,
    HumanMessagePromptTemplate,
)

human_message_prompt = HumanMessagePromptTemplate.from_template(prompt)
chat_prompt = ChatPromptTemplate.from_messages(
     [system_message_prompt, human_message_prompt]
)


with get_openai_callback() as cb:
    openai_chain = LLMChain(prompt=chat_prompt, llm=model)
    response = openai_chain.run({})
    n_tokens = cb.total_tokens
    total_cost = cb.total_cost
```

How do I use `HumanMessage` with the callback?
LCEL nits
Labels: name: auto:documentation

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
HTMLHeaderTextSplitter won't run (maxHead)
Labels: name: area: doc loader, name: auto:bug

### System Info

Notebook with latest langchain

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [X] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Trying in notebook the HTMLHeaderTextSplitter
```jupyter-notebook
from langchain.text_splitter import RecursiveCharacterTextSplitter

headers_to_split_on = [
    ("h1", "Header 1"),
    ("h2", "Header 2"),
    ("h3", "Header 3"),
    ("h4", "Header 4"),
]

html_splitter = HTMLHeaderTextSplitter(headers_to_split_on=headers_to_split_on)

html_header_splits = html_splitter.split_text_from_file("/content/X.html")

chunk_size = 500
chunk_overlap = 30
text_splitter = RecursiveCharacterTextSplitter(
    chunk_size=chunk_size, chunk_overlap=chunk_overlap
)

Split
splits = text_splitter.split_documents(html_header_splits)
splits[80:85]`

``` 
```jupyter-notebook


---------------------------------------------------------------------------
XSLTApplyError                            Traceback (most recent call last)
[<ipython-input-54-bd3edea942d1>](https://localhost:8080/#) in <cell line: 12>()
     10 html_splitter = HTMLHeaderTextSplitter(headers_to_split_on=headers_to_split_on)
     11 
---> 12 html_header_splits = html_splitter.split_text_from_file("/content/X.html")
     13 
     14 chunk_size = 500

[/usr/local/lib/python3.10/dist-packages/langchain/text_splitter.py](https://localhost:8080/#) in split_text_from_file(self, file)
    586         xslt_tree = etree.parse(xslt_path)
    587         transform = etree.XSLT(xslt_tree)
--> 588         result = transform(tree)
    589         result_dom = etree.fromstring(str(result))
    590 

src/lxml/xslt.pxi in lxml.etree.XSLT.__call__()

XSLTApplyError: maxHead``` 
Is the HTML just to large to be handled by the textsplitter?

### Expected behavior

Load the html..
Added ability to pass arguments to the Playwright browser
Labels: name: lgtm, name: auto:improvement, name: size:S

  - **Description:** Enhanced `create_sync_playwright_browser` and `create_async_playwright_browser` functions to accept a list of arguments. These arguments are now forwarded to `browser.chromium.launch()` for customizable browser instantiation.
  - **Issue:** #13143
  - **Dependencies:** None
  - **Tag maintainer:** @eyurtsev,
  - **Twitter handle:** Dr_Bearden
How to retrieve custom number of docs from parent retriver document , I have 100 docs and i don't want the reteiver to just retrieve only 4 docs
Labels: name: area: doc loader, name: auto:question

### Issue you'd like to raise.

I have 100 docs , from which i am trying to retrieve top 10 docs which are most relevant to my query using parent document retriever. How can this be achieved.

### Suggestion:

_No response_
Add ability to pass arguments when creating Playwright browser
Labels: name: auto:improvement, name: integration: aws

### Feature request

Need the ability to pass arguments to `browser.chromium.launch()` in the `create_sync_playwright_browser` and `create_async_playwright_browser` functions located in [`langchain.tools.playwright.utils.py`](https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/tools/playwright/utils.py).

### Motivation

I created an agent that uses `PlayWrightBrowserToolkit`. All local testing worked, but I encounter errors when deploying to an AWS Lambda. I encountered two errors when testing within the AWS Lambda UI: `Target page, context or browser has been closed` and `Timeout 30000ms exceeded`. I did not experience these issues locally. I identified the issue occurred when the agent used the `navigate_browser` tool. I needed to launch the browser with `browser.chromium.launch(headless=True, args=["--disable-gpu", "--single-process"])`

### Your contribution

I have modified `utils.py` for Playwright and will submit a PR.
OpenAIEmbeddings: retry_min_seconds/retry_max_seconds parameters
Labels: name: lgtm, name: area: embeddings, name: auto:improvement, name: size:M

  - **Description:** new parameters in OpenAIEmbeddings() constructor (retry_min_seconds and retry_max_seconds) that allow parametrization by the user of the former min_seconds and max_seconds that were hidden in _create_retry_decorator() and _async_retry_decorator()
  - **Issue:** #9298, #12986
  - **Dependencies:** none
  - **Tag maintainer:** @hwchase17
  - **Twitter handle:** @adumont

make format ✅
make lint ✅
make test ✅
DOC: Install openai with langchain[openai]
Labels: name: area: doc loader, name: auto:documentation

### Issue with current documentation:

The documentation recommends to install openai with `pip install openai`.
If you do it currently (langchain 0.0.332) then an incompatible openai version is installed (1.1.2).
It's better to install it using `pip install langchain[openai]` as that will pick a version compatible with langchain.
So I propose to replace it in the docs.
WDYT ?

### Idea or request for content:

_No response_
Update huggingface_hub.py
Labels: name: area: embeddings, name: auto:improvement

also allows the e5 and gbe embedder families to be used

  - **Description:** at the moment allowing only embedders from the sentence-transformer repo is too reductive, I have extended to two other repos with families of embeddings performing and integrated with sentence-transformer, 
  - **Issue:** #12069 ,
  - **Tag maintainer:** @hwchase17,
Import changed of Vertex chatModel
Labels: name: auto:bug, name: area: models

https://github.com/langchain-ai/langchain/blob/c52725bdc5958d5295c2d563fa9b7fcb6ed09a3e/libs/langchain/langchain/chat_models/vertexai.py#L136C29-L136C29

seems there is an issue 
from vertexai.preview.language_models import ChatModel
changed into
from vertexai.language_models import ChatModel
Modify mysql query before execution
Labels: name: area: agent, name: auto:improvement

### Feature request

We are using langchain create_sql_agent to build a chat engine with database.

generation and execution of query will be handled by create_sql_agent.
I want to modify the query before execution.
Please find below example:
original query : select * from transaction where type = IPC
modified query : select * from (select * from transaction where account_id = 123) where type = IPC

Can you please provide me way forward to achieve this ?

### Motivation

To protect some data from client, need this functionality

### Your contribution

I'm not familiar with the internal code, but I can help if guided.
How do I use multiple GPU when using a model with hugging face?
Labels: name: auto:question, name: area: models

### Issue you'd like to raise.

Hi. 
I am currently using the model of the hugging face.
The code I use is below.

`llm = HuggingFacePipeline.from_model_id(model_id="beomi/llama-2-ko-7b", task="text-generation", model_kwargs={"max_length": 2000}, device=0)`

Currently I am only using device=0, but I would like to load the model using multiple devices (device=0,1,2,3).
But I couldn't find a way to handle multiple devices for HuggingFacePipeline.from_model_id.

If you know anything about this, I hope you can help.

### Suggestion:

_No response_
Streaming not working when using RunnableLambda
Labels: name: auto:bug

### System Info

langchain          0.0.332

### Who can help?

@hwchase17 

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

The default `Runnable#stream` doesn't stream, but rather defers to `invoke`. As `RunnableLambda` doesn't override the `stream` function, this causes the entire chain to run with `invoke` instead of `stream` when it is used in any part of the chain. 

```python
chain = RunnableLambda(lambda x: x) | ChatOpenAI()

for part in chain.stream("hello"):
   print(part)
```

### Expected behavior

What I'd expect is that I could use a `LambdaRunnable` to transform my inputs on-the-fly, then send it through to the rest of my chain and still use the stream/batch/etc functionalities that LCEL should bring.

If the early runnables don't support streaming, that doesn't mean that the final output can't be streamed. Currently when any of the runnables in a chain don't support streaming, the entire chain doesn't support streaming.
Custom implementation for query checker tool
Labels: name: area: agent, name: auto:enhancement

### Feature request

We are using langchain create_sql_agent to build a chat engine with database.

I want to write custom implementation for tool : query_sql_checker_tool
In this custom logic, we want to modify the sql query before execution.

Example:
original query : select * from transaction where type = IPC
modified query : select * from (select * from transaction where account_id = 123) where type = IPC

Can you please provide me way forward to achieve this ?
 

### Motivation

To protect some data from client, need this functionality

### Your contribution

I'm not familiar with the internal code, but I can help if guided.
Adding timeout to ChatOpenAI raises ConfigError
Labels: name: auto:bug, name: area: models

### System Info

Running langchain==0.0.332 with python 3.11 and openai==1.2.0 on Windows.

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

```python
from langchain.chat_models import ChatOpenAI

ChatOpenAI(
    model="gpt-3.5-turbo",
    request_timeout=5,
)
```

### Expected behavior

Should run without errors.

Likely due to newly introduced `httpx.Timeout` type in `request_timeout` (https://github.com/langchain-ai/langchain/pull/12948). Always importing httpx and tiktoken (i.e. not conditionally on `TYPE_CHECKING`) fixes the issue.
httpx openai support
Labels: name: auto:enhancement, name: area: models

### Feature request

please  support  httpx_client  in openai version 1.1.1
just one more parameter
if there is another solution  i would like to get it 
thanks 

### Motivation

cant work without ssl cert

### Your contribution

in AzureChatOpenAI  and AzureOpenAI class 
add  parameter openai_http_client
in method validate_environment
please add  at line 134 the next code
values["http_client"] = get_from_dict_or_env(
            values, "openai_http_client", "OPENAI_HTTP_CLIENT", default=""
        )
ModuleNotFoundError: No module named 'langchain.server'
Labels: name: auto:bug

### System Info

Ubuntu 22.04
langchain 0.0.332
python 3.10


### Who can help?

_No response_

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [X] Callbacks/Tracing
- [ ] Async

### Reproduction

pip install langchain --upgrade
langchain-server

### Expected behavior

Traceback (most recent call last):
  File "/home/ps/anaconda3/envs/langchain/bin/langchain-server", line 6, in <module>
    from langchain.server import main
ModuleNotFoundError: No module named 'langchain.server'
LCEL stream function doesn't use LLM cache
Labels: name: auto:bug, name: area: models

### System Info

LangChain version : 0.0.327
Python version : 3.10.11
Platform : Windows 11

### Who can help?

@agola11 @hwchase17

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [X] Callbacks/Tracing
- [ ] Async

### Reproduction

I was using streaming callbacks with LLMChains.
I switched now to use LangChain Expression Language (LCEL), and using the stream/astream functions.
They don't seem to save the results to the configured LLM cache and don't load the results from there.

I'm not sure if this is intentional, or a bug.
If it is intentional I'm wondering how I should use the cache.

Code to reproduce the issue:

```python
from langchain.prompts import ChatPromptTemplate
from langchain.chat_models import ChatOpenAI
from langchain.globals import set_llm_cache
from langchain.cache import InMemoryCache

set_llm_cache(InMemoryCache())

model = ChatOpenAI(cache=True)
prompt = ChatPromptTemplate.from_template(
    "tell me a joke in about 5 sentences about {topic}"
)
chain = prompt | model

for s in chain.stream({"topic": "bears"}):
    print(s.content, end="", flush=True)

print("\n\nFINISHED FIRST COMPLETION\n")

for s in chain.stream({"topic": "bears"}):
    print(s.content, end="", flush=True)
```

### Expected behavior

The second result should be the same and should be retrieved quickly from the LLM cache.
Fusion_RAG + Semi_Structured_RAG?
Labels: name: auto:enhancement, name: area: models

### Feature request

Langchain is a great work!
Is there any possible that you guys can combine [Fusion](https://github.com/run-llama/llama_index/blob/main/docs/examples/low_level/fusion_retriever.ipynb) and [Semi_Structured_RAG](https://github.com/langchain-ai/langchain/blob/master/cookbook/Semi_Structured_RAG.ipynb)?

### Motivation

There are a lot of tables and text in my data. First of all, I have tried Fusion_RAG, which is much better than the baseline, but it is limited to the text, and the table can not be processed, so I wondered if there a way could combine Semi_Structured_RAG and Fusion_RAG so that I could deal with both text and table at the same time. ^-^

### Your contribution

Please make the fusion: BM25+Vec+Table
Qianfan llm error calling "ERNIE-Bot-4"
Labels: name: auto:bug, name: area: models

### System Info

langchain version 0.0.332, affect all platforms
There is a mistake in the file:
qianfan\resources\llm\completion.py
this `endpoint="/chat/completions-pro`, is not correct ,it should be `endpoint="/chat/completions_pro`,"
like below:
``
"ERNIE-Bot-4": QfLLMInfo(
                endpoint="/chat/completions_pro",
                required_keys={"messages"},
                optional_keys={
                    "stream",
                    "temperature",
                    "top_p",
                    "penalty_score",
                    "user_id",
                    "system",
                },
            ),
``

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

llm = QianfanLLMEndpoint(streaming = True,temperature = 0.5)
# llm.model = "ERNIE-Bot-turbo"
llm.model = "ERNIE-Bot-4"
# llm.model = "ChatGLM2-6B-32K"
res = llm("hi")

this will report error

### Expected behavior

should work without error