delayedkarma/langchain-issues huggingface dataset:
Insert split text to milvus error.
Labels: name: area: embeddings, name: area: vector store, name: auto:bug

### System Info

langchain:  0.0.250
python:  3.11.4
windows:  10

### Who can help?

@agola11 @hwchase17 

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [X] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

I want to add extend fields in milvus collection, Here are some of my code:
......
extendField = [{"ck": "aa_bb_cc"}]  # the extend field.
  split_text = text_splitter.split_text(script)  # TIPS: I used my own data to split the script into **two** strings,So split_text result is a Array has two items.
  milvusDb = Milvus.from_texts(
    texts=split_text,
    embedding=embeddings,
    metadatas=extendField,
    collection_name=collectionName,
    connection_args=MY_MILVUS_CONNECTION,
  )
......
Excute the code above, I got this exception:

 File "C:\Users\XXXX\scoop\apps\python\current\Lib\site-packages\pymilvus\client\prepare.py", line 508, in _parse_batch_request
    raise ParamError(
pymilvus.exceptions.ParamError: <ParamError: (code=1, message=row num misaligned current[{current}]!= previous[{row_num}])>
......
Finally, i find this code in  langchain/vectorstores/miluvs.py
`
 # Collect the metadata into the insert dict.
        if metadatas is not None:
            for d in metadatas:
                for key, value in d.items():
                    if key in self.fields:
                        insert_dict.setdefault(key, []).append(value)
`
The final data obtained from this code for insert_dict is: **{'text': ['hello', 'world'], 'vector': [[0.1, 0.2], [0.3, 0.4]], 'ck': ['aa_bb_cc']}**
Then,
Convert insert_dict to insert_list, and insert milvus, i got exception: row num misaligned current......

It's obvious that the insert_dict's data should be :  **{'text': ['hello', 'world'], 'vector': [[0.1, 0.2], [0.3, 0.4]], 'ck': ['aa_bb_cc','aa_bb_cc']}**

I modifed the code to:
`
 # Collect the metadata into the insert dict.
        if metadatas is not None:
            for d in metadatas:
                for key, value in d.items():
                    if key in self.fields:
                         insert_dict.setdefault(key, [])
                         for t in texts:
                             insert_dict[key].append(value)
`
 It works!

### Expected behavior

 expect insert_dict data: **{'text': ['hello', 'world'], 'vector': [[0.1, 0.2], [0.3, 0.4]], 'ck': ['aa_bb_cc','aa_bb_cc']}**
Bug fix: feature/issue-7804-chroma-client_settings-bug follow-up on PR #8267 is_persistent also needs to be set to True
Labels: name: area: vector store, name: auto:bug

<!-- Thank you for contributing to LangChain!

  - Description: is_persistent also needs to be set to True when persist_directory exists
  - Issue: PR #8267 fixed issue-7804 but introduced another issue where is_persistent is not set to True (defaulted to False). This causes ChromaDB not persisting data to disk.
  - Dependencies: N/A
  - Tag maintainer: N/A
  - Twitter handle: peidaqi
  
Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the same people again.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->
Added new use case docs for Web Scraping, Chromium loader, BS4 transformer
Labels: name: auto:documentation

  - Description: Added a new use case category called "Web Scraping", and a tutorial to scrape websites using OpenAI Functions Extraction chain to the docs.
  - Tag maintainer:@baskaryan @hwchase17 ,
  - Twitter handle: https://www.linkedin.com/in/haiphunghiem/ (I'm on LinkedIn mostly)
ssue: How to add a intermediate layer between langchain and azure openai<Please write a comprehensive title after the 'Issue: ' prefix>
Labels: name: auto:question, name: auto:improvement

### Issue you'd like to raise.

Hello, 

We are looking to add an intermediate layer between langchain and azure openai calls so that all requests pass through our custom api instead of azure openai.
The request body and url parameters are slightly different 

Is there any recommended way of accomplishing this with minimal changes and maintenance.also will this allow all functions to work?

Thanks


### Suggestion:

_No response_
Streaming issue regarding pusher 
Labels: name: auto:question, name: auto:improvement

### Issue you'd like to raise.

class MyCallbackHandler(BaseCallbackHandler):
    def __init__(self,new_payload : Dict):
        self.user_id = new_payload["user_id"]
        self.session_id = new_payload["session_id"]
        self.message_id = new_payload["message_id"]
        self.status = "processing"
        self.session_id_channel = f"{self.session_id}_channel"
        self.count = 0
        self.temp_count = 0
        self.response = ""
        self.flag = True
        self.threshold_no = int(os.getenv("CHUNK_SIZE"))
        self.text = ""

    def on_llm_new_token(self, token, **kwargs) -> None:

        if token != "":
            # logging.info(f"LLM token: {token}")
            pusher_client.trigger(self.session_id_channel ,'query_answer_stream', {
                        "user_id": self.user_id,
                        "session_id": self.session_id,
                        "message_id": self.message_id,
                        "answer": self.response,
                        "status": self.status
                        },
                        )


when i am using this custom callbacks MyCallbackHandler my streaming is getting slow. Any suggestion to make it fast ?

### Suggestion:

_No response_
on_llm_start callback doesn't change the prompts sent to LLM
Labels: name: area: agent, name: auto:bug

### System Info

Python 3.10.12
langchain                                    0.0.233

### Who can help?

@agola11

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [X] Callbacks/Tracing
- [ ] Async

### Reproduction

```
from typing import Dict, Union, Any, List

from langchain.callbacks.base import BaseCallbackHandler
from langchain.schema import AgentAction
from langchain.agents import AgentType, initialize_agent, load_tools
from langchain.callbacks import tracing_enabled
from langchain.llms import OpenAI
from langchain.schema.agent import AgentAction
from langchain.schema.output import LLMResult
from uuid import UUID

class MyCustomHandlerOne(BaseCallbackHandler):
    def on_llm_start(
        self, serialized: Dict[str, Any], prompts: List[str], **kwargs: Any
    ) -> Any:
        print(f"on_llm_start {prompts}")
        prompts[0] = "Tell me a joke?"
        print(f"on_llm_start {prompts}")

import langchain
langchain.verbose = True
langchain.debug = True

handler1 = MyCustomHandlerOne()
llm = OpenAI(temperature=0, streaming=True, callbacks=[handler1], verbose=True)
print(llm.__call__("""Context : Mr. Carl Smith is a 31-year-old man who has been experiencing homelessness 
                   on and off for all his adult life.
                   Questioin: How old is Mr. Carl Smith?"""))
```

The output is
```
on_llm_start ['Context : Mr. Carl Smith is a 31-year-old man who has been experiencing homelessness \n                   on and off for all his adult life.\n                   Questioin: How old is Mr. Carl Smith?']
on_llm_start ['Tell me a joke?']
[llm/start] [1:llm:OpenAI] Entering LLM run with input:
{
  "prompts": [
    "Tell me a joke?"
  ]
}
[llm/end] [1:llm:OpenAI] [699.067ms] Exiting LLM run with output:
{
  "generations": [
    [
      {
        "text": "\n\nAnswer: Mr. Carl Smith is 31 years old.",
        "generation_info": {
          "finish_reason": "stop",
          "logprobs": null
        }
      }
    ]
  ],
  "llm_output": {
    "token_usage": {},
    "model_name": "text-davinci-003"
  },
  "run": null
}


Answer: Mr. Carl Smith is 31 years old.
```

### Expected behavior

LLM will respond to the new prompt set by the callback
ERROR: langchain 0.0.251 has requirement pydantic<2,>=1, but you'll have pydantic 2.1.1 which is incompatible.
Labels: name: auto:bug

### System Info

Ubuntu

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

pip install with the following requierments.txt fail:


fastapi[all]
uvicorn[standard]
torch
transformers
accelerate
einops
xformers

docquery
# Start - xlm-roberta-large-squad2
sentencepiece
protobuf==3.20.*
# End   - xlm-roberta-large-squad2

panel
hvplot

langchain==0.0.251

### Expected behavior

langchain should work with latest pydantic
ChromaDB integration should properly handle the new Ephemeral/Persistent client objects in recent Chroma version.
Labels: name: area: vector store, name: auto:improvement

With the new Chroma version introducing EphemeralClient and PersistentClient objects, I believe the LangChain integration should be updated to properly handle this.

Below is the relevant code, I believe, where this should be done in the Chroma constructor (chroma.py)

```python
            elif persist_directory:
                # Maintain backwards compatibility with chromadb < 0.4.0
                major, minor, _ = chromadb.__version__.split(".")
                if int(major) == 0 and int(minor) < 4:
                    _client_settings = chromadb.config.Settings(
                        chroma_db_impl="duckdb+parquet",
                    )
                else:
                    _client_settings = chromadb.config.Settings(is_persistent=True)
                _client_settings.persist_directory = persist_directory
            else:
                _client_settings = chromadb.config.Settings()
            self._client_settings = _client_settings
            self._client = chromadb.Client(_client_settings)
            self._persist_directory = (
                _client_settings.persist_directory or persist_directory
            )
```

My thought, is set self._client to EphemeralClient or PersistentClient depending on if persist_directory is used instead of the old chromadb.Client way.

Is there any work being done on this? Noticing the comment about chromadb 0.4.0, it seems like its been noted, but not corrected.

Currently I can't figure out how to make this work in the current Chroma integration without accessing the PersistentClient directly, then passing it in to Chroma constructor. Here is how I am doing it currently:

```python
client = chromadb.PersistentClient(path=persist_directory, settings=settings)
db = Chroma(
    client=client,
    embedding_function=embedding_function,
    client_settings=settings,
    persist_directory=persist_directory
)
```
Getting NameError: name 'partition_pdf' is not defined when running "documents = loader.load()"
Labels: name: area: doc loader, name: auto:bug

### System Info

I have been running this code for weeks, and today it looks like something changed to break it. I'm using the following code snippets...

from langchain.document_loaders import DirectoryLoader
...
loader = DirectoryLoader(directory_path, glob='**/*.pdf')
documents = loader.load()

This is the error I am getting...

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
[<ipython-input-9-8159690dd515>](https://localhost:8080/#) in <cell line: 2>()
      1 loader = DirectoryLoader(directory_path, glob='**/*.pdf')
----> 2 documents = loader.load()
      3 print("Number of documents: ", len(documents))
      4 
      5 timestampit()

5 frames
[/usr/local/lib/python3.10/dist-packages/unstructured/partition/auto.py](https://localhost:8080/#) in partition(filename, content_type, file, file_filename, url, include_page_breaks, strategy, encoding, paragraph_grouper, headers, skip_infer_table_types, ssl_verify, ocr_languages, pdf_infer_table_structure, xml_keep_tags, data_source_metadata, **kwargs)
    219         )
    220     elif filetype == FileType.PDF:
--> 221         elements = partition_pdf(
    222             filename=filename,  # type: ignore
    223             file=file,  # type: ignore

NameError: name 'partition_pdf' is not defined



### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [X] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

loader = DirectoryLoader(directory_path, glob='**/*.pdf')
documents = loader.load()

### Expected behavior

Loaded documents.
Cryptic error message trying to load NGramOverlapSelector
Labels: name: area: doc loader, name: auto:bug

### System Info

langchain-0.0.251 pydantic-1.10.12
python 3.9
AWS Linux 6.1.38-59.109.amzn2023.x86_64


### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [X] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

from langchain.prompts.example_selector.ngram_overlap import NGramOverlapExampleSelector
from langchain.prompts import PromptTemplate


examples = [
    dict(text="Call me Ishmael."),
    dict(text="In a hole in the ground there lived a hobbit.")
]

example_prompt = PromptTemplate(input_variables=["text"], template="EXAMPLE: {text}")

selector = NGramOverlapExampleSelector(examples=examples, example_prompt=example_prompt)
t = selector.select_examples(dict(text="hole in ground"))
print(t)

### Expected behavior

Should print the selected examples. Instead I get:

```
Traceback (most recent call last):
  File "/work/2023/08/partial/src/ngram_demo.py", line 12, in <module>
    selector = NGramOverlapExampleSelector(examples=examples, example_prompt=example_prompt)
  File "pydantic/main.py", line 341, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 1 validation error for NGramOverlapExampleSelector
__root__
  Not all the correct dependencies for this ExampleSelect exist (type=value_error)
```
Issue: Issue providing LLMChain with multiple variables when memory is used
Labels: name: area: memory, name: auto:bug, name: auto:question

### Issue you'd like to raise.

I'm having an issue with providing the LLMChain class with multiple variables when I provide it with a memory object. It works fine when I don't have memory attached to it. I followed the example given in this document: [LLM Chain Multiple Inputs](https://python.langchain.com/docs/modules/chains/foundational/llm_chain#additional-ways-of-running-llm-chain)

Here is the code that I used, which is mostly based on the example from the above documentation, besides I've added memory.

```
# Multiple inputs example

template = """Tell me a {adjective} joke about {subject}."""
prompt = PromptTemplate(template=template, input_variables=["adjective", "subject"])
llm = OpenAI(temperature=0)
memory = ConversationKGMemory(llm=llm)
llm_chain = LLMChain(prompt=prompt, llm=OpenAI(temperature=0))

llm_chain.predict(adjective="sad", subject="ducks")
```

With the above code, I get the following error:
```
ValueError: One input key expected got ['adjective', 'subject']
```

Python version: 3.11 
LangChain version: 0.0.250

### Suggestion:

Is there support for using multiple input variables when memory is involved?
(issue #5163) Append reminder to nest multi-prompt router prompt output in JSON markdown code block, resolving JSON parsing error.
Labels: name: auto:bug

Resolves occasional JSON parsing error when some predictions are passed through a `MultiPromptChain`.

Makes [this modification](https://github.com/langchain-ai/langchain/issues/5163#issuecomment-1652220401) to `multi_prompt_prompt.py`, which is much cleaner than appending an entire example object, which is another community-reported solution.

@hwchase17, @baskaryan

cc: @SimasJan
Allow GPT4AllEmbeddings() constructor to be configured
Labels: name: area: embeddings, name: auto:improvement

### Feature request

I would be great to be able to pass in configuration info to the `Embeddings` from the `GTP4AllEmbeddings()` constructor in the same manner as the `GPT4All()` constructor.  For example. To be able to turn of allow_download, model, etc..

`embeddings = GPT4AllEmbeddings(allow_download=False)`

### Motivation

It would allow for the same level of customization as the `GPT4All()` constructor.  Right now there is no way to run FAISS in completely offline mode.  Trying to do so will hang and error out with the models lookup call to https://gpt4all.io/models/models.json.  Being able to pass in configuration information would fix issues such as these.  As it is right now, I have to download the gpt4all repo and manually patch the `Embeddings` to pass in the configuration I need.

### Your contribution

Probably not at this time.  I would love to, but I'm a bit too busy with work at the moment.
Deterministic Fake Embedding Model
Labels: name: lgtm, name: area: embeddings, name: auto:improvement

<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the same people again.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->

Solves #8644 
This embedding models output identical random embedding vectors, given the input texts are identical.
Useful when used in unittest.
@baskaryan
Add tensor_split support to LlamaCpp
Labels: name: area: memory, name: auto:improvement

Adds the option `tensor_split` to the `LlamaCpp` binding, for specifying how to split the model memory onto multiple GPUs.

Example running airoboros 33b on 4x Nvidia RTX 2080Ti, (11 GB each)

```python
os.environ["CUDA_VISIBLE_DEVICES"]="0,1,2,3"
LlamaCpp(
            model_path="airoboros-33b-gpt4-2.0.ggmlv3.q8_0.bin",
            n_gpu_layers=63,
            tensor_split=[0.1, 0.3, 0.3, 0.3],
            ...
```

This leaves space on GPU0 for the tokenizer etc.

More info about the option in the [llama-cpp-python documentation](https://llama-cpp-python.readthedocs.io/en/latest/api-reference/#llama_cpp.llama.Llama.__init__).
wip -- high temp
Labels: name: auto:enhancement

<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the same people again.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->
Support evaluating runnables and arbitrary functions
Labels: name: auto:improvement

Added a couple of "integration tests" for these that I ran.

Main design point of feedback: at this point, would it just be better to have separate arguments for each type? Little confusing what is or isn't supported and what is the intended usage at this point since I try to wrap the function as runnable or pack or unpack chains/llms.

```
run_on_dataset(
...
llm_or_chain_factory = None,
llm = None,
chain = NOne,
runnable=None,
function=None
):
# raise error if none set
```

Downside with runnables and arbitrary function support is that you get much less helpful validation and error messages, but I don't think we should block you from this, at least.