delayedkarma/langchain-issues huggingface dataset:
langchain & chroma - Basic Example
Labels: name: area: embeddings, name: area: vector store, name: auto:bug, name: auto:question, name: integration: chroma

### Issue you'd like to raise.

_No response_

### Suggestion:

```
# import
from langchain.embeddings.sentence_transformer import SentenceTransformerEmbeddings
from langchain.text_splitter import CharacterTextSplitter
from langchain.vectorstores import Chroma
from langchain.document_loaders import TextLoader
from silly import no_ssl_verification
from langchain.embeddings.huggingface import HuggingFaceEmbeddings

with no_ssl_verification():
    # load the document and split it into chunks
    loader = TextLoader("state_of_the_union.txt")
    documents = loader.load()

    # split it into chunks
    text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)
    docs = text_splitter.split_documents(documents)

    # create the open-source embedding function
    embedding_function = SentenceTransformerEmbeddings(model_name="all-MiniLM-L6-v2")
    # hfemb = HuggingFaceEmbeddings()
    # load it into Chroma
    db = Chroma.from_documents(docs, embedding_function)

    # query it
    query = "What did the president say about Ketanji Brown Jackson"
    docs = db.similarity_search(query)

    # print results
    print(docs[0].page_content)
```

i use this code but i get the following error:
ValueError: Expected EmbeddingFunction.__call__ to have the following signature: odict_keys(['self', 'input']), got odict_keys(['self', 'args', 'kwargs'])
Please see https://docs.trychroma.com/embeddings for details of the EmbeddingFunction interface.
Please note the recent change to the EmbeddingFunction interface: https://docs.trychroma.com/migration#migration-to-0416---november-7-2023 


can you give me fixed code and explain?
Add a Wrapper vectorstore, compatible with SelfQueryRetriever
Labels: name: area: vector store, name: auto:improvement, name: size:XS

**Description:** 
The [`SelfQueryRetriever`](https://python.langchain.com/docs/modules/data_connection/retrievers/self_query/) component is able to convert filter clauses for different vectorstore implementations.
If we wish to offer advanced use of vectorstore, by modifying for example certain methods, via a wrapper type approach, we lose the ability to use `SelfQueryRetriever`.

We propose to standardize vectorstore wrapper exploitation, to allow flexible integration with `SelfQueryRetriever`. 
The idea is to add to `_get_builtin_translator()`, taking into account the wrapper. 
Thus, every vectorstore wrapper automatically benefits from the SelfQueryRetriever.

This is the prelude to another full-request, which needs a vectorstore wrapper, while being compatible 
with `SelfQueryRetriever`.

**Tag maintainer:**
@baskaryan

**Twitter handle:**
@pprados


<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Make it easier to subclass runnable binding with custom init args
Labels: name: auto:improvement

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Binding stop to a local llm does not work?
Labels: name: auto:bug, name: area: models

### System Info

newest version

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

When loading a local model it keeps hallucinating a conversation, how to make it stop at "Human:" ? In the create_llm function you see two ways I tried, giving kwars and binding, but both did not work. Is the bind specifically made for LCEL?
Is there some other way to make the stopping work?

```
from langchain.llms import CTransformers
from langchain.chains import LLMChain
from langchain.prompts import PromptTemplate
from langchain.memory import ConversationBufferMemory

def create_llm(model_path='./models/mistral-7b-instruct-v0.1.Q4_K_M.gguf', model_type="mistral"):
    llm = CTransformers(model=model_path, model_type=model_type,kwargs={'stop': ['Human:']})
    llm.bind(stop=["Human:"])
    return llm

def create_memory():
    memory = ConversationBufferMemory(memory_key="history")
    return memory

def create_memory_prompt():
    template = """You are an AI chatbot having a conversation with a human. Answer his questions.
    Previous conversation: {history}
    Human: {human_input}
    AI: """
    prompt = PromptTemplate(input_variables=["history", "human_input"], template=template)
    return prompt

def create_normal_chain(llm, prompt, memory):
    llm_chain = LLMChain(llm=llm, prompt=prompt, memory=memory)
    return llm_chain

chain = create_normal_chain(create_llm(), create_memory_prompt(), create_memory())

out = chain.invoke({"human_input" : "Hello"})
print(out)
```
### Expected behavior

Generation stops at "Human:".
QAGenerationChain output in different languages
Labels: name: auto:question, name: auto:improvement, name: area: models

### Issue you'd like to raise.

**My code:**
loader_pdf = PyMuPDFLoader("/Users/python/test_pdf.pdf")
doc_pdf = loader_pdf.load()

llm = ChatOpenAI(temperature=0)
chain = QAGenerationChain.from_llm(llm=llm)

print("pdf:\n",doc_pdf[3].page_content)
qa_pdf = chain.run(doc_pdf[3].page_content)
![image](https://github.com/langchain-ai/langchain/assets/94101986/03b32f31-58cb-43cf-a9de-656605775de4)


The PDF I am using is a Chinese data file，but the output is in English.
There are no relevant records in the document. I would like to know how to solve this problem or can I set a Prompt to solve it?

### Suggestion:

_No response_
Setting a custom http_client fails with pydantic ConfigError
Labels: name: auto:bug, name: area: models

### System Info

<pre>
platform = macOS-10.16-x86_64-i386-64bit
version = 3.10.13 (main, Sep 11 2023, 08:39:02) [Clang 14.0.6 ]
langchain.version = 0.0.332
pydantic.version = 1.10.13
openai.version = 1.2.0
</pre>

### Who can help?

@eyurtsev

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

I'm trying to initialize an instance of AzureChatOpenAI with a custom http_client. Following is the code I'm using - 

<pre>
from langchain.chat_models import AzureChatOpenAI
from langchain.schema import HumanMessage, SystemMessage
import httpx

messages = [
    SystemMessage(content="You are a helpful assistant that translates English to Spanish"),
    HumanMessage(content="Good morning Vietnam!")
]
client = httpx.Client()


chat = AzureChatOpenAI(
    openai_api_key="my-secret-key",
    openai_api_version="2023-07-01-preview",
    model="gpt-4",
    deployment_name="my-deployment-name",
    azure_endpoint="https://my-instance.openai.azure.com",
    http_client=client
)
# AzureChatOpenAI.update_forward_refs()
chat(messages)
</pre>

This fails with the following pydantic config error:

<pre>
Traceback (most recent call last):
  File "azure-custom.py", line 24, in <module>
    chat = AzureChatOpenAI(
  File "langchain-env/lib/python3.10/site-packages/langchain/load/serializable.py", line 97, in __init__
    super().__init__(**kwargs)
  File "pydantic/main.py", line 339, in pydantic.main.BaseModel.__init__
  File "pydantic/main.py", line 1076, in pydantic.main.validate_model
  File "pydantic/fields.py", line 860, in pydantic.fields.ModelField.validate
pydantic.errors.ConfigError: field "http_client" not yet prepared so type is still a ForwardRef, you might need to call AzureChatOpenAI.update_forward_refs().
</pre>

### Expected behavior

As per the [ChatOpenAI API Doc](https://api.python.langchain.com/en/latest/chat_models/langchain.chat_models.openai.ChatOpenAI.html), `http_client` seems to be a valid parameter to set in order to customize the httpx client in the background. But I'm unable to do that customization. 
Adding `AzureChatOpenAI.update_forward_refs()` also does not solve this issue - the code raises the error even before hitting that line.
Remove `_get_kwarg_value` function
Labels: name: lgtm, name: area: vector store, name: auto:refactor

`_get_kwarg_value` function is useless, one can rely on python builtin functionalities to do the exact same thing. 

  - **Description:** Removed `_get_kwarg_value`. Helps with code readability.
  - **Issue:** the issue # it fixes (if applicable),
  - **Twitter handle:** @Guillem_96
docugami cookbook
Labels: name: auto:documentation

Adds a cookbook for semi-structured RAG via Docugami. This follows the same outline as the semi-structured RAG with Unstructured cookbook: https://github.com/langchain-ai/langchain/blob/master/cookbook/Semi_Structured_RAG.ipynb

The main change is this cookbook uses Docugami instead of Unstructured to find text and tables, and shows how XML markup in the output helps with retrieval and generation.

We are \@docugami on twitter, I am \@tjaffri
Llamacppserver client
Labels: name: auto:enhancement, name: area: models, name: size:L

  - **Description:** a client for the llama.cpp native server 
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** emanuel248

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Add SQLDocStore
Labels: name: area: vector store, name: auto:enhancement, name: size:L

**Description:** 
langchain offers three technologies to save data:
- [vectorstore](https://python.langchain.com/docs/modules/data_connection/vectorstores/)
- [docstore](https://js.langchain.com/docs/api/schema/classes/Docstore)
- [record manager](https://python.langchain.com/docs/modules/data_connection/indexing)

If you want to combine these technologies in a single transaction, to guarantee the consistency of all backups, 
you need a common implementation for each. This allows, for example, to combine the RecordManager with a vector 
store using an SQL database.

But, there is no SQL docstore.

**Tag maintainer:**
@baskaryan

**Twitter handle:**
@pprados
DOC: Setting proxy for the whole langchain library, especially web_research_retriever
Labels: name: area: doc loader, name: auto:improvement, name: auto:documentation

### Issue with current documentation:

Hello guys,
I have to use a proxy to access Azure OpenAi because I'm using a VPN for my company.
However when I try to use the [](https://python.langchain.com/docs/modules/data_connection/retrievers/web_research) , I block at fetching the pages because the code inside the langchain library does not use the proxy.

So I managed to make it work for GoogleSearchAPIWrapper by setting it in the os.environ but it does not work for the inside request and html_load call in the langchain library.

![bugwebresearch](https://github.com/langchain-ai/langchain/assets/10726395/0db2de66-9cab-4710-a58b-3e4af721848b)


### Idea or request for content:

So I would like to have a proxy parameter when I set up a web_research_retriever = WebResearchRetriever.from_llm(
    vectorstore=vectorstore, llm=llm, search=search
)
Or any other tips that can help me to dodge this proxy issue.

Thanks in advance
Document retriever from Knowledge-graph based source
Labels: name: area: embeddings, name: area: vector store, name: auto:enhancement

### Feature request

As the title suggests, there is currently no built-in method to retrieve chunks linked through edges in graph-based structures.

This is especially relevant in cases where documents are not self-contained and chunks reference other chunks, either within the same document or in other documents. These "children" chunks are often necessary to properly answer user queries.

Specifically, I would like to have the ability to perform the following within the same chain:

```
1. Get the user query
2. Embed it
3. Perform approximate nearest neighbor (ANN) lookup and retrieve the most relevant chunk
4. Given the ID of the chunk, expand the set of matches by navigating the graph
5. Retrieve all of these to provide to the Large Language Model (LLM)
6. Send everything to the LLM to answer the user query
```

To further contextualize my suggestion, I have been considering the following approach to begin with:

```python
import networkx as nx

GraphType = Union[nx.DiGraph, nx.Graph]


class NXInMemoryGraphRetriever(BaseRetriever):

    def __init__(
        self,
        bootstrap_retriever: BaseRetriever,
        graph: GraphType,
        navigate_graph_function: Callable[[GraphType, str], List[Document]],
    ):
        self._graph = graph
        self._navigate_graph_fn = navigate_graph_function
        ...

    def get_relevant_documents(
        self,
        query: str,
        *,
        callbacks: Optional[List[Callable]] = None,
        tags: Optional[List[str]] = None,
        metadata: Optional[Dict[str, Any]] = None,
        run_name: Optional[str] = None,
        **kwargs: Any,
    ) -> List[Document]:
        base_documents = self.bootstrap_retriever.get_relevant_documents(...)
        expanded_set_of_docs = set(base_documents)
        for document in base_documents:
            expanded_set_of_docs.update(
                self.navigate_graph_function(self.graph, document.metadata["id"])
            )
        ...
        return list(expanded_set_of_documents)

# Example of callable function to pass
def get_everything_one_hop_away(graph: GraphType, source_doc_id: str) -> List[Documents]:
    related_docs = []
    for source_doc, related_doc in nx.dfs_edges(
        graph, source=source_doc_id, depth_limit=1
    ):
        related_docs.add(Document(...)) # From the related docs id.
    return related_docs
```

Additionally, expanding this to support Neo4J or similar should be rather straightforward. Don't pass the graph but the DB driver and the `navigate_graph_function` with a cypher query string/literal.

### Motivation

Use langchain to navigate graph related datasets easily.

### Your contribution

Happy to send a PR your way :)
Support OpenAI seed for deterministic outputs
Labels: name: auto:improvement, name: area: models

### Feature request

Support OpenAI seed and fingerprint parameters to get more consistent outputs for the same inputs and model version.

https://cookbook.openai.com/examples/deterministic_outputs_with_the_seed_parameter#seed
https://platform.openai.com/docs/api-reference/chat/create#chat-create-seed

### Motivation

Running unit tests that are dependent on LLM responses often leads to random flakiness.

### Your contribution

PR
Simple Retrieval QA Example
Labels: name: area: embeddings, name: area: doc loader, name: area: vector store, name: auto:question, name: integration: chroma

### Issue you'd like to raise.

`
# Import necessary libraries
from llama_index import (
    LangchainEmbedding,
)
from langchain.embeddings.huggingface import HuggingFaceEmbeddings
from llama_index import VectorStoreIndex, SimpleDirectoryReader, ServiceContext
from llama_index.vector_stores import ChromaVectorStore
from llama_index.storage.storage_context import StorageContext
import chromadb


  # Create client and a new collection
  chroma_client = chromadb.EphemeralClient()
  chroma_collection = chroma_client.create_collection("quickstart")

  hfemb = HuggingFaceEmbeddings()
  embed_model = LangchainEmbedding(hfemb)

  documents = SimpleDirectoryReader("./docs/examples/data/paul_graham").load_data()


  # Set up ChromaVectorStore and load in data
  vector_store = ChromaVectorStore(chroma_collection=chroma_collection)
  storage_context = StorageContext.from_defaults(vector_store=vector_store)
  service_context = ServiceContext.from_defaults(llm=None, embed_model=embed_model)
  index = VectorStoreIndex.from_documents(
      documents, storage_context=storage_context, service_context=service_context, show_progress=True
  )

  # Query Data
  query_engine = index.as_query_engine()
  response = query_engine.query("What did the author do growing up?")

  print(response)
`

I used this code with llama-index retrieval mechanism for question&answer with documents. How can i write this code's LangChain form? I should'nt use model. I retrieve answer from documents. And i should use HuggingFaceEmbeddings for embedding.

### Suggestion:

_No response_
OpenAI latest Update fixes
Labels: name: auto:improvement, name: area: models

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Problem With OpenAI Error update 
Labels: name: auto:bug, name: area: models

### System Info

langchain==0.0.333
openai == 1.2.2


### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [x] My own modified scripts

### Related Components

- [x] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Make a simple OpenAI call with the latest OpenAI version

### Expected behavior

OpenAI in there recent changes have updated their error API so no longer it has to be referenced with OpenAI.error.

The error is in this file.
langchain/libs/langchain/langchain/chat_models/openai.py

So the error list that is 

    errors = [
        openai.error.Timeout,
        openai.error.APIError,
        openai.error.APIConnectionError,
        openai.error.RateLimitError,
        openai.error.ServiceUnavailableError,
    ]

should be  updated to
    errors = [
        openai.Timeout,
        openai.APIError,
        openai.APIConnectionError,
        openai.RateLimitError,
        openai.ServiceUnavailableError,
    ]
Improve youtube tool prompting and arg parsing
Labels: name: auto:improvement

**Description:** 

The youtube search tool does not work well with openai function calling.  OpenAI infers that it should provide two arguments: `topic: str` and `num_results: int`, but the tool accepts a single argument `query: str` which it tries to parse into a topic and num_results.

This is a backwards compatible fix that adds an argument `num_results: Optional[int]` to the tool while maintaining the previous parsing logic if the argument is not provided.

This also improves the prompting for the tool.  Previously the tool description mentioned only that you could search for a person on YouTube, when in fact the tool can be used to search for any topic.

- **Issue:** N/A
- **Dependencies:** N/A
- **Tag maintainer:** @baskaryan
Add MyScaleWithoutJSON which allows user to wrap columns into Document's Metadata
Labels: name: lgtm, name: area: vector store, name: auto:improvement

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Replace this entire comment with:
  - **Description:** Add MyScaleWithoutJSON which allows user to wrap columns into Document's Metadata
  - **Tag maintainer:** @baskaryan