delayedkarma/langchain-issues huggingface dataset:
Fix wrong class instantiation in docs MMR example
Labels: name: lgtm

# Fix wrong class instantiation in docs MMR example

<!--
Thank you for contributing to LangChain! Your PR will appear in our release under the title you set. Please make sure it highlights your valuable contribution.

Replace this with a description of the change, the issue it fixes (if applicable), and relevant context. List any dependencies required for this change.

After you're done, someone will review your PR. They may suggest improvements. If no one reviews your PR within a few days, feel free to @-mention the same people again, as notifications can get lost.

Finally, we'd love to show appreciation for your contribution - if you'd like us to shout you out on Twitter, please also include your handle!
-->

When looking at the Maximal Marginal Relevance ExampleSelector example at https://python.langchain.com/en/latest/modules/prompts/example_selectors/examples/mmr.html, I noticed that there seems to be an error. Initially, the `MaxMarginalRelevanceExampleSelector` class is used as an `example_selector` argument to the `FewShotPromptTemplate` class. Then, according to the text, a comparison is made to regular similarity search. However, the `FewShotPromptTemplate` still uses the `MaxMarginalRelevanceExampleSelector` class, so the output is the same. 

To fix it, I added an instantiation of the `SemanticSimilarityExampleSelector` class, because this seems to be what is intended.


## Who can review?

@hwchase17
Tracking of time to generate text

### Feature request

Hey Team,
I would like to propose a new feature that will enhance the visibility of the LLM's response time. In addition to providing information about token usage and cost, I suggest incorporating the time taken to generate the text. This additional metric will offer valuable insights into the efficiency and performance of the system.

### Motivation

By including the response time, we can provide a comprehensive picture of the different LLM's API's performance, ensuring that we have a more accurate measure of its capabilities. This information will be particularly useful for evaluating and optimizing different LLMs, as it will shed light on the latency of the system.

### Your contribution

We can easily implement this by adding additional variables in callbacks of LLM's. I would like to implement this feature.

Here the example code:

```
class BaseCallbackHandler:
    """Base callback handler that can be used to handle callbacks from langchain."""
    time_take_by_llm_to_generate_text: float = 0
    start_time: float = 0
    def on_llm_start(
        self, serialized: Dict[str, Any], prompts: List[str], **kwargs: Any
    ) -> Any:
        """Run when LLM starts running."""
        self.start_time = datetime.now()
   

    def on_llm_end(self, response: LLMResult, **kwargs: Any) -> Any:
        """Run when LLM ends running."""
        self.end_time = datetime.now()
        self.time_take_by_llm_to_generate_text += end_time - start_time
```
QuickFix for FinalStreamingStdOutCallbackHandler: Ignore new lines & white spaces

# Make FinalStreamingStdOutCallbackHandler more robust by ignoring new lines & white spaces

`FinalStreamingStdOutCallbackHandler` doesn't work out of the box with `ChatOpenAI`, as it tokenized slightly differently than `OpenAI`. The response of `OpenAI` contains the tokens `["\nFinal", " Answer", ":"]` while `ChatOpenAI` contains `["Final", " Answer", ":"]`.

This PR make `FinalStreamingStdOutCallbackHandler` more robust by ignoring new lines & white spaces when determining if the answer prefix has been reached.

Fixes #5433

## Who can review?
Community members can review the PR once tests pass. Tag maintainers/contributors who might be interested:
Tracing / Callbacks
- @agola11

Twitter: [@UmerHAdil](https://twitter.com/@UmerHAdil) | Discord: RicChilligerDude#7589
simple code update

# Update the comments and python logic 

<!--
Thank you for contributing to LangChain! Your PR will appear in our release under the title you set. Please make sure it highlights your valuable contribution.

Replace this with a description of the change, the issue it fixes (if applicable), and relevant context. List any dependencies required for this change.

After you're done, someone will review your PR. They may suggest improvements. If no one reviews your PR within a few days, feel free to @-mention the same people again, as notifications can get lost.

Finally, we'd love to show appreciation for your contribution - if you'd like us to shout you out on Twitter, please also include your handle!
-->

<!-- Remove if not applicable -->

Fixes # (issue)

## Before submitting

<!-- If you're adding a new integration, please include:

1. a test for the integration - favor unit tests that does not rely on network access.
2. an example notebook showing its use


See contribution guidelines for more information on how to write tests, lint
etc:

https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
-->

## Who can review?

Community members can review the PR once tests pass. Tag maintainers/contributors who might be interested:

<!-- For a quicker response, figure out the right person to tag with @

  @hwchase17 - project lead

  Tracing / Callbacks
  - @agola11

  Async
  - @agola11

  DataLoaders
  - @eyurtsev

  Models
  - @hwchase17
  - @agola11

  Agents / Tools / Toolkits
  - @vowelparrot

  VectorStores / Retrievers / Memory
  - @dev2049

 -->
feat: support NebulaGraph DB, add NebulaGraphQAChain
Labels: name: lgtm

# feat: support NebulaGraph DB, add NebulaGraphQAChain

Bring the support of [NebulaGraph](http://github.com/vesoft-inc/nebula), the open-source distributed graph database with NebulaGraphQAChain

## Before submitting

- [x] a test for the integration - favor unit tests that do not rely on network access.
- [x] an example notebook showing its use

## Who can review?

Community members can review the PR once tests pass. Tag maintainers/contributors who might be interested:

@tomasonjo
@vowelparrot
Issue: LLM callback handler not printing in Docker

### Issue you'd like to raise.

I have a Flask with LangChain setup in docker-compose, and I don't see LLM ChatOpenAI streaming output from CallbackHandlers in console, but everything works when I run it locally without Docker.

My CallbackHandler code (StreamingStdOutCallbackHandler also doesn't work):
```
from typing import Any
from langchain.callbacks.base import BaseCallbackHandler

class StreamingOutput(BaseCallbackHandler):
    def on_llm_new_token(self, token: str, **kwargs: Any) -> None:
        print(token, end="", flush=True)
```

- ChatOpenAI has streaming and verbose flags set to true
- ConversationChain has verbose flag set to True
- Flask is run with `CMD ["flask", "run", "--debug", "--with-threads"]`

I tried setting the PYTHONUNBUFFERED env variable but it didn't help - what am I doing wrong?

### Suggestion:

_No response_
Error while importing Langchain 

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Input In [6], in <cell line: 1>()
----> 1 import langchain

File ~\anaconda3\lib\site-packages\langchain\__init__.py:6, in <module>
      3 from importlib import metadata
      4 from typing import Optional
----> 6 from langchain.agents import MRKLChain, ReActChain, SelfAskWithSearchChain
      7 from langchain.cache import BaseCache
      8 from langchain.chains import (
      9     ConversationChain,
     10     LLMBashChain,
   (...)
     18     VectorDBQAWithSourcesChain,
     19 )

File ~\anaconda3\lib\site-packages\langchain\agents\__init__.py:2, in <module>
      1 """Interface for agents."""
----> 2 from langchain.agents.agent import (
      3     Agent,
      4     AgentExecutor,
      5     AgentOutputParser,
      6     BaseMultiActionAgent,
      7     BaseSingleActionAgent,
      8     LLMSingleActionAgent,
      9 )
     10 from langchain.agents.agent_toolkits import (
     11     create_csv_agent,
     12     create_json_agent,
   (...)
     21     create_vectorstore_router_agent,
     22 )
     23 from langchain.agents.agent_types import AgentType

File ~\anaconda3\lib\site-packages\langchain\agents\agent.py:13, in <module>
     10 from typing import Any, Callable, Dict, List, Optional, Sequence, Tuple, Union
     12 import yaml
---> 13 from pydantic import BaseModel, root_validator
     15 from langchain.agents.agent_types import AgentType
     16 from langchain.agents.tools import InvalidTool

File ~\anaconda3\lib\site-packages\pydantic\__init__.py:2, in init pydantic.__init__()

File ~\anaconda3\lib\site-packages\pydantic\dataclasses.py:48, in init pydantic.dataclasses()

File ~\anaconda3\lib\site-packages\pydantic\main.py:120, in init pydantic.main()

TypeError: dataclass_transform() got an unexpected keyword argument 'field_specifiers'
Gfrenkel/sql columns option
Labels: name: auto:improvement

# Add column names to SQLDatabase.run() result

Description:
Current state: SQLDatabase.run() method returns a result from the database after executing a SQL query but there is no option of adding the column names as well to the returned result.

PR offer the following solution:

Adding a argument "include_column_names_on_query" to the SQLDatabase constructor.
By default, this argument will be set to False, but when set to True, SQLDatabase.run() method will add the column names to the result (if the result is not empty).


Would love to hear some feedback!

  -  @hwchase17
  - @vowelparrot
[SSL: CERTIFICATE_VERIFY_FAILED] while load from SitemapLoader

### System Info

langchain: 0.0.181
platform: windows
python: 3.11.3

### Who can help?

@eyurtsev 

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [X] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

```py
site_loader = SitemapLoader(web_path="https://help.glueup.com/sitemap_index.xml")

docs = site_loader.load()
print(docs[0])

# ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1002)
```

### Expected behavior

print the frist doc
Add option to ignore column names in SQLDatabase.run()

# Add column names to SQLDatabase.run() result

Description:
Current state: SQLDatabase.run() method returns a result from the database after executing a SQL query but there is no option of adding the column names as well to the returned result.

PR offer the following solution:

Adding a argument "ignore_column_names_on_query" to the SQLDatabase constructor.
By default, the argument is set to True, but when set to False, SQLDatabase.run() method will add the column names to the result (if the result is not empty).


Would love to hear some feedback!

  -  @hwchase17
  - @vowelparrot
[Feature Request] Supoprts document loader caching

### Feature request

I want langchain to implement caching for document loaders in a way similar to how it caches LLM calls, like this:

```python
from langchain.cache import InMemoryCache
langchain.document_loader_cache = InMemoryCache()
```

### Motivation

Loading from certain documents with langchain document loader can be an expensive operation (for example, I implemneted a custom PDF loader using OCR that's slow, or loaders that involves network calls).


### Your contribution

If langchain would accept such a PR, I'd try to implement the logic and file a PR.
Update youtube.py - Fix metadata validation error in YoutubeLoader
Labels: name: lgtm

This commit addresses a ValueError occurring when the YoutubeLoader class tries to add datetime metadata from a YouTube video's publish date. The error was happening because the ChromaDB metadata validation only accepts str, int, or float data types.

In the `_get_video_info` method of the `YoutubeLoader` class, the publish date retrieved from the YouTube video was of datetime type. This commit fixes the issue by converting the datetime object to a string before adding it to the metadata dictionary.

Additionally, this commit introduces error handling in the `_get_video_info` method to ensure that all metadata fields have valid values. If a metadata field is found to be None, a default value is assigned. This prevents potential errors during metadata validation when metadata fields are None.

The file modified in this commit is youtube.py.

# Your PR Title (What it does)

<!--
Thank you for contributing to LangChain! Your PR will appear in our release under the title you set. Please make sure it highlights your valuable contribution.

Replace this with a description of the change, the issue it fixes (if applicable), and relevant context. List any dependencies required for this change.

After you're done, someone will review your PR. They may suggest improvements. If no one reviews your PR within a few days, feel free to @-mention the same people again, as notifications can get lost.

Finally, we'd love to show appreciation for your contribution - if you'd like us to shout you out on Twitter, please also include your handle!
-->

<!-- Remove if not applicable -->

Fixes # (issue)

## Before submitting

<!-- If you're adding a new integration, please include:

1. a test for the integration - favor unit tests that does not rely on network access.
2. an example notebook showing its use


See contribution guidelines for more information on how to write tests, lint
etc:

https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
-->

## Who can review?

Community members can review the PR once tests pass. Tag maintainers/contributors who might be interested:

<!-- For a quicker response, figure out the right person to tag with @

  @hwchase17 - project lead

  Tracing / Callbacks
  - @agola11

  Async
  - @agola11

  DataLoaders
  - @eyurtsev

  Models
  - @hwchase17
  - @agola11

  Agents / Tools / Toolkits
  - @vowelparrot

  VectorStores / Retrievers / Memory
  - @dev2049

 -->
IndexError: list index out of range when use Chroma.from_documents

### System Info

Lang Chain    0.0.186
Mac OS Ventura
Python 3.10

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

why i got IndexError: list index out of range when use Chroma.from_documents


import os

from langchain.document_loaders import BiliBiliLoader
from langchain.embeddings.openai import OpenAIEmbeddings
from langchain.vectorstores import Chroma
from langchain.text_splitter import RecursiveCharacterTextSplitter

os.environ["OPENAI_API_KEY"] = "***"

loader = BiliBiliLoader(["https://www.bilibili.com/video/BV18o4y137n1/"])

documents = loader.load()


text_splitter = RecursiveCharacterTextSplitter(
  chunk_size=1000,
  chunk_overlap=20
)


documents = text_splitter.split_documents(documents)


embeddings = OpenAIEmbeddings()


db = Chroma.from_documents(documents, embeddings, persist_directory="./db")
db.persist()


Traceback (most recent call last):
  File "/bilibili/bilibili_embeddings.py", line 28, in <module>
    db = Chroma.from_documents(documents, embeddings, persist_directory="./db")
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/langchain/vectorstores/chroma.py", line 422, in from_documents
    return cls.from_texts(
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/langchain/vectorstores/chroma.py", line 390, in from_texts
    chroma_collection.add_texts(texts=texts, metadatas=metadatas, ids=ids)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/langchain/vectorstores/chroma.py", line 160, in add_texts
    self._collection.add(
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/chromadb/api/models/Collection.py", line 103, in add
    ids, embeddings, metadatas, documents = self._validate_embedding_set(
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/chromadb/api/models/Collection.py", line 354, in _validate_embedding_set
    ids = validate_ids(maybe_cast_one_to_many(ids))
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/chromadb/api/types.py", line 82, in maybe_cast_one_to_many
    if isinstance(target[0], (int, float)):
IndexError: list index out of range

### Expected behavior

index gen succefully in the persist_directory
Getting only the instance of the vector store without adding text

### Feature request

Hi,
Currently the from_documents method will add the embeddings and then return the instance of the store. Why don't we have a method to just return the store. This is useful when I already have a loaded vector store and I just need the instance of the store. It will be like the below code without the _store.add_texts_

```
store = cls(
            connection_string=connection_string,
            collection_name=collection_name,
            embedding_function=embedding,
            distance_strategy=distance_strategy,
            pre_delete_collection=pre_delete_collection,
        )

        store.add_texts(texts=texts, metadatas=metadatas, ids=ids, **kwargs)
        return store
```

### Motivation

This is required when I already have a loaded vector store

### Your contribution

If this change is acceptable, I can add this functionality and create a PR
llm_chain.llm.save("llm.json")  # method not found

### System Info

llm_chain.llm.save("llm.json")  # method not found
bug in .ipynb：
docs/modules/chains/generic/serialization.ipynb


### Who can help?

_No response_

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

llm_chain.llm.save("llm.json")  # method not found
bug in .ipynb：
docs/modules/chains/generic/serialization.ipynb

### Expected behavior

llm_chain.llm.save("llm.json")  # method not found
bug in .ipynb：
docs/modules/chains/generic/serialization.ipynb
Qdrant Document object is not behaving correct

### System Info

Langchain Version: 0.0.186
MacOS Ventura 13.3 - M1
Python 3.10.8

### Who can help?

@hwchase17 
@agola11 

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [X] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

There is an error in the Qdrant Vectorstore code ([`qdrant.py`](https://github.com/hwchase17/langchain/blob/master/langchain/vectorstores/qdrant.py)). Specifically, with the function `_document_from_scored_point` on line 468 of [`qdrant.py`](https://github.com/hwchase17/langchain/blob/master/langchain/vectorstores/qdrant.py). The Document object is from the [`schema.py`](https://github.com/hwchase17/langchain/blob/master/langchain/schema.py) The function takes as few arguments:

page_content: str
metadata: dict = Field(default_factory=dict)

The [`qdrant.py`](https://github.com/hwchase17/langchain/blob/master/langchain/vectorstores/qdrant.py) file incorrectly makes the *metadata* parameter a string instead of a dict. 

This creates a few problems: 
1. If the *metadata* parameter in the function `_document_from_scored_point` is passed anything but None or a key that is not in the score_point object (a.k.a. None), it will error out. This is because this variable should be a dict, which is not returned from a dictionary *get* method.
2. The *metadata_payload_key* parameter does not seem to have a purpose / does not make sense given the above context.
 3. It is impossible for metadata to be returned when using the Qdrant *similarity_search* function within Langchain due to this issue.

### Expected behavior

I would like to be able to return metadata when using similarity_search with Qdrant. If you run [this](https://www.pinecone.io/learn/langchain-retrieval-augmentation/) example / focus on the vectorstore part and swap out the Pinecone work for Qdrant, there does not seem to be a way to use similarity search with metadata similar to how the example shows it.
Ability to specify credentials wihen using Google BigQuery as a data loader
Labels: name: lgtm

# Adds ability to specify credentials when using Google BigQuery as a data loader

Fixes #5465 . Adds ability to set credentials which must be of the `google.auth.credentials.Credentials` type. This argument is optional and will default to `None.

## Who can review?

Community members can review the PR once tests pass. Tag maintainers/contributors who might be interested:
@eyurtsev
Bedrock llm and embeddings
Labels: name: enhancement, name: llms, name: area: embeddings

# Bedrock LLM and Embeddings
This PR adds a new LLM and an Embeddings class for the [Bedrock](https://aws.amazon.com/bedrock) service. The PR also includes example notebooks for using the LLM class in a conversation chain and embeddings usage in creating an embedding for a query and document.

**Note**:  AWS is doing a private release of the Bedrock service on 05/31/2023; users need to request access and added to an allowlist in order to start using the Bedrock models and embeddings. Please use the [Bedrock Home Page](https://aws.amazon.com/bedrock) to request access and to learn more about the models available in Bedrock.

<!-- For a quicker response, figure out the right person to tag with @

  @hwchase17 - project lead

  Tracing / Callbacks
  - @agola11

  Async
  - @agola11

  DataLoaders
  - @eyurtsev

  Models
  - @hwchase17
  - @agola11

  Agents / Tools / Toolkits
  - @vowelparrot

  VectorStores / Retrievers / Memory
  - @dev2049

 -->