delayedkarma/langchain-issues huggingface dataset:
fix(llms): on_llm_start could change prompts
Labels: name: auto:bug, name: area: models

<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: on_llm_start could change prompts
  - Issue: the issue #8725 
  - Tag maintainer: @baskaryan @agola11


Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the same people again.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->
  - Description: on_llm_start could change prompts
  - Issue: the issue #8725 
  - Tag maintainer: @baskaryan @agola11
how to expand the integrated content
Labels: name: auto:question, name: auto:documentation, name: area: models

### Issue with current documentation:

Hello, how to expand the integrated content, such as the SQLDatabaseToolkit class, after executing the existing tools to continue other work, I checked the following code in the SQLDatabaseToolkit class,`
def get_tools(self) -> List[BaseTool]:
        """Get the tools in the toolkit."""
        return [
            QuerySQLDataBaseTool(db=self.db),
            InfoSQLDatabaseTool(db=self.db),
            ListSQLDatabaseTool(db=self.db),
            QueryCheckerTool(db=self.db, llm=self.llm),
        ]`
 how can I add my own tool class to expand it?

I tried to subclass BaseTool according to the instructions in the documentation, and then override the get_tools method of the SQLDatabaseToolkit class to add my custom BaseTool. However, it failed.
I'm also considering how to extend the integration for other components to achieve my task scenario.

### Idea or request for content:

_No response_
How to retrieve vectors by ids for LangChain vectorstore FAISS?
Labels: name: area: vector store, name: auto:enhancement

### Feature request

In Qdrant vectorstore, we have a method:
```
    def retrieve(
            self,
            collection_name: str,
            ids: Sequence[types.PointId],
            with_payload: Union[bool, Sequence[str], types.PayloadSelector] = True,
            with_vectors: Union[bool, Sequence[str]] = False,
            consistency: Optional[types.ReadConsistency] = None,
    ) -> List[types.Record]:
        """Retrieve stored points by IDs

        Args:
            collection_name: Name of the collection to lookup in
            ids: list of IDs to lookup
            with_payload:
                - Specify which stored payload should be attached to the result.
                - If `True` - attach all payload
                - If `False` - do not attach any payload
                - If List of string - include only specified fields
                - If `PayloadSelector` - use explicit rules
            with_vectors:
                - If `True` - Attach stored vector to the search result.
                - If `False` - Do not attach vector.
                - If List of string - Attach only specified vectors.
                - Default: `False`
            consistency:
                Read consistency of the search. Defines how many replicas should be queried before returning the result.
                Values:
                - int - number of replicas to query, values should present in all queried replicas
                - 'majority' - query all replicas, but return values present in the majority of replicas
                - 'quorum' - query the majority of replicas, return values present in all of them
                - 'all' - query all replicas, and return values present in all replicas

        Returns:
            List of points
        """
```

 I have not see simarly method in FAISS implementation, how to retrieve vectors `with_vectors=True` in FAISS vectorstore?

### Motivation

I am using FAISS vectorstore, and now:
1. use FAISS added some doc and returned doc ids;
2. I want a method like in Qdrant, to retrieve the embedding from this doc ids;

Here is the Qdrant simarliy code:
```
ids_inserted = qdrant_client.add_texts(
    [description],
    [{
        "source": "todo"
    }],
)

# now I want to get the embeddings
records_inserted = qdrant_client.retrieve(
    collection_name="test",
    ids=ids_inserted,
    with_vectors=True
)
self.embedding = records_inserted[0].vector
```

### Your contribution

Sorry, no.
Change default Metaphor search example to use prompt optimizer
Labels: name: auto:documentation

- fix install command
- change example notebook to use Metaphor autoprompt by default

<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the same people again.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->
Fix metaphor install command in guide
Labels: name: auto:documentation

<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the same people again.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->
chunk_size arg is not used in OpenAIEmbeddings's embed_documents and aembed_documents methods
Labels: name: area: embeddings, name: auto:bug

### System Info

v0.0.256

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Add `print(f"chunk-size: {_chunk_size}")` in OpenAIEmbeddings._get_len_safe_embeddings after `_chunk_size = chunk_size or self.chunk_size`, and pass chunk_size argument to embed_documents other than the default value of 1000. 1000 will be printed instead of the chunk_size passed as argument.

### Expected behavior

chunk_size arg is not used in OpenAIEmbeddings's embed_documents and aembed_documents methods. It should be passed to self._get_len_safe_embeddings.
Check: https://github.com/langchain-ai/langchain/blob/v0.0.256/libs/langchain/langchain/embeddings/openai.py#L463
Bagatur/docsly
Labels: name: auto:documentation

<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the same people again.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->
Simplify linting workflow by adding to pre-commit
Labels: name: auto:improvement

### Feature request

## Description
Add format/lint actions to the pre-commit tool, so user's don't have to run these locally on their machines. This is a 2 stage process:

1. Make this part of the local commit process. This will ensure that all commits run these explicitly before pushing the change. 
2. Adding this to the CI will ensure that users who choose to ignore setting up the pre-commit in their local repo, can rely on the CI to fix some of the formatting and linting (fixable). This also avoids some of the system specific issues/differences that users might end up having on their local machines (stale versions of lint tools) by ensuring that the CI is running these on one consistent up-to-date platform. 

### Motivation

The current manual linting process adds churn and friction to the dev workflow as most devs end up pushing their changes without running these steps, and rely on the CI to remind them on running them. Some of the formatting and linting can be automated with CI, and does not require a dev's direct involvement. 

### Your contribution

I am happy to work on this, but would need the core team's input before going ahead with this change.
conversational-retrieval-agent   using  AgentTokenBufferMemory : Still cannot prevent maximum content length errors 
Labels: name: area: memory, name: area: agent, name: auto:bug, name: area: models

### System Info

Based on https://github.com/hwchase17/conversational-retrieval-agent/blob/master/streamlit.py, did few changes to code.  
Since the beginning of the tests could not prevent the max content length error.  

the best change was using: 
memory = AgentTokenBufferMemory(memory_key="history", llm=llm, max_history=5, max_token_limit= 3000)
llm = ChatOpenAI(temperature=0, streaming=True, model="gpt-3.5-turbo",max_tokens=500)

It took longer but still got the error message. All the conversation was logged using LangSmith , username rubensmau@gmail.com   ,, https://smith.langchain.com/projects/p/2e2c3411-5910-4785-985d-a4b31702e6c7/r/2893c334-9650-4125-99df-af6e18481b12

It seemed that the context was under control,  but the latest question broke it.

Python version - 3.11.3
Langchain  - 0.0.252

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [X] Memory
- [X] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

memory = AgentTokenBufferMemory(memory_key="history", llm=llm, max_history=5, max_token_limit= 3000)
llm = ChatOpenAI(temperature=0, streaming=True, model="gpt-3.5-turbo",max_tokens=500)

### Expected behavior

All the conversation was logged using LangSmith , username rubensmau@gmail.com   ,, https://smith.langchain.com/projects/p/2e2c3411-5910-4785-985d-a4b31702e6c7/r/2893c334-9650-4125-99df-af6e18481b12
Add LabelStudio integration
Labels: name: auto:enhancement

This PR introduces [Label Studio](https://labelstud.io/) integration with LangChain via `LabelStudioCallbackHandler`:

- sending data to the Label Studio instance
- labeling dataset for supervised LLM finetuning
- rating model responses
- tracking and displaying chat history
- support for custom data labeling workflow

### Example

```
chat_llm = ChatOpenAI(callbacks=[LabelStudioCallbackHandler(mode="chat")])
chat_llm([
    SystemMessage(content="Always use emojis in your responses."),
        HumanMessage(content="Hey AI, how's your day going?"),
    AIMessage(content="🤖 I don't have feelings, but I'm running smoothly! How can I help you today?"),
        HumanMessage(content="I'm feeling a bit down. Any advice?"),
    AIMessage(content="🤗 I'm sorry to hear that. Remember, it's okay to seek help or talk to someone if you need to. 💬"),
        HumanMessage(content="Can you tell me a joke to lighten the mood?"),
    AIMessage(content="Of course! 🎭 Why did the scarecrow win an award? Because he was outstanding in his field! 🌾"),
        HumanMessage(content="Haha, that was a good one! Thanks for cheering me up."),
    AIMessage(content="Always here to help! 😊 If you need anything else, just let me know."),
        HumanMessage(content="Will do! By the way, can you recommend a good movie?"),
])
```

<img width="906" alt="image" src="https://github.com/langchain-ai/langchain/assets/6087484/0a1cf559-0bd3-4250-ad96-6e71dbb1d2f3">


### Dependencies
- [label-studio](https://pypi.org/project/label-studio/)
- [label-studio-sdk](https://pypi.org/project/label-studio-sdk/)

https://twitter.com/labelstudiohq
 

<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the same people again.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->
Fix colab link for extraction ntbk
Labels: name: auto:documentation

<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the same people again.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->
Introduce Nebula LLM to LangChain
Labels: name: lgtm, name: auto:enhancement

<!-- Thank you for contributing to LangChain!

Replace this comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure you're PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use.

Maintainer responsibilities:
  - General / Misc / if you don't know who to tag: @baskaryan
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
  - Models / Prompts: @hwchase17, @baskaryan
  - Memory: @hwchase17
  - Agents / Tools / Toolkits: @hinthornw
  - Tracing / Callbacks: @agola11
  - Async: @agola11

If no one reviews your PR within a few days, feel free to @-mention the same people again.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md
 -->

## Description

This PR adds Nebula to the available LLMs in LangChain.

Nebula is an LLM focused on conversation understanding and enables users to extract conversation insights from video, audio, text, and chat-based conversations. These conversations can occur between any mix of human or AI participants.

Examples of some questions you could ask Nebula from a given conversation are:
- What could be the customer’s pain points based on the conversation?
- What sales opportunities can be identified from this conversation?
- What best practices can be derived from this conversation for future customer interactions?

You can read more about Nebula here:
https://symbl.ai/blog/extract-insights-symbl-ai-generative-ai-recall-ai-meetings/

#### Integration Test 

An integration test is added, but it requires network access. Since Nebula is fully managed like OpenAI, network access is required to exercise the integration test.

#### Linting

- [x] make lint
- [x] make test (TODO: there seems to be a failure in another non-related test??? Need to check on this.)
- [x] make format

### Dependencies

No new dependencies were introduced.

### Twitter handle

[@symbldotai](https://twitter.com/symbldotai)
[@dvonthenen](https://twitter.com/dvonthenen)


If you have any questions, please let me know.

cc: @hwchase17, @baskaryan
Issue: ConversationalRetrievalChain.from_llm sometime answer with the following up question
Labels: name: auto:bug

### Issue you'd like to raise.

Hi, Im using  ConversationalRetrievalChain.from_llm and I notice that sometimes the answer is the following up question.
Also sometimes the answer generates a few Q and A all togheter.

Anyone with the same problem?

regards.

### Suggestion:

_No response_
ChatOpenAI agenerate does not use internal _agenerate and does not support message roles
Labels: name: auto:bug

### System Info

langchain==0.0.250
Python 3.11.3
22.5.0 Darwin Kernel Version 22.5.0

### Who can help?

@agola11

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [X] Async

### Reproduction

This code runs perfectly well if `agenerate` is replaced with `_agenerate`, meaning that the super function is not using the internal implementation correctly.
```
llm = ChatOpenAI(temperature=0.7, max_tokens=max_tokens)

messages = [
    SystemMessagePromptTemplate.from_template("You are a funny chatbot"),
    HumanMessagePromptTemplate.from_template("Tell me a joke about {topic}")
]

ChatPromptTemplate.from_messages(
        messages
    )

formatted_messages = chat_prompt.format_prompt(
                topic='horses'
).to_messages()

async def async_generate(llm, formatted_messages):
    return await llm.agenerate(messages=formatted_messages)

res = await async_generate(llm, formatted_messages)
```


### Expected behavior

agenerate should work the same as _agenerate
Create ArcGISLoader & example notebook
Labels: name: lgtm, name: area: doc loader, name: auto:enhancement

  - Description: Adds the ArcGISLoader class to `langchain.document_loaders`
  - Allows users to load data from ArcGIS Online, Portal, and similar
  - Users can authenticate with `arcgis.gis.GIS` or retrieve public data anonymously
  - Uses the `arcgis.features.FeatureLayer` class to retrieve the data
  - Defines the most relevant keywords arguments and accepts `**kwargs`
  - Dependencies: Using this class requires `arcgis` and, optionally, `bs4.BeautifulSoup`.

Tagging maintainers:
  - DataLoaders / VectorStores / Retrievers: @rlancemartin, @eyurtsev
Quality of answers gets drastically bad with longer chains in Langchain
Labels: name: auto:bug, name: auto:question

I think most of the people who works with Langchain in building products or automating stuff noticed how bad the quality of answers degrades when you ask multiple of questions using the same chain (e.g: the chain gets longer). It gets to answer the first 2 or 3 questions right then it may hallucinate returning a wrong answer, it may fail to answer your question or even crash regardless of the GPT model used.

Is there a way anyone found in order to optimize this? One way I always try is to prompt engineering things but it doesn't help as much.