delayedkarma/langchain-issues huggingface dataset:
Issue: How to modify the actual prompt to add in new input parameters of the open ai functions agent, so that during the running of the agent, we pass those parameters also
Labels: name: area: agent, name: auto:question, name: auto:improvement

### Issue you'd like to raise.

I want to modify the prompt of the open ai function agent, so i can add more parameters that i can pass as inputs during execution

initialize_agent(tools=tool_items, 
llm=llm, 
agent=AgentType.OPENAI_FUNCTIONS)

and during execution such as 

result = agent_chain.run({"input": "Whats the distance between Malmo and Stockholm"})

i want to be able to pass several input parameters so that for example it becomes like this

 agent_chain.run({"input": "Whats the distance between Malmo and Stockholm","language":"French"}) etc

I have been able to do that successfully for other chains, but not for this one
How do i achieve this

### Suggestion:

_No response_
Add lock for using global config enum weak map
Labels: name: auto:improvement

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
ConversationTokenBufferMemory doesn't work with db based chat history
Labels: name: area: memory, name: auto:bug, name: auto:improvement, name: area: models

### System Info

Langchain version: 0.0.272
Python Version: 3.11.0

### Who can help?

@hwchase17 
@ag

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [X] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

The `save_context` method of the `ConversationTokenBufferMemory` does pop operation directly on `chat_memory.messages` [link to code](https://github.com/langchain-ai/langchain/blob/31f264169db4ab23689f2e179983f1cfdfd1a33a/libs/langchain/langchain/memory/token_buffer.py#L48-L49)

It works only with in-memory memory and not with db-backed memory like `PostgresChatMessageHistory` because the `buffer`( `self.chat_memory.messages` ) is immutable

```python
    def save_context(self, inputs: Dict[str, Any], outputs: Dict[str, str]) -> None:
        """Save context from this conversation to buffer. Pruned."""
        super().save_context(inputs, outputs)
        # Prune buffer if it exceeds max token limit
        buffer = self.chat_memory.messages
        curr_buffer_length = self.llm.get_num_tokens_from_messages(buffer)
        if curr_buffer_length > self.max_token_limit:
            pruned_memory = []
            while curr_buffer_length > self.max_token_limit:
                pruned_memory.append(buffer.pop(0))
                curr_buffer_length = self.llm.get_num_tokens_from_messages(buffer)
```

### Expected behavior

The `ChatMessageHistory` should work like an interface with`ConversationTokenBufferMemory`.

Step to resolve
- remove direct calls to `self.chat_memory.messages`
- add a private variable to hold chat history
Export merge_configs function
Labels: name: auto:improvement

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Anonymizer small fixes
Labels: name: auto:nit

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Support for OpenAI quotas
Labels: name: auto:bug, name: area: models

### System Info

Python 3.11
Lanchain 315

### Who can help?

@hwchase17
@agola11 

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

import openai
import os
from langchain.llms import OpenAI
llm = OpenAI(temperature=0.3)
print(llm.predict("What is the capital of India?"))

### Expected behavior

When OpenAI quotas are reached (or no payment method is defined), requests should not be retried but should raise an appropriate error.

Error in console :
```
Retrying langchain.chat_models.openai.acompletion_with_retry.<locals>._completion_with_retry in 4.0 seconds as it raised RateLimitError: You exceeded your current quota, please check your plan and billing details..
```

Equivalent TS issue : https://github.com/langchain-ai/langchainjs/issues/1929
Fix in TS land : https://github.com/langchain-ai/langchainjs/pull/1934
Add validation for configurable keys passed to .with_config()
Labels: name: auto:improvement

- Fix some typing issues found while doing that

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Ensure that configurable fields with enums support deduplication
Labels: name: auto:improvement

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
How to use Multiple Retrieaval Sources and Added Memory at SQLDatabaseChain
Labels: name: area: memory, name: area: doc loader, name: auto:question, name: area: models

### Discussed in https://github.com/langchain-ai/langchain/discussions/11846

<div type='discussions-op-text'>

<sup>Originally posted by **Marbyun** October 16, 2023</sup>
Hi folks!
i have case to create chatbot that will use 2 sources as the dataset (text file and sqlite). I use Multiple Retrieaval Sources [document](https://python.langchain.com/docs/use_cases/question_answering/multiple_retrieval) as my main code. But the code can not do continuous conversation like this:

> q:'what's name of employee id xx.xx'
> a:'his name is xxx xxx'
> q:'what's his email?'
> a:'his email is xxx@email.com'

and then i get code from this [update](https://github.com/langchain-ai/langchain/pull/8597/files) by @keenborder786 that can perfomance my q&a, but rigth now i get confuse how to combine it? can you guys help me please, i am very new in here...

> Ps:
> - I have 2 sources Text file have data about company profile and SQLite have employee data
> - Need to combine the sources and can do continuous conversation</div>
Enable token usage count for AWS Bedrock
Labels: name: auto:enhancement, name: area: models, name: integration: aws

### Feature request

There should be a callback handler like [OpenAICallbackHandler](https://github.com/langchain-ai/langchain/blob/31f264169db4ab23689f2e179983f1cfdfd1a33a/libs/langchain/langchain/callbacks/openai_info.py#L120) for AWS Bedrock models, so that we can easily get the token usage and monitor cost.

It looks like the AWS Bedrock API doesn't return with token usage. Unlike OpenAI.

Can we have similar feature, perhaps [using the internal Anthropic tokenizer to count](https://github.com/langchain-ai/langchain/blob/31f264169db4ab23689f2e179983f1cfdfd1a33a/libs/langchain/langchain/llms/bedrock.py#L405)?

### Motivation

Cost monitoring

### Your contribution

PR
logger.info(output.decode().strip()) of common.py  will raise a mistake when use convert_office_doc to convert .doc file to .docx file because it is not encode by 'utf-8'
Labels: name: area: doc loader, name: auto:bug, name: auto:improvement

### Issue with current documentation:

I want to load doc type file use UnstructuredFileLoader, andI installed all require library, but I got this mistake:
Traceback (most recent call last):
  File "D:\soft\Anaconda3\lib\site-packages\unstructured\partition\doc.py", line 67, in partition_doc
    convert_office_doc(
  File "D:\soft\Anaconda3\lib\site-packages\unstructured\partition\common.py", line 294, in convert_office_doc
    logger.info(output.decode().strip())
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd0 in position 42: invalid continuation byte.
Then I see the code of function convert_office_doc that can not change encoding type, this mistake is 'utf-8' is not valid encoding type, so I changed this to 'gb2312', it worked.
So, whether a parameter should be added for the encoding type when reading the file?

### Idea or request for content:

So, whether a parameter should be added for the encoding type when reading the file?
For example, UnstructuredFileLoader(file_path, file_name, coding='utf-8').
fix: use jsonb instead of json for pgvector
Labels: name: area: vector store, name: auto:bug

## Description
When trying to integrate pgvector as a vector store with langchain, I got the following error:

```
2023-10-16 18:27:54.326 EDT [87858] ERROR:  operator does not exist: json @> unknown at character 106
2023-10-16 18:27:54.326 EDT [87858] HINT:  No operator matches the given name and argument types. You might need to add explicit type casts.
2023-10-16 18:27:54.326 EDT [87858] STATEMENT:
              SELECT *, embedding <=> $1 as "_distance"
              FROM langchain_pg_embedding
              WHERE cmetadata @> $2
              ORDER BY "_distance" ASC
              LIMIT $3;
```

## The problem
From my brief research, it appears that the `@>` operator only works with jsonb. So I'm trying to figure out how this ever worked in the first place, but I'm not a postgres expert.

## Testing / Reproduced

To test this, I ran the following against my database after langchain created it and this seemed to fix my issue:
```sql
ALTER TABLE langchain_pg_embedding ALTER COLUMN cmetadata SET DATA TYPE jsonb;
```

I'm using this python version of langchain for running my document loaders and storing my embeddings in PG, but then I have a chat client that's using the langchain-js lib to query it, so I happened to then go see if pgvector had the same problem over there, and it doesn't.
https://github.com/langchain-ai/langchainjs/blob/aa9b618786757c301e93caeb3253c56b56d05602/langchain/src/vectorstores/pgvector.ts#L244C38-L244C43

Let me know if the fix isn't as simple as this or if there's a better solution that should be considered like type casting or something else?

cc:
@baskaryan, @eyurtsev, @hwchase17
Use traceable context
Labels: name: auto:improvement

Example

```
from langchain.schema.runnable import RunnableLambda
from langsmith import traceable

chain = RunnableLambda(lambda x: x)

@traceable(run_type = "chain")
def my_traceable(a):
    chain.invoke(a)
my_traceable(5)
```

Would have a nested result.

This would NOT work for interleaving chains and traceables. E.g., things like thiswould still not work well

```
from langchain.schema.runnable import RunnableLambda
from langsmith import traceable

@traceable()
def other_traceable(a):
    return a

def foo(x):
    return other_traceable(x)
    
chain = RunnableLambda(foo)

@traceable(run_type = "chain")
def my_traceable(a):
    chain.invoke(a)
my_traceable(5)
```
LangChain with Stable Diffusion
Labels: name: auto:enhancement, name: area: models

### Feature request

combines LangChain with Stable Diffusion to generate a text-related image

### Motivation

The motivation behind this feature proposal is to enhance the capabilities of LangChain by integrating it with Stable Diffusion to enable the generation of images based on text inputs. This integration serves several purposes:

Enriched User Experience: The ability to generate images from text can significantly enrich user experiences in various applications. It can be applied to chatbots, content generation, creative tools, and more.

Creative Content Generation: This feature can empower users to generate creative content, artwork, or visualizations based on their textual ideas or descriptions.

Enhanced Language Model Integration: By integrating with Stable Diffusion, LangChain can harness the power of state-of-the-art generative models to create meaningful images that complement the textual context.

Research and Innovation: This integration can also serve as a research platform to explore the synergy between language models and generative image models, advancing the field of AI and creative content generation.

### Your contribution

Identify Relevant Repositories, Submit a Pull Request, Engage with the Community
API deployment not found when using Azure with embeddings
Labels: name: area: embeddings, name: auto:bug

### System Info

I'm using jupyter notebook and Azure OpenAI
Python 3.11.5
langchain==0.0.315

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [X] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

I got the error:
InvalidRequestError: The API deployment for this resource does not exist. If you created the deployment within the last 5 minutes, please wait a moment and try again.

When I run 

```
from langchain.embeddings.openai import OpenAIEmbeddings
embedding = OpenAIEmbeddings()
sentence1 = "i like dogs"
embedding1 = embedding.embed_query(sentence1)
```

But if I run - not using Lanchain - it works fine:
```
response = openai.Embedding.create(
    input="Your text string goes here",
    model="text-embedding-ada-002",
    engine="embeddingstest"
)
embeddings = response['data'][0]['embedding']
```


### Expected behavior

I would expect the embeddings of my string.
feat(llm): add together.xyz as an LLM provider
Labels: name: lgtm, name: auto:enhancement, name: area: models

  - **Description:** added together.xyz as an LLM provider, 
  - **Issues:** fix some linting issues
  - twitter handle @jilijeanlouis 

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access, ✅
  2. an example notebook showing its use. It lives in `docs/extras` directory. ❌ => classic LLM provider
fix: more robust check whether the HF model is quantized
Labels: name: auto:bug, name: area: models

Removes the check of `model.is_quantized` and adds more robust way of checking for 4bit and 8bit quantization in the `huggingface_pipeline.py` script. I had to make the original change on the outdated version of `transformers`, because the models had this property before. Seems redundant now.

Fixes: https://github.com/langchain-ai/langchain/issues/11809 and https://github.com/langchain-ai/langchain/issues/11759
Add option to RetrievalQA chains to use provided documents in call() and acall() methods
Labels: name: auto:improvement, name: area: models

#### Description
This PR adds an option to the RetrievalQA chains to use provided documents in `_call()` and`_acall()` methods like: `qa_chain({"query": question, "from_documents": documents})` where documents is `List[Document]`

Currently there is no way to reuse documents found from a `similarity_search` in a `RetrievalQA`. This PR enables the following sequence:
1. User embeds query and gets documents
2. User implements desired logic based on documents
3. User calls `RetrievalQA` without needing to re-embed query to find relevant documents.

This option will also be helpful if the user wants to gather documents in any custom way, like combining from different sources.


#### Usage



Example Usage:
```

qa_chain  =  RetrievalQA.from_chain_type(
     llm=llm,
     retriever=vector_store.as_retriever(),
)

documents = vector_store.similarity_search("Is the sky blue?")

if "blue" in documents[0].page_content:
	# Reuse documents in LLM call since they passed our condition
	response  =  qa_chain({"query": question, "from_documents": documents})
	
#True
response["source_documents"] == documents 
```
Fix issues in runnable docs examples
Labels: name: area: doc loader, name: auto:documentation

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Accept uuid argument when calling weaviate.from_documents
Labels: name: area: vector store, name: auto:improvement, name: size:XS

The `weaviate.from_documents` api should accept the `uuid` argument.

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->