delayedkarma/langchain-issues huggingface dataset:
ModuleNotFoundError at import langchain.chains
Labels: name: auto:bug

### System Info

**Platform:**
Linux Ubuntu 22.04.1

**Python:** 
3.10.12

**Langchain:**
- langchain                0.0.353
- langchain-community      0.0.7
- langchain-core           0.1.5
- langsmith                0.0.77


### Who can help?

@hwchase17 

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

```py
from langchain.chains import *
```

### Expected behavior

I expect the import won't fail, but I got an exception:

```
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ubuntu/mychatassist-api/env/lib/python3.10/site-packages/langchain/chains/__init__.py", line 20, in <module>
    from langchain.chains.api.base import APIChain
  File "/home/ubuntu/mychatassist-api/env/lib/python3.10/site-packages/langchain/chains/api/base.py", line 11, in <module>
    from langchain.callbacks.manager import (
  File "/home/ubuntu/mychatassist-api/env/lib/python3.10/site-packages/langchain/callbacks/__init__.py", line 10, in <module>
    from langchain_community.callbacks.aim_callback import AimCallbackHandler
  File "/home/ubuntu/mychatassist-api/env/lib/python3.10/site-packages/langchain_community/callbacks/__init__.py", line 24, in <module>
    from langchain_community.callbacks.manager import (
  File "/home/ubuntu/mychatassist-api/env/lib/python3.10/site-packages/langchain_community/callbacks/manager.py", line 14, in <module>
    from langchain_community.callbacks.tracers.wandb import WandbTracer
  File "/home/ubuntu/mychatassist-api/env/lib/python3.10/site-packages/langchain_community/callbacks/tracers/__init__.py", line 4, in <module>
    from langchain_core.tracers.langchain_v1 import LangChainTracerV1
ModuleNotFoundError: No module named 'langchain_core.tracers.langchain_v1'
```
langchain[patch]: Release 0.0.354
Labels: name: auto:nit, name: size:XS

<!-- Thank you for contributing to LangChain!

Please title your PR "<package>: <description>", where <package> is whichever of langchain, community, core, experimental, etc. is being modified.

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes if applicable,
  - **Dependencies:** any dependencies required for this change,
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` from the root of the package you've modified to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://python.langchain.com/docs/contributing/

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/docs/integrations` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
How long can I use Langchain to call the chatglm3 API
Labels: name: auto:enhancement, name: area: models

### Feature request

Chatglm3 has added many new features compared to previous chatglm and chatglm2, which is particularly useful for users. So there will definitely be more user demands in using Langchain to build a knowledge base, and it's unclear how long it will take for the community to adapt.

chatglm3 github path ：https://github.com/THUDM/ChatGLM3

### Motivation

Chatglm3 has added many new features compared to previous chatglm and chatglm2, which is particularly useful for users. So there will definitely be more user demands in using Langchain to build a knowledge base, and it's unclear how long it will take for the community to adapt.

chatglm3 github path ：https://github.com/THUDM/ChatGLM3

### Your contribution

i am soryy.  i can not.
Add TTL support for DynamoDBChatMessageHistory
Labels: name: auto:improvement, name: integration: aws

### Feature request

DynamoDBChatMessageHistory class is missing a TTL feature that would allow for history to automatically expire and be deleted by AWS DynamoDB service.

### Motivation

While implementing a chat history using DynamoDBChatMessageHistory, I encoutered a growing history session table. Since AWS DynamoDB supports automatic deletion of items using TTL, it would be nice to have this feature enabled in DynamoDBChatMessageHistory class when writing message into AWS DynamoDB table.

### Your contribution

I am currently in the process of submitting a PR related to this feature request.
Issue: Large database schema too big for context window using SQLDatabaseChain
Labels: name: auto:question, name: area: models

### Issue you'd like to raise.

How do I use SQLDatabaseChain with a big db schema?
I'm using:
`db = SQLDatabase.from_uri(f"postgresql://localhost:5432/test")
db_chain = SQLDatabaseChain(llm=llm, database=db, verbose=True)
`
And all models complain about context window. A smaller database schema works with some models, but that's not my use case. My models _are_ big. Is there a tried and tested method for this? 

### Suggestion:

_No response_
Using multiple templates as starter for LCEL
Labels: name: auto:question, name: area: lcel

### Issue with current documentation:

Dear all, 

Happy new year! 

Due to work legacy stuff, I am still forced to use this library. I am wondering if there is a way to pass multiple prompt template (system, human and ai) instead of just one 

Hopefully my man Dosubot can help!

Thanks you so much

Cheers,

Fra

### Idea or request for content:

_No response_
Issue: Issue while implementing RAG with Gemini LLM
Labels: name: auto:bug, name: area: models

### Issue you'd like to raise.

`document_variable_name="\n\n---\n\n".join([doc.page_content for doc in result["source_documents"]])
model = ChatGoogleGenerativeAI(model="gemini-pro",google_api_key=GOOGLE_API_KEY,temperature=0.2,convert_system_message_to_human=True)
template = f"""Use the following pieces of context to answer the question at the end. If you don't know the answer, just say that you don't know, don't try to make up an answer. Keep the answer as concise as possible. Always say "thanks for asking!" at the end of the answer.
{context}
Question: {question}
Helpful Answer:"""
QA_CHAIN_PROMPT = PromptTemplate(
    input_variables=["document_variable_name","question"],
    template=template,
)
print(QA_CHAIN_PROMPT)
qa_chain = RetrievalQA.from_chain_type(
    llm=model,
    retriever=vector_index,
    return_source_documents=True,
    chain_type="stuff",
    document_variable_name=context,
    chain_type_kwargs={"prompt": QA_CHAIN_PROMPT}
)`

below is the error I am getting:
ValidationError: 1 validation error for StuffDocumentsChain
__root__
  document_variable_name context was not found in llm_chain input_variables: [] (type=value_error)

### Suggestion:

_No response_
S3FileLoader doesn't provide the way of passing extra (unstructured_kwargs) parameters
Labels: name: area: doc loader, name: auto:improvement, name: integration: aws

### System Info

python 3.11
langchain==0.0.350

### Who can help?

@eyurtsev

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [X] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

This is the current implementation of the `S3FileLoader` class (https://github.com/langchain-ai/langchain/blob/baeac23/libs/community/langchain_community/document_loaders/s3_file.py#L13-L30):

```
class S3FileLoader(UnstructuredBaseLoader):
    """Load from `Amazon AWS S3` file."""

    def __init__(
        self,
        bucket: str,
        key: str,
        *,
        region_name: Optional[str] = None,
        api_version: Optional[str] = None,
        use_ssl: Optional[bool] = True,
        verify: Union[str, bool, None] = None,
        endpoint_url: Optional[str] = None,
        aws_access_key_id: Optional[str] = None,
        aws_secret_access_key: Optional[str] = None,
        aws_session_token: Optional[str] = None,
        boto_config: Optional[botocore.client.Config] = None,
    ):
...
```

Existing `__init__()` method and current implementation of this class doesn't support extra arguments `unstructured_kwargs`. So there is no way to manage `partition` call through these extra arguments (https://github.com/langchain-ai/langchain/blob/baeac23/libs/community/langchain_community/document_loaders/s3_file.py#L126).

### Expected behavior

The implementation of `S3FileLoader` class should follow common approach used for all ancestors of `UnstructuredBaseLoader` which support passing through extra `**unstructured_kwargs` arguments to manage `partition` call in a more granular way.
in libs/langchain/langchain/text_splitter.py comments before class/function will be splitter from class/function itself
Labels: name: area: doc loader, name: auto:improvement

### Feature request

Hello

The best practice is using comments inside the class/function, 
however, comments/TODOs have been placed before in many cases, 
Splitting it from the class/function hides some free text that can help understand the purpose and intention of class/function and may cause misinterpreted code blocks before attaching text unrelated to this block.


more visual:

```
import module


#TODO fix bug described in JIRA-WL313, wrong calculation in corner case
class1():
    pass
    
#TODO make classes great again,by adding better parsing
class2():
    pass
```
    
chunks would be 

chunk1

```
import module


#TODO fix bug 
```

chunk2

```
def class1():
    pass
    
#TODO make class great again one day  
```

chunk3

```
def class2():
    pass
```

I suggest making the split aware of such a possibility.

Best regards
Ilya

### Motivation

enrich code chunks by attaching relevant information to right chunks even if someone not follow best practice


### Your contribution

i would be happy to submit PR in case you guys think it will improve langchain which I absolutely love
Support CHM files
Labels: name: area: doc loader, name: auto:enhancement

### Feature request

Add document loader for CHM (Microsoft Compiled HTML Help) documents, possibly using pychm.

### Motivation

A lot of Windows applications provide documentation in the form of CHM files. Being able to directly load those into the language model, would greatly simplify the workflow of ingesting documentation.

### Your contribution

At this time, I'm unable to provide help with writing the loader.
Issue: How to customize prompt
Labels: name: area: agent, name: auto:question

### Issue you'd like to raise.

The following is the code I ran, referring to the official documentation. It looks like prompt_template is not effective and there is no explicit call of prompt_template in the code. How should I customize a prompt for initialize_agent.

```python
prompt_template = """  
Translate the sentence into English
{input}  
"""  
prompt_template = PromptTemplate.from_template(prompt_template)

llm = AzureOpenAI(
    model_name="gpt-4",
    engine="gpt-4"
)


@tool  
def get_word_length(word):  
    """get the length of a word"""  
    return len(word)  


tools = [get_word_length]  

agent = initialize_agent(  
    tools,  
    llm,  
    agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION,  
    verbose=True,
    prompt = prompt_template
)  


answer = agent.run("今天天气怎么样")
```

The output is
```
> Entering new AgentExecutor chain...
This is a Chinese language query asking about the weather today. I don't have access to real-time weather data and besides, my provided actions are limited to getting the length of a word.
Final Answer: I'm sorry, I cannot answer that question.
```

### Suggestion:

_No response_
Issue:While implementing Gemini, getting error while using prompt template 
Labels: name: auto:bug, name: area: models, name: integration: chroma

### Issue you'd like to raise.

below is my code
`context_text="\n\n---\n\n".join([doc.page_content for doc in result["source_documents"]])
print(context_text,"======================")
question = "Describe the Multi-head attention layer in detail?"
embeddings = GoogleGenerativeAIEmbeddings(model="models/embedding-001",google_api_key=GOOGLE_API_KEY)
vector_index = Chroma.from_texts(texts, embeddings).as_retriever(search_kwargs={"k":5})
template = f"""Use the following pieces of context to answer the question at the end. If you don't know the answer, just say that you don't know, don't try to make up an answer. Keep the answer as concise as possible. Always say "thanks for asking!" at the end of the answer.
{context_text}
Question: {question}
Helpful Answer:"""
QA_CHAIN_PROMPT = PromptTemplate.from_template(template)# Run chain
qa_chain = RetrievalQA.from_chain_type(
    model,
    retriever=vector_index,
    return_source_documents=True,
    chain_type_kwargs={"prompt": QA_CHAIN_PROMPT, "document_variable_name": "source_documents"}
)`
below is the error I am getting 
ValidationError                           Traceback (most recent call last)
[<ipython-input-99-f9533899bc31>](https://localhost:8080/#) in <cell line: 9>()
      7 Helpful Answer:"""
      8 QA_CHAIN_PROMPT = PromptTemplate.from_template(template)# Run chain
----> 9 qa_chain = RetrievalQA.from_chain_type(
     10     model,
     11     retriever=vector_index,

4 frames
/usr/local/lib/python3.10/dist-packages/pydantic/main.cpython-310-x86_64-linux-gnu.so in pydantic.main.BaseModel.__init__()

ValidationError: 1 validation error for StuffDocumentsChain
__root__
  document_variable_name source_documents was not found in llm_chain input_variables: [] (type=value_error)

### Suggestion:

_No response_
"Multiple top-level packages discovered in a flat-layout" when installing from source

### System Info

Mint 20.3
Python 3.11
Conda 23.9
Pip 23.3.2
Setuptools 69.0.3

### Who can help?

_No response_

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Clone github repo
CD into that folder
Activate conda env
Run `pip install -e .`

### Expected behavior

Langchain should be installed from source, but it doesn't
[Improvement] File_Tools added
Labels: name: area: agent, name: auto:improvement, name: size:XS

now you can provice selected_tools like below.
```
from langchain_community.agent_toolkits.file_management.toolkit import File_Tools, FileManagementToolkit

toolkit = FileManagementToolkit(root_dir=clone_path,selected_tools=[File_Tools.write_file,File_Tools.list_dir])
tools = toolkit.get_tools()
```

<!-- Thank you for contributing to LangChain!

Please title your PR "<package>: <description>", where <package> is whichever of langchain, community, core, experimental, etc. is being modified.

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes if applicable,
  - **Dependencies:** any dependencies required for this change,
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` from the root of the package you've modified to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://python.langchain.com/docs/contributing/

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/docs/integrations` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Community: Support different Pinecone initializations depending on the version
Labels: name: area: vector store, name: auto:improvement, name: integration: pinecone, name: size:M

**Description**: Adding support for different initialization implementations depending on the Pinecone client version.
**Issue**: N/A
**Dependencies**: N/A
**Twitter handle**: N/A

Tested by installing different versions of the Pinecone client and trying to create an index (both regular and serverless for the new version) from documents, and using the `Pinecone` vector store to query the index. The process ran smoothly and as expected in both cases.
Feat: add url support for UnstructuredHTMLLoader
Labels: name: area: doc loader, name: auto:enhancement, name: size:M

usage:

```python
from langchain.document_loaders import UnstructuredHTMLLoader

source_url = "https://www.sec.gov/ix?doc=/Archives/edgar/data/40987/000004098720000010/gpc-12312019x10k.htm"
loader = UnstructuredHTMLLoader(source_url)
documents = loader.load()
print(documents)

```
Issue: Passing list of index while retreiver in opensearch vectorstore
Labels: name: area: vector store, name: auto:question, name: integration: elasticsearch

### Issue you'd like to raise.

Hello Team,

let me clearify the issue below:

we are using vectordb conversational retriever. we can look into the attached code for more clarity. My issue is i have 4 documents stored in 2 different index. **End-user can select multiple document lets say 3 or 4 document, and ask questions related to any of the selected 4 document, he dont care whatever the document he selects its there on same index.** As a backend logic we can create something index document map by some logic which will give a map of documents existing in different index. 

**Now the problem here is i need to pass all the index at once where this 3-4 documents are there, there may be possibility we get more than 2 index where the selected document exist.** 

is there a way to pass multi-index while search into below function. i see index accept string argument. let me know if you suggest any workaround for this. we need solution on some better workaround then we are using.


**Below workaround we are using:**
we are creating one temp index on the go and get all the embeddings and text from all the index where the selected document exist, then use that index with query and in the same request we delete that temp index. 


Let me know in case you need any other additional details
![multi_index](https://github.com/langchain-ai/langchain/assets/34799343/a059803b-7fd9-4b37-a70e-9ed67d2665f9)


### Suggestion:

_No response_
community: allow SQLDatabase to run queries with parameters
Labels: name: auto:improvement, name: area: langserve, name: size:S

**Description:** Allow binding parameters in queries run by SQLDatabase
**Issue:** https://github.com/langchain-ai/langchain/issues/15449
**Dependencies:** N/A
**Twitter handle:** N/A

Let me know if you need me to write tests, it looks like this class doesn't have any tests.
Allow bind variables in SQLDatabase queries
Labels: name: area: doc loader, name: auto:improvement

### Feature request

Allow using [bind parameters](https://docs.sqlalchemy.org/en/20/core/connections.html#sqlalchemy.engine.Connection.execute.params.parameters) in SQLDatabase's [run method](https://github.com/langchain-ai/langchain/blob/65afc13b8b53a1ca41a1a3998dad9eb8d83ca917/libs/community/langchain_community/utilities/sql_database.py#L426).

### Motivation

I don't see a way to use bind params in SQLDatabase queries.

### Your contribution

Happy to help out with a PR if needed (and if confirmed that this functionality doesn't exist and is wanted :) )
refactored `memory`
Labels: name: area: memory, name: auto:refactor, name: size:XL

Created `langchain_community.memory`. Moved there two classes from `langchain.memory`

This movement triggered another changes:

1.
There is a lint rule that `langchain_community` code should not depend on `langchain`. Below, I show these dependencies:

```
`ZepMemory`
  --> langchain.memory.buffer.`ConversationBufferMemory`
       --> langchain.memory.chat_memory.`BaseChatMemory`
            --> langchain_community.chat_message_histories.in_memory.`ChatMessageHistory`
       --> langchain.memory.utils.`get_prompt_input_key`  # the same as in MotorheadMemory
      
`MotorheadMemory`
   --> langchain.memory.chat_memory.BaseChatMemory   # then the same dependency as in ZepMemory
        --> langchain_community.chat_message_histories.in_memory.ChatMessageHistory
   --> langchain.memory.utils.get_prompt_input_key

So, these classes and a function were moved into `core` because of the above dependencies:
- `ConversationBufferMemory`
- `BaseChatMemory`
- `ChatMessageHistory`
- `get_prompt_input_key`
```

2.
Then the next lint error forced a change in `experimental`:
mkdir -p .mypy_cache; poetry run mypy langchain_experimental --cache-dir .mypy_cache
langchain_experimental/autonomous_agents/autogpt/memory.py:3: error: Module "langchain.memory.chat_memory" has no attribute "get_prompt_input_key"  [attr-defined]

-- this was a bug in langchain_experimental/autonomous_agents/autogpt/memory.py:3 (wrong import)
refactor `utils`
Labels: name: lgtm, name: auto:refactor, name: area: models, name: size:L

The `langchain` [still holds several artifacts](https://api.python.langchain.com/en/latest/langchain_api_reference.html#module-langchain.utils) that belongs to `community`. If they moved then `langchain.utils` namespace would be  removed completely.
- moved `ernie_functions` artifacts to `community`
Accessing Gemini though Vertex AI SDK
Labels: name: auto:enhancement, name: area: models

### Feature request

Gemini API is not available in Canada, but i believe it is available through `vertexai.preview.generative_models` in pre-GA mode.

Would it be possible to add a feature using the Vertex AI SDK instead of Gemini API, which i assume is what it is using?

### Motivation

Canada access to Gemini through Langchain

### Your contribution

I could test?
docs, experimenta[patch], langchain[patch], community[patch]: update storage imports
Labels: name: area: vector store, name: auto:refactor, name: template, name: size:L

ran 

```bash
g grep -l "langchain.vectorstores" | xargs -L 1 sed -i '' "s/langchain\.vectorstores/langchain_community.vectorstores/g"
g grep -l "langchain.document_loaders" | xargs -L 1 sed -i '' "s/langchain\.document_loaders/langchain_community.document_loaders/g"
g grep -l "langchain.chat_loaders" | xargs -L 1 sed -i '' "s/langchain\.chat_loaders/langchain_community.chat_loaders/g"
g grep -l "langchain.document_transformers" | xargs -L 1 sed -i '' "s/langchain\.document_transformers/langchain_community.document_transformers/g"
g grep -l "langchain\.graphs" | xargs -L 1 sed -i '' "s/langchain\.graphs/langchain_community.graphs/g"
g grep -l "langchain\.memory\.chat_message_histories" | xargs -L 1 sed -i '' "s/langchain\.memory\.chat_message_histories/langchain_community.chat_message_histories/g"
gco master libs/langchain/tests/unit_tests/*/test_imports.py
gco master libs/langchain/tests/unit_tests/**/test_public_api.py
```