delayedkarma/langchain-issues huggingface dataset:
CosmosDBHistoryMessage
Labels: name: area: memory, name: auto:bug, name: area: models

### System Info

CosmosDBHistory messages are wiping out the session of each run. 
Need the library to not recreate the object for the same session or else this cannot be used as chatbot which is stateless


### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [X] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

`import os
from time import perf_counter

from langchain.chat_models import AzureChatOpenAI
from langchain.memory.chat_message_histories import CosmosDBChatMessageHistory
from langchain.schema import (
    SystemMessage,
    HumanMessage
)

from logger_setup import logger



llm = AzureChatOpenAI(
    deployment_name=os.getenv('OPENAI_GPT4_DEPLOYMENT_NAME'),
    model=os.getenv('OPENAI_GPT4_MODEL_NAME')
)



cosmos_history = CosmosDBChatMessageHistory(
    cosmos_endpoint=os.getenv('AZ_COSMOS_ENDPOINT'),
    cosmos_database=os.getenv('AZ_COSMOS_DATABASE'),
    cosmos_container=os.getenv('AZ_COSMOS_CONTAINER'),
    session_id='1234',
    user_id='user001',
    connection_string=os.getenv('AZ_COSMOS_CS')
)


# cosmos_history.prepare_cosmos()


sys_msg = SystemMessage(content='You are a helpful bot that can run various SQL queries')
# cosmos_history.add_message(sys_msg)

human_msg = HumanMessage(content='Can you tell me how things are done in database')
cosmos_history.add_message(human_msg)



messages = []

messages.append(sys_msg)
messages.append(human_msg)

start_time = perf_counter()
response = llm.predict_messages(messages=messages)
end_time = perf_counter()

logger.info('Total time taken %d s', (end_time - start_time))

print(response)

messages.append(response)

cosmos_history.add_message(response)

`

### Expected behavior

Need this to save it for each subsequent web service calls with the same session id


Also this code does not retrieve data 
try:
    logger.info(f"Reading item with session_id: {self.session_id}, user_id: {self.user_id}")
    item = self._container.read_item(
        item=self.session_id,
        partition_key=self.user_id
    )
except CosmosHttpResponseError as ex:
    logger.error(f"Error reading item from CosmosDB: {ex}")
    return


But using sql does
query = f"SELECT * FROM c WHERE c.id = '{self.session_id}' AND c.user_id = '{self.user_id}'"
items = list(self._container.query_items(query, enable_cross_partition_query=True))

if items:
    item = items[0]
    # Continue with reading the item
else:
    logger.info("Item not found in CosmosDB")
    return
Add some properties to NotionDBLoader
Labels: name: lgtm, name: area: doc loader, name: auto:improvement

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->

fix #13356

Add supports following properties for metadata to NotionDBLoader.

- `checkbox`
- `email`
- `number`
- `select`

There are no relevant tests for this code to be updated.
Issue: Notion DB loader for doesn't supports some properties 
Labels: name: area: doc loader, name: auto:improvement

### Issue you'd like to raise.

notion page properties
https://developers.notion.com/reference/page-property-values

Current version Notion DB loader for  doesn't supports following properties for metadata

- `checkbox`
- `email`
- `number`
- `select`

### Suggestion:

I would like to make a PR to fix this issue if it's okay.
AsyncHtmlLoader update due to fetching errors in not available webpages
Labels: name: area: doc loader, name: auto:bug

  - **Description:** When running the `qa_chain` with the `WebResearchRetriever` class as a retriever, the code would throw an error even with other possible/valid sources. This would happen when there was no access/connection to an URL.
  - **Issue:** None,
  - **Dependencies:** None,

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
Ability to suppress warning when supplying a "not default parameter" to OpenAI Chat Models
Labels: name: auto:improvement, name: area: models

### Feature request

Supplying a not default parameter to a model is not something that will require always showing a warning. It should be easy to suppress that specific warning. 

### Motivation

The logging gets full of those warnings just because you are supplying not default parameters such as "top_p", "frequency_penalty" or "presence_penalty".
Issue: <Please write a comprehensive title after the 'Issue: ' prefix>
Labels: name: area: memory, name: auto:bug, name: area: models

### Issue you'd like to raise.

I have my code and for the first section of it I am inputting a memory key based in the session_id as a UUID
I am receiving the following error: Any idea what the issue is?

> Traceback (most recent call last):
>   File "/Users/habuhassan004/Desktop/VIK/10x-csv-chat/llm.py", line 62, in <module>
>     print(chat_csv(session_id,None,'Hi how are you?'))
>           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>   File "/Users/habuhassan004/Desktop/VIK/10x-csv-chat/llm.py", line 37, in chat_csv
>     response = conversation({"question": question})
>                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>   File "/Users/habuhassan004/Desktop/VIK/10x-csv-chat/venv/lib/python3.11/site-packages/langchain/chains/base.py", line 286, in __call__
>     inputs = self.prep_inputs(inputs)
>              ^^^^^^^^^^^^^^^^^^^^^^^^
>   File "/Users/habuhassan004/Desktop/VIK/10x-csv-chat/venv/lib/python3.11/site-packages/langchain/chains/base.py", line 443, in prep_inputs
>     self._validate_inputs(inputs)
>   File "/Users/habuhassan004/Desktop/VIK/10x-csv-chat/venv/lib/python3.11/site-packages/langchain/chains/base.py", line 195, in _validate_inputs
>     raise ValueError(f"Missing some input keys: {missing_keys}")
> ValueError: Missing some input keys: {'session_id'}

here is the input:
`session_id = uuid.uuid4()
print(chat_csv(session_id,None,'Hi how are you?'))`

Here is the code:
```
def chat_csv(
    session_id: UUID = None,
    file_path: str = None,
    question: str = None
):
    # session_id = str(session_id)
    memory = ConversationBufferMemory(
        memory_key=str(session_id),
        return_messages=True
    )
    if file_path == None:
        template = """You are a nice chatbot having a conversation with a human.
            Previous conversation:
            {session_id}

            New human question: {question}
            Response:"""
        prompt = PromptTemplate.from_template(template)
        conversation = LLMChain(
            llm=OpenAI(temperature=0),
            prompt=prompt,
            verbose=False,
            memory=memory
        )
        response = conversation({"question": question})
```

### Suggestion:

_No response_
Make it easier to subclass RunnableEach
Labels: name: auto:improvement

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Issue: <Please write a comprehensive title after the 'Issue: ' prefix>
Labels: name: auto:question, name: area: models

### Issue you'd like to raise.

```
llm = ChatOpenAI(
    openai_api_key= os.environ["OPENAI_API_KEY"],
    temperature=0, 
    streaming=True, 
    callbacks=[AsyncIteratorCallbackHandler()]
)
embeddings = OpenAIEmbeddings()

loader = TextLoader("static/faq/ecommerce_faq.txt")
documents = loader.load()
text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)
texts = text_splitter.split_documents(documents)

docsearch = Chroma.from_documents(texts, embeddings)

faq_chain = RetrievalQA.from_chain_type(
    llm, 
    chain_type="stuff", 
    retriever=docsearch.as_retriever(),
)

async def call_openai(question):

    callback = AsyncIteratorCallbackHandler()

    # coroutine = wait_done(model.agenerate(messages=[[HumanMessage(content=question)]]), callback.done)

    coroutine = wait_done(faq_chain.arun(question), callback.done)

    task = asyncio.create_task(coroutine)

    async for token in callback.aiter():
        print('token', token)
        yield f"data: {token}\n\n"

    await task

app = FastAPI()

@app.get("/")
async def homepage():
    return FileResponse('static/index.html')

@app.post("/ask")
def ask(body: dict):
    return StreamingResponse(call_openai(body['question']), media_type="text/event-stream")

if __name__ == "__main__":
   uvicorn.run(host="127.0.0.1", port=8888, app=app)

```
I can run it normally using model.agenerate, but it cannot run after using faq_chain.arun. what is the reason for this?



### Suggestion:

_No response_
PowerPoint loader crashing
Labels: name: area: doc loader, name: auto:bug

### System Info

Langchain Version: 0.0.335
Platform: Windows 10
Python Version = 3.11.3
IDE: VS Code

### Who can help?

@eyurtsev 

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [X] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

MWE:
A PowerPoint presentation named "test.pptx" in the same folder as the script. Content is a title slide with sample text.

```python
from langchain.document_loaders import UnstructuredPowerPointLoader
def ingest_docs():
  loader= UnstructuredPowerPointLoader('test.pptx')
  docs = loader.load()
  return docs
```

I get a problem in 2/3 tested environments:

1. Running the above MWE with `ingest_docs()` in a simple python script will yield no problem. The content of the PowerPoint (text on the title slide) is displayed.
2. Running the above MWE in a Jupyter Notebook with `ingest_docs()` will cause the cell to run indefinetely. Trying to interrupt the kernel results in: `Interrupting the kernel timed out`. A fix is to restart the kernel.
3. Running the MWE in Streamlit (see code below) will result the spawned server to die immediately. (The cmd Window simply closes)
```python
import streamlit as st
from langchain.document_loaders import UnstructuredPowerPointLoader
def ingest_docs():
  [as above]

st.write(ingest_docs())
```

### Expected behavior

I expect the MWE to work the same in the Notebook and Streamlit environments just as in the simple Python script.
Why no constructor (init function) in Langchain while pydantics are everywhere?
Labels: name: auto:question

### Issue you'd like to raise.

Hi all. Can anyone help to answer why there's no init function (constructor) for classes in LangChain, while Pydantic occurs everywhere. It seems hard for IDEs to jump to related codes. Is that one kind of anti-pattern of Python programming?

### Suggestion:

_No response_
add more powerfull clean html function
Labels: name: auto:improvement

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** I add a function to scape html the default transform function can't scape my webside data and this worked function , 
  - **Issue:** Can't scape webside data my webside ["https://www.presight.io/privacy-policy.html"]# it fixed
  - **Dependencies:** BeautifulSoup,NavigableString from bs4 and re,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** Yes

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
DOC: Wrong documentation link
Labels: name: auto:documentation

### Issue with current documentation:

Hi all, on main website [langchain.com](https://www.langchain.com/), above the folder, to the right of hero section both boxes (python and js) point to the [python docs](https://python.langchain.com/docs/get_started/introduction). 

### Idea or request for content:

It's better and clearer if each block points to the related lang. Of course, it's not urgent or critical :)
docs: Install langsmith from conda-forge
Labels: name: lgtm, name: auto:documentation

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->

langsmith is available on conda-forge as well and also a dependency of the package so it gets installed either way by conda https://github.com/conda-forge/langchain-feedstock/blob/306ed13308c051716ced37a28962a0ac80a9aa7f/recipe/meta.yaml#L43
Add POST API Functionality to LangChain
Labels: name: auto:enhancement

### Feature request

LangChain supports GET functions, but there is no support for POST functions. This feature request proposes the addition of POST API functionality to enhance the capabilities of LangChain.

### Motivation

The motivation behind this feature request is to extend the capabilities of LangChain to handle not only GET requests but also POST requests. 

### Your contribution

I am willing to contribute to the development of this feature. I will carefully follow the contributing guidelines and provide a pull request to implement the POST API functionality.
Reduce embeddings size
Labels: name: area: embeddings, name: area: vector store, name: auto:bug, name: auto:question, name: integration: elasticsearch

### Issue you'd like to raise.

I'm trying to replicate [this example](https://python.langchain.com/docs/integrations/vectorstores/elasticsearch#basic-example) of langchain. I'm using ElasticSearch as the database to store the embedding. In the given example I have replaced `embeddings = OpenAIEmbeddings()` with `embeddings = OllamaEmbeddings(model="llama2")` which one can import `from langchain.embeddings import OllamaEmbeddings`. I'm running `Ollama` locally. But, I'm running into below error:
```
raise HTTP_EXCEPTIONS.get(status_code, TransportError)(
elasticsearch.exceptions.RequestError: RequestError(400, 'mapper_parsing_exception', 'The number of dimensions for field [vector] should be in the range [1, 2048] but was [4096]')
```
The Ollama model always create the embedding of size `4096` even when I set the chunk size of `500`. Is there any way to reduce the size of embedding? or is there anyway to store larger size embeddings in `ElasticSearch`

### Suggestion:

_No response_
Add missing doc key to metadata field in AzureSearch Vectorstore
Labels: name: lgtm, name: area: vector store, name: auto:bug, name: size:M

  - **Description:** Adds doc key to metadata field when adding document to Azure Search.
  - **Issue:** -,
  - **Dependencies:** -,
  - **Tag maintainer:** @eyurtsev,
  - **Twitter handle:** @finnless

Right now the document key with the name FIELDS_ID is not included in the FIELDS_METADATA field, and therefore is not included in the Document returned from a query. This is really annoying if you want to be able to modify that item in the vectorstore.

Other's thoughts on this are welcome.
Enhancing iMessageChatLoader to prevent skipping messages with extractable content
Labels: name: area: doc loader, name: auto:improvement

### Feature request

Implement extraction of message content whenever iMessageChatLoader skips messages with null `text` field even though the content is present in other fields.

### Motivation

Due to iMessage's database schema change in its new MacOS Ventura, newer messages now have content encoded in the `attributedBody` body, with the `text` field being null.

This issue was originally raised by @idvorkin in Issue #10680.

### Your contribution

We intend to submit a pull request for this issue close to the end of November.
Update the imports for various exceptions in the ChatOpenAI class to be consistent with OpenAI API
Labels: name: auto:nit, name: area: models

- **Description**: Update the imports for various exceptions in the ChatOpenAI class to be consistent with OpenAI API.
- **Issue**: https://github.com/langchain-ai/langchain/issues/13323,
- **Dependencies**: openai==1.2.3
- **Tag** maintainer: @hwchase17
OpenAI exception classes have different import path in openai 1.2.3, causing breaking change in ChatOpenAI - Simple fix
Labels: name: auto:bug, name: area: models

### System Info

- Langchain version: langchain==0.0.335
- OpenAI version: openai==1.2.3
- Platform: Darwin MacBook Pro 22.4.0 Darwin Kernel Version 22.4.0: Mon Mar  6 21:00:41 PST 2023; root:xnu-8796.101.5~3/RELEASE_ARM64_T8103 arm64
- Python version: Python 3.9.6

### Who can help?

@hwchase17

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Example script:

```python
from langchain.chat_models import ChatOpenAI
llm = ChatOpenAI()
user_input = input("Input prompt: ")
response = llm.invoke(user_input)
```

```python
    return self._generate(
  File "...lib/python3.9/site-packages/langchain/chat_models/openai.py", line 360, in _generate
    response = self.completion_with_retry(
  File "...lib/python3.9/site-packages/langchain/chat_models/openai.py", line 293, in completion_with_retry
    retry_decorator = _create_retry_decorator(self, run_manager=run_manager)
  File "...lib/python3.9/site-packages/langchain/chat_models/openai.py", line 73, in _create_retry_decorator
    openai.error.Timeout,

Exception module 'openai' has no attribute 'error'
```

Cause appears to be in `langchain.chat_models.openai`: https://github.com/langchain-ai/langchain/blob/5a920e14c06735441a9ea28c1313f8bd433dc721/libs/langchain/langchain/chat_models/openai.py#L82-L88

Modifying above to this appears to resolve the problem:

```python
  errors = [
      openai.Timeout,
      openai.APIError,
      openai.APIConnectionError,
      openai.RateLimitError,
      openai.ServiceUnavailableError,
]
```

### Expected behavior

Chat model invocation should return a BaseMessage.
Added llama2-13b-chat-v1 on bedrock
Labels: name: auto:enhancement, name: area: models

  - **Description:** Added `llama2-13b-chat-v1` for Bedrock client
  - **Issue:** #13316
  - **Dependencies:** None

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.