delayedkarma/langchain-issues huggingface dataset:
AsyncChromiumloader gives attribute error : COBOL
Labels: name: area: doc loader, name: auto:bug

### System Info

python 3.9.13

langchain==0.0.316
langchain-community==0.0.1
langchain-core==0.0.13


### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

import pprint

from langchain.text_splitter import RecursiveCharacterTextSplitter

from langchain_community.document_loaders import AsyncChromiumLoader
from langchain_community.document_transformers import BeautifulSoupTransformer






from langchain.chains import create_extraction_chain

from langchain_community.chat_models import ChatOpenAI

### Expected behavior

i am getting the following error when i try to import asyncchromiumloader 

Traceback (most recent call last):
  File "C:\Users\roger\OneDrive\Desktop\testlinkedinllm.py", line 5, in <module>
    from langchain_community.document_loaders import AsyncChromiumLoader
  File "C:\Users\roger\anaconda3new\lib\site-packages\langchain_community\document_loaders\__init__.py", line 51, in <module>
    from langchain_community.document_loaders.blackboard import BlackboardLoader
  File "C:\Users\roger\anaconda3new\lib\site-packages\langchain_community\document_loaders\blackboard.py", line 10, in <module>
    from langchain_community.document_loaders.pdf import PyPDFLoader
  File "C:\Users\roger\anaconda3new\lib\site-packages\langchain_community\document_loaders\pdf.py", line 18, in <module>
    from langchain_community.document_loaders.parsers.pdf import (
  File "C:\Users\roger\anaconda3new\lib\site-packages\langchain_community\document_loaders\parsers\__init__.py", line 5, in <module>
    from langchain_community.document_loaders.parsers.language import LanguageParser
  File "C:\Users\roger\anaconda3new\lib\site-packages\langchain_community\document_loaders\parsers\language\__init__.py", line 1, in <module>
    from langchain_community.document_loaders.parsers.language.language_parser import (
  File "C:\Users\roger\anaconda3new\lib\site-packages\langchain_community\document_loaders\parsers\language\language_parser.py", line 24, in <module>
    "cobol": Language.COBOL,
  File "C:\Users\roger\anaconda3new\lib\enum.py", line 429, in __getattr__
    raise AttributeError(name) from None
AttributeError: COBOL
RuntimeError when calling asyncio.run() from a running event loop
Labels: name: area: doc loader, name: auto:bug

https://github.com/langchain-ai/langchain/blob/7a93356cbc5d89cc0f7dd746d8f1bb52666fd0f1/libs/community/langchain_community/document_loaders/chromium.py#L78C40-L78C44

Hello,

I encountered a RuntimeError when running the code that uses the AsyncChromiumLoader class. The error message is asyncio.run() cannot be called from a running event loop.

Here is the relevant part of the traceback:

```python
File ~/.../.venv/lib/python3.10/site-packages/langchain_community/document_loaders/chromium.py:78, in AsyncChromiumLoader.lazy_load(self)
     77 for url in self.urls:
---> 78     html_content = asyncio.run(self.ascrape_playwright(url))
```
It seems that asyncio.run() is being called inside a running event loop, which is not allowed. This happens in the lazy_load method of the AsyncChromiumLoader class.

I think the issue could be resolved by refactoring the code to ensure that asyncio.run() is not called from a running event loop. One possible solution could be to use await instead of asyncio.run() to run the self.ascrape_playwright(url) coroutine, and then use asyncio.run() to run the lazy_load method in the main program.

Could you please look into this issue and confirm if this is a bug or if there's something I'm missing in my usage of the AsyncChromiumLoader class?

Thank you for your time and help.
The current support for Milvus in Langchain seems insufficient in my opinion. Can it be strengthened?
Labels: name: area: vector store, name: auto:improvement

### Issue you'd like to raise.

The current support for Milvus in Langchain seems insufficient in my opinion. Can it be strengthened?

### Suggestion:

The current support for Milvus in Langchain seems insufficient in my opinion. Can it be strengthened?
Issue:  Unable to use custom parser to parse the (intermediate) LLM chain output
Labels: name: area: agent, name: auto:bug

### Issue you'd like to raise.

I have been trying to use mixtral-7B as LLM agent with langchain. Agent has been provided a PythonREPL tool for any kind of code execution. 

While providing the output, it either runs into agent timeout error or else it provide wrong answer (but in correct format). By further analysis I got to know that It is trying to use the REPL tool but due to invalid input and output it is unable to.

please find the debug logs for whole LLM chain attached: 
[debug.txt](https://github.com/langchain-ai/langchain/files/13826496/debug.txt)

As per my knowledge, I may feel there is some parsing issue [here](https://github.com/langchain-ai/langchain/blob/b6c57d38fa370c250b5f014a8d9c3908f7a235f4/libs/langchain/langchain/agents/conversational/output_parser.py#L25).
It is unable to parse the agent input properly due to the occurrence of the \n post agent input in the LLM response. This is just a thought. I am open to any other suggestions as well. 

Hence, I wanted to add the custom parsing to LLM Output, So that I may able to parse Action & Action Input Accordingly. I have tried using the custom parser to achieve same, but I am unable to view any changes in the logs.

Configurations for custom parser are mentioned below:
```
from langchain.schema.output_parser import BaseLLMOutputParser

class MyOutputParser(BaseLLMOutputParser):
    def __init__(self):
        super().__init__()

    def parse_result(self, output):
        output = output.replace("Action Input:\n", "Action Input: ")
        return output
```

**Agent Configurations**:
```
from langchain.agents import AgentType, Tool, initialize_agent
from langchain.llms import OpenAI
from langchain.memory import ConversationBufferMemory
from langchain.utilities import SerpAPIWrapper

memory = ConversationBufferMemory(memory_key="chat_history",input_key="input")
con_agent = initialize_agent(
    tools,
    llm,
    agent=AgentType.CONVERSATIONAL_REACT_DESCRIPTION,
    verbose=True,
    memory=memory,
    handle_parsing_errors=True,
    output_parser=MyOutputParser(),
    agent_kwargs={'input_variables': ['input','chat_history','data'],
                                       'format_instructions': INSTRUCTIONS, 
                  'suffix': SUFFIX, 'prefix': PREFIX}
)
```

### Suggestion:

_No response_
text_splitter module not found in  langchain version 0.0.353.
Labels: name: area: doc loader, name: auto:bug

### System Info

langchain version 0.0.353.
python : 3.9


### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [X] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

<img width="685" alt="image" src="https://github.com/langchain-ai/langchain/assets/66120014/8ae76db3-ee47-4994-81a1-d587d75e27f7">

<img width="643" alt="image" src="https://github.com/langchain-ai/langchain/assets/66120014/726640b5-b7b6-4734-b6eb-5b5c50a7106c">

<img width="622" alt="image" src="https://github.com/langchain-ai/langchain/assets/66120014/011d05f2-1a6f-4fc1-abd1-08a8241c4a49">

### Expected behavior

Should be able to load the module
Issue:  Tracking Token Consumption for Async Chain Calls
Labels: name: auto:question, name: area: models

### Issue you'd like to raise.

Problem Statement:
I am currently working on tracking token consumption for asynchronous chain calls in my application. I am utilizing the AsyncIteratorCallbackHandler and its aiter() method to stream tokens to my client. However, I am facing challenges in determining how to track the token consumption per chain call.

Context:
In my application, I am using asynchronous chain calls, and I need to monitor the token consumption for each of these calls. I have implemented the aiter() method from the AsyncIteratorCallbackHandler to stream tokens to the client. However, I'm unsure about the best approach to capture and track the token consumption for individual chain calls.

Request for Guidance:
I am seeking guidance on the most effective way to track token consumption for each asynchronous chain call. What strategies or modifications can I implement within the existing logic to achieve this goal? Any insights or recommendations would be greatly appreciated.

Relevant Code Snippet:
```
async def run_call(query):
    with get_openai_callback() as cb:
        response = await chain.acall(query)
        # Rest of the code
        return response

async def create_gen(query):
    task = asyncio.create_task(run_call(query))
    try:
        async for token in handler.aiter():
            print("2. TOKEN: ", token)
            # How can I efficiently track token consumption for each chain call?
            yield f"data: {json.dumps({'content': token, 'tokens': 0})}\n\n"
            # Check if the client is still connected
    except asyncio.CancelledError:
        print("Generator canceled")
    finally:
        await task
        print("Done with task")

query = {"question": sanitized_question, "chat_history": conversation_history}
gen = create_gen(query)
return StreamingResponse(gen, media_type="text/event-stream")
```
Desired Outcome:
I aim to effectively track token consumption for each asynchronous chain call and seek advice on the most appropriate modifications or strategies to achieve this goal within the existing logic.

### Suggestion:

_No response_
langchain: Introducing Google Semantic Retriever and Attributed Question and Answering (AQA)
Labels: name: area: embeddings, name: auto:enhancement, name: size:XXL

**Description:**

Introducing two new features from Google's Generative Language API.

* Bundled embeddings calculation and vector storage for enabling semantic retrieval.
* State-of-the-art fine-tuned model for answering questions grounded in passages.

**Dependencies:**

The Python package [google-ai-generativelanguage](https://ai.google.dev/api/python/google/ai/generativelanguage)

**Twitter handle:**

https://twitter.com/googledevs
Langchain-Community LLM TextGen has wrong API endpoint
Labels: name: auto:bug, name: area: models

### System Info

Windows 10 & Ubuntu 22.04
langchain==0.0.354
langchain-community==0.0.8
langchain-core==0.1.5
Python 3.10.13


https://github.com/langchain-ai/langchain/blob/master/libs/community/langchain_community/llms/textgen.py

### Who can help?

@agola11 @hwchase17 


### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

With the latest TextGen WebUI install the API endpoints are openAI

Lanchain has the endpoints:
Example Line 213
url = f"{self.model_url}/api/v1/generate"

when used this returns a 404.

What works is 
url = f"{self.model_url}/v1/chat/completions"

A fix was attempted:  https://gist.github.com/ddomen/8eaa49879d42a4a42a243437b5ddfa83

It works for me if I set  legacy_api=False,
but his truncates responses to about 20 or so characters.

### Expected behavior

certianly not a 404.

My app that is using langchain was working with an install of TextGen fomr like amonth ago.  I went to deploy in a new environment and pulled the latest TextGen and Langchain stopped working.  When I dug  into the problem I saw they now force an OpenAI API interface.
Is The Limit Parameter Used to Retrieve Top_k?
Labels: name: area: vector store, name: auto:bug

In cookbook 3 for multimodal retrieval, `limit = 6` is set while retireving documents but the number of returned documents is always 4, regardless of the asked query or the value of `limit`. How can I retrieve `top_k` documents in this code? [Specific line is here](https://github.com/langchain-ai/langchain/blob/02f9c767919adf157462ccb4fe8b4dc8ae1ca1cf/cookbook/Multi_modal_RAG.ipynb#L633)
Can't Specify Top-K retrieved Documents in Multimodal Retrievers
Labels: name: area: vector store, name: auto:bug

### Issue you'd like to raise.

I've been playing around with the multimodal notebooks introduced in the [docs here.](https://blog.langchain.dev/semi-structured-multi-modal-rag/). However, the number of retrieved documents for every query is always 4. Specifically, for [cookbook 3](https://github.com/langchain-ai/langchain/blob/master/cookbook/Multi_modal_RAG.ipynb) the `limit` parameter is set to 6, but the number of retrieved documents is always 4 regardless of the value of `limit`. Is there another way to get `top_k`?

### Suggestion:

_No response_
ImportError: cannot import name 'tracing_enabled' from 'langchain_core.tracers.context'
Labels: name: auto:bug, name: area: models, name: integration: aws

### System Info

Full traceback:

```
File "/src/app.py", line 9, in <module>
    from langchain.chains import ConversationalRetrievalChain
  File "/venv/lib/python3.11/site-packages/langchain/chains/__init__.py", line 20, in <module>
    from langchain.chains.api.base import APIChain
  File "/venv/lib/python3.11/site-packages/langchain/chains/api/base.py", line 11, in <module>
    from langchain.callbacks.manager import (
  File "/venv/lib/python3.11/site-packages/langchain/callbacks/__init__.py", line [45] in <module>
    from langchain_core.tracers.context import (
ImportError: cannot import name 'tracing_enabled' from 'langchain_core.tracers.context' (/venv/lib/python3.11/site-packages/langchain_core/tracers/context.py)
```

### Who can help?

@hwchase17 @agola

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

This issue seems to originate from the import:

```
from langchain.chains import ConversationalRetrievalChain
```

### Expected behavior

The modules should import successfully.
Embeddings - Persist between batches
Labels: name: area: embeddings, name: auto:improvement, name: integration: chroma

### Feature request

Able to `persist` between batch when the embedding is between built:

```python
db = Chroma.from_documents(
        documents=documents, embedding=embeddings, persist_directory=persist_directory)
    db.persist()
    return db
```

would be nice to be :

```python
db = Chroma.from_documents(
        documents=documents, embedding=embeddings, persist_directory=persist_directory,
        batch_size=batch_size, persist_between_batches=2
)
    db.persist()
    return db
```

to specify a batch size and persist between every `2` batches.

### Motivation

My computer is slow and now that embedding has started, I am afraid it might crash and if it does, I will have to start all over again. 

### Your contribution

Not really sorry
openai[minor]: implement langchain-openai package
Labels: name: area: embeddings, name: auto:enhancement, name: size:XXL, name: partner

Todo

- [x] copy over integration tests
- [x] update docs with new instructions in #15513 
- [x] add linear ticket to bump core -> community, community->langchain, and core->openai deps
- [ ] (optional): add `pip install langchain-openai` command to each notebook using it
- [x] Update docstrings to not need `openai` install
- [x] Add serialization
- [x] deprecate old models

Contributor steps:

- [x] Add secret names to manual integrations workflow in .github/workflows/_integration_test.yml
- [x] Add secrets to release workflow (for pre-release testing) in .github/workflows/_release.yml

Maintainer steps (Contributors should not do these):

- [x] set up pypi and test pypi projects
- [x] add credential secrets to Github Actions
- [ ] add package to conda-forge


Functional changes to existing classes:

- now relies on openai client v1 (1.6.1) via concrete dep in langchain-openai package

Codebase organization

- some function calling stuff moved to `langchain_core.utils.function_calling` in order to be used in both community and langchain-openai
Fix HuggingFaceHub LLM Integration
Labels: name: auto:improvement, name: area: models

### System Info

0.0.354

### Who can help?

@hwchase17 @agol

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

1. replace HuggingFaceHub InferenceAPI to InferenceClient
2. replace max_length to max_new_tokens

PR: #

Reference: [https://huggingface.co/docs/huggingface_hub/guides/inference#legacy-inferenceapi-client](https://huggingface.co/docs/huggingface_hub/guides/inference#legacy-inferenceapi-client)

### Expected behavior

PR #15498 address this issues
[Warning Fixed] HuggingFace Hub LLM Integration
Labels: name: auto:bug, name: area: models, name: size:S

 This PR fixes the following warning
```
/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_deprecation.py:131: FutureWarning: 'InferenceApi' (from 'huggingface_hub.inference_api') is deprecated and will be removed from version '1.0'. `InferenceApi` client is deprecated in favor of the more feature-complete `InferenceClient`. Check out this guide to learn how to convert your script to use it: https://huggingface.co/docs/huggingface_hub/guides/inference#legacy-inferenceapi-client.
  warnings.warn(warning_message, FutureWarning)
```

Fixes made:
- Replace `InferenceAPI` with `InferenceClient`
- Replace `max_length` to `max_new_tokens` as per new HuggingFace docs
PGVector method for HNSW
Labels: name: area: vector store, name: auto:enhancement

### Feature request

Provide a method to create the HNSW for the PGVector vectorstore

### Motivation

There is a similar method implemented for PGEmbedding but the embedding extension will be deprecated

### Your contribution

https://github.com/pgvector/pgvector?tab=readme-ov-file#hnsw

https://github.com/langchain-ai/langchain/blob/6e90b7a91bba16d84689d07d1016a941eddf4f64/libs/community/langchain_community/vectorstores/pgembedding.py#L184-L212
Issue: No connection adaptors were found
Labels: name: auto:bug, name: area: models

### Issue you'd like to raise.

Hi all, 

I am trying to create an app that can act upon a natural language prompt to interact with the Monday API and then successfully carry out the relevant action.

The code I currently have is as follows:

> MondayDocs =  """
> 
> 
> The below is Monday  API documentation showing how to create an item. 
> 
> The endpoint is: 
> 
> "https://api.monday.com/v2". 
> 
> ///Example to create a new item:
> 
> 
> mutation_query = '''
>     mutation {
>         create_item (board_id: id, item_name: "itemname") {
>             id
>         }
>     }
> '''
> 
>
> data = {'query': mutation_query}
> 
> 
> response = requests.post(url=apiUrl, json=data, headers=headers)
> 
> 
> Use the below authorisation:
> 
> headers = {"Authorization": "X"}
> 
> 
> """
> 
> 
> llm = ChatOpenAI(temperature=0, model= 'gpt-3.5-turbo-1106', openai_api_key="Y")
> 
> Test = APIChain.from_llm_and_api_docs(llm, MondayDocs,limit_to_domains=None, verbose=True)
> 
> Test.run("Create a new item named James.")
> 

However, when I run this, I get the error message: 

> No connection adapters were found 

How can I fix this?


### Suggestion:

_No response_
fix: Ollama import statements
Labels: name: lgtm, name: auto:documentation, name: size:XS

<!-- Thank you for contributing to LangChain!

Please title your PR "<package>: <description>", where <package> is whichever of langchain, community, core, experimental, etc. is being modified.

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes if applicable,
  - **Dependencies:** any dependencies required for this change,
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` from the root of the package you've modified to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://python.langchain.com/docs/contributing/

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/docs/integrations` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
community: changing the .dict() to .model_dump() since it is deprecated
Labels: name: area: embeddings, name: auto:improvement, name: size:XS

  - **Description:** The method `.dict()` was replaced by `.model_dump` since its deprecated in Pydantic V2 and will be removed in Pydantic V.3, 
  - **Issue:** None,
  - **Dependencies:** None,
  - **Twitter handle:** @hhvedoveli

<!-- Thank you for contributing to LangChain!

Please title your PR "<package>: <description>", where <package> is whichever of langchain, community, core, experimental, etc. is being modified.

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes if applicable,
  - **Dependencies:** any dependencies required for this change,
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` from the root of the package you've modified to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: https://python.langchain.com/docs/contributing/

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/docs/integrations` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
HELP!: Example of using HuggingFaceTextGenInference, llmchain, and fastapi
Labels: name: auto:question, name: area: models

### Issue you'd like to raise.

I've scoured the internet trying to find an example that would use a custom model (Mistral) with ```HuggingFaceTextGenInference``` which uses ```LLMChain``` to return a **streaming** response via ```fastapi```. 

Does anyone have a working example?

### Suggestion:

_No response_