delayedkarma/langchain-issues huggingface dataset:
Fix documentation typo in Clickhouse Class
Labels: name: lgtm, name: area: vector store, name: auto:documentation

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
  - **Description:**  The return info in the documentation for similarity_search_by_vector and similarity_search_with_relevance_scores is wrong
Tools for Dictionary APIs
Labels: name: area: agent, name: auto:enhancement

### Feature request

It would be nice to have agents that could access dictionary APIs such as the Merriam-Webster API or Urban Dictionary API (for slang).

### Motivation

It can be useful to be able to look up definitions for words using a dictionary to provide additional context. With no current dictionary tools available, it would be beneficial for there to be an implemented dictionary tool available at all.

### Your contribution

We will open a PR that adds a new tool for accessing the Merriam-Webster Collegiate Dictionary API (https://dictionaryapi.com/products/api-collegiate-dictionary[/](https://www.dictionaryapi.com/)), which provides definitions for English words, as soon as possible. In the future this could be extended to support other Merriam-Webster APIs such as their Medical Dictionary API (https://dictionaryapi.com/products/api-medical-dictionary) or Spanish-English Dictionary API (https://dictionaryapi.com/products/api-spanish-dictionary).

We may also open another PR for Urban Dictionary API integration.
nc/runnable-dynamic-schemas-from-config
Labels: name: auto:improvement

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Revert "nc/runnable-dynamic-schemas-from-config"
Labels: name: auto:refactor

This reverts commit a46eef64a78edcbfb15d13165224b4574b668c18.

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Bedrock chain not working with AsyncCallbackHandler
Labels: name: auto:bug, name: area: models, name: integration: pinecone

### System Info

Name: langchain
Version: 0.0.317

### Who can help?

_No response_

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [X] Chains
- [X] Callbacks/Tracing
- [X] Async

### Reproduction

Hello @agola11
I got this runtime warning:
RuntimeWarning: coroutine 'AsyncCallbackManagerForLLMRun.on_llm_new_token' was never awaited
  run_manager.on_llm_new_token(chunk.text, chunk=chunk)

I try to stream over a websocket the generated tokens. When I try to add a AsyncCallbackHandler to manage this streaming and run acall the warning occurs and nothing is streamed out.

class StreamingLLMCallbackHandler(AsyncCallbackHandler):

    def __init__(self, websocket):
        self.websocket = websocket
        self.intermediate_result = []

    async def on_llm_new_token(self, token: str, **kwargs: Any) -> None:
        self.intermediate_result.append(token)
        await self.websocket.send_text(token)

    async def on_llm_end(self, token: str, **kwargs: Any) -> None:
        await self.websocket.send_text("[END]")

stream_handler = StreamingLLMCallbackHandler()

model_kwargs = {
        "max_tokens_to_sample": 8000,
        "temperature": 0.7,
        # "top_k": 250,
        # "top_p": 1,
        "stop_sequences" : ['STOP_LLM']
        }

llm = Bedrock(
        client=bedrock,
        model_id="anthropic.claude-v2",
        # provider_stop_sequence_key_name_map={'anthropic': 'stop_sequences'},
        streaming=True,
        callbacks=[stream_handler],
        model_kwargs=model_kwargs
        )

prompt_template = f"""
Human:
{{context}} {{question}}
Assistant:"""


PROMPT = PromptTemplate(template=prompt_template, input_variables=["context", "question"])
chain_type_kwargs = {"prompt": PROMPT}

pinecone.init(api_key=PINECONE_API_KEY, environment=PINECONE_API_ENV)
chat_vectorstore = Pinecone.from_existing_index(index_name='intelligencechat1', embedding= OpenAIEmbeddings(openai_api_key= OPENAI_API_KEY), namespace='146')

chain = RetrievalQA.from_chain_type(
            llm=llm,
            chain_type="stuff",
            return_source_documents=True,
            chain_type_kwargs=chain_type_kwargs,
            retriever=chat_vectorstore.as_retriever(search_kwargs={'k' : 4}),)

query = 'summarize'
result = await chain.acall(({"query": query}))

### Expected behavior

The expected behavior is that each token is streamed sequentially over the websocket.
the chain work in syncronous mode without 'acall'
Add default value
Labels: name: auto:nit

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
add _acall method to YandexGPT
Labels: name: auto:improvement, name: area: models

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
  - **Description:** Add async support for YandexGPT LLM model
Fix langsmith walkthrough doc dataset
Labels: name: auto:documentation

- **Description:** the walkthrough notebook for langsmith created a dataset with an unequal number of input questions and reference answers.
- **Issue:** when the dataset was created with a python `zip` iteration, an off-by-one error was created which meant the last few questions had incorrect reference answers
- **Dependencies:** none
- **Tag maintainer:** Not sure
- **Twitter handle:** none
QA Privacy Nit
Labels: name: auto:documentation

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
how can i get only 1 or 2 words output from my llm?
Labels: name: auto:enhancement, name: area: models

### Feature request

We want some output parser or feature that will restrict llm to generate specifi number of words.in json or any format.
some times user want 10 lines output,sometime only 2 words etc
so this is very helpful feature .thanks


### Motivation

i want to build the next word auto suggetion model using llm,but my llm is giving me crazy output every time
instead of giving only 1 words


example: my input is
input: what is 
output : your name

input: good
output: morning

input: what is the meaninig
output: of


so it will give me only 1 word suggetion,insted of giving all other stuff. want to restrict llm for specific words.

### Your contribution

will help with promt.but few quantised llm ar not good with giving only one word
Elasticsearch Query Retriever: Use match + fuzziness for LIKE
Labels: name: area: vector store, name: auto:improvement, name: integration: elasticsearch

Updated the elasticsearch self query retriever to use the match clause for LIKE operator instead of the non-analyzed fuzzy search clause.

Other small updates include:
- fixing the stack inference integration test where the index's default pipeline didn't use the inference pipeline created
- adding a user-agent to the old implementation to track usage
- improved the documentation for ElasticsearchStore filters
Add Tencent Hunyuan chat model
Labels: name: auto:enhancement, name: area: models

### Description:
The Tencent Hunyuan model, developed by Tencent, is a large language model by robust Chinese text generation capabilities, adeptness in logical reasoning within complex contexts, and reliable task execution proficiency.For more information, see [https://cloud.tencent.com/document/product/1729](https://cloud.tencent.com/document/product/1729)
self._scrape in lazy_load method is not taken any parameters except path given instantiating WebBaseLoader
Labels: name: area: doc loader, name: auto:bug

### System Info

langchain 0.0.312
Python 3.11.6

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [X] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

```python
from bs4 import SoupStrainer
from langchain.document_loaders import WebBaseLoader
from langchain.text_splitter import CharacterTextSplitter
def load_html():
  only_body = SoupStrainer('body')
  loader = WebBaseLoader(['https://example.com/'],  bs_kwargs={'parse_only': only_body})
  docs = loader.load()
  
  text_splitter = CharacterTextSplitter(
    separator = "\n",
    chunk_size = 300,
    chunk_overlap = 50,
    length_function = len,
  )
  print(text_splitter.split_documents(docs))
  # -> get all texts in html, not filtered by bs_kwargs passed in WebBaseLoader
```

### Expected behavior

expected filtered texts passed by parse_only in bs_kwargs when instantiate WebBaseLoader.

https://github.com/langchain-ai/langchain/blob/12f8e87a0e89a8ff50fc7dbab612ac6770f3d258/libs/langchain/langchain/document_loaders/web_base.py#L245

in lazy_load method, self._scrape is called with path but not other parameters
fix: superfluous List Parser doc
Labels: name: auto:documentation

Replace this entire comment with:
  - **Description:** This PR fixes the superfluous [List Parser documentation](https://python.langchain.com/docs/modules/model_io/output_parsers/comma_separated) that unnecessary modules are imported inside the example, 
  - **Issue:** N/A,
  - **Dependencies:** N/A,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** @dosuken123 

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
Issue:  How to Automatically Generate Embeddings for Updated Documents in a Confluence Space and Enable Real-Time Question Answering on the Updated Data?
Labels: name: area: embeddings, name: area: doc loader, name: area: vector store, name: auto:question, name: auto:enhancement, name: integration: chroma

### Issue you'd like to raise.

I'm currently working on accessing a Confluence space using Langchain and performing question answering on its data. The embeddings of this data are stored in a Chromadb vector database once I provide user name,API keyand Space key.
However, I'm looking for a way to automatically generate embeddings for any documents that change in real-time within the Confluence space and enable real-time question answering on the updated data. Any suggestions or solutions on how to achieve this would be greatly appreciated!

### Suggestion:

_No response_
Issue:doubt about  generating embeddings for both PDF documents and Confluence content and storing them in a single 'embeddings' folder
Labels: name: area: embeddings, name: area: doc loader, name: area: vector store, name: auto:question

### Issue you'd like to raise.

    **Below code is for generating embeddings from pdf**
    loader = PyPDFLoader(f"{file_path}")
    document = loader.load()
    text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=0)
    texts = text_splitter.split_documents(documents=document)
    embedding = OpenAIEmbeddings()
    
    **Below code is for generating embeddings from confluence**
        embedding = OpenAIEmbeddings()
    loader = ConfluenceLoader(
        url=confluence_url,
        username=username,
        api_key=api_key
    )
    for space_key in space_key:
        documents.extend(loader.load(space_key=space_key,limit=100))
    # Split the texts
    text_splitter = CharacterTextSplitter(chunk_size=6000, chunk_overlap=0)
    texts = text_splitter.split_documents(documents)
    text_splitter = TokenTextSplitter(chunk_size=1000, chunk_overlap=10, encoding_name="cl100k_base")
    texts = text_splitter.split_documents(texts)
    
I would like to inquire about the feasibility of generating embeddings for both PDF documents and Confluence content and storing them in a single 'embeddings' folder. This would allow us to have the flexibility to perform question answering from either Confluence or multiple PDF sources without switching between different folders. Can you provide guidance on how to achieve this integrated storage approach?

### Suggestion:

_No response_
TypeError: UnstructuredYoloXModel.initialize() got an unexpected keyword argument 'extract_images_in_pdf' while running Semi_structured_multi_modal_RAG_LLaMA2.ipynb
Labels: name: area: vector store, name: auto:bug

### System Info

Hi, I am facing an issue when attempting to run the "Semi_structured_multi_modal_RAG_LLaMA2.ipynb" notebook from the cookbook.
![image](https://github.com/langchain-ai/langchain/assets/91419837/0a29dd3e-40a1-4264-8e07-bf0edc63b882)
**Environment Details**
Langchain Version: 0.0.317
I would appreciate any assistance in resolving this issue. Thank you.

### Who can help?

_No response_

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

TypeError: UnstructuredYoloXModel.initialize() got an unexpected keyword argument 'extract_images_in_pdf'

### Expected behavior

The notebook should run without any issues and produce the expected output as documented in the cookbook
KayAiRetriever: without metadata filtering, wrong results
Labels: name: area: vector store, name: auto:bug

### System Info

Hello! Even though [API](https://api.python.langchain.com/en/latest/retrievers/langchain.retrievers.kay.KayAiRetriever.html) mentions a metadata param, it's not found in [code](https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/retrievers/kay.py#L9). 

Without metadata filtering, querying “Tell me about the returns of Palantir Technologies Inc.” returns docs with 'company_name': 'ETSY INC'.

Thank you

### Who can help?

@eyurtsev? 

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

from langchain.retrievers import KayAiRetriever
retriever = KayAiRetriever.create(dataset_id="company", data_types=["10-K"], num_contexts=1)
retriever.get_relevant_documents(“Tell me about the returns of Palantir Technologies Inc.?")


### Expected behavior

[Document(page_content="Company Name: ETSY INC \n Company Industry: SERVICES-BUSINESS SERVICES, NEC \n Form Title: 10-K 2020-FY \n Form Section: Risk Factors \n Text: Any events causing ... abroad.", metadata={'chunk_type': 'text', 'chunk_years_mentioned': [], 'company_name': 'PALANTIR TECHNOLOGIES INC', 'company_sic_code_description': 'SERVICES-BUSINESS SERVICES, NEC', 'data_source': '10-K', 'data_source_link': 'https://www.sec.gov/Archives/edgar/data/1370637/000137063721000012', 'data_source_publish_date': '2020-01-01T00:00:00Z', 'data_source_uid': '0001370637-21-000012', 'title': 'ETSY INC |  10-K 2020-FY '})]