delayedkarma/langchain-issues huggingface dataset:
Support for CPP in LangaugeParser is not complete
Labels: name: area: doc loader, name: auto:improvement, name: stale

### System Info

Langchain v0.0.225

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [X] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

from langchain.text_splitter import Language
from langchain.document_loaders.generic import GenericLoader
from langchain.document_loaders.parsers import LanguageParser

loader = GenericLoader.from_filesystem(
    repo_path,
    glob="**/*",
    suffixes=[".h", ".c"],
    parser=LanguageParser(language=Language.CPP)
)



File ~/ML/llama-venv/lib64/python3.11/site-packages/langchain/document_loaders/parsers/language/language_parser.py:116, in LanguageParser.lazy_parse(self, blob)
    107     yield Document(
    108         page_content=code,
    109         metadata={
   (...)
    112         },
    113     )
    114     return
--> 116 self.Segmenter = LANGUAGE_SEGMENTERS[language]
    117 segmenter = self.Segmenter(blob.as_string())
    118 if not segmenter.is_valid():

**KeyError: <Language.CPP: 'cpp'>**

### Expected behavior

Currently only JS and Python are supported in LANGUAGE_SEGMENTERS and the ask is to support CPP as well.
Issue: TypeError: request() got an unexpected keyword argument 'vectors' when using AzureSearchVectorStore
Labels: name: area: vector store, name: auto:bug

I'm encountering an error when running the code snippet below. The error doesn't occur when I use AzureCognitiveSearchRetriever as the retriever, but persists with `retriever2 = AzureSearchVectorStoreRetriever(vectorstore=vector_store, search_type="hybrid", k=8)`.  

Appreciate the help, thank you!

```
chain35 = RetrievalQAWithSourcesChain.from_chain_type(
    model, 
    chain_type="stuff", 
    retriever=retriever2,
    chain_type_kwargs=chain_type_kwargs,
    max_tokens_limit=7000,
    reduce_k_below_max_tokens=True,
    return_source_documents=True
    )

with get_openai_callback() as cb:
    answer = chain35({"question": query}, return_only_outputs=True)
```

### Suggestion:

_No response_
docs: add vLLM chat notebook
Labels: name: auto:documentation, name: area: models

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->

This PR aims at showcasing how to use vLLM's OpenAI-compatible chat API. 

### Context
Lanchain already supports vLLM and its OpenAI-compatible `Completion` API. However, the `ChatCompletion` API was not aligned with OpenAI and for this reason I've waited  for this [PR](https://github.com/vllm-project/vllm/pull/852) to be merged before adding this notebook to langchain.
Better usage of the Template Prompt: More Structure
Labels: name: area: agent, name: auto:improvement, name: stale, name: area: models

### Feature request

I used the Custom LLM Chat Model Agent and created my agent like this:

```
class CustomPromptTemplate(StringPromptTemplate):
    template: str
    tools: List[Tool]

    def format(self, **kwargs) -> str:
        # Get the intermediate steps (AgentAction, Observation tuples)
        # Format them in a particular way
        intermediate_steps = kwargs.pop("intermediate_steps")
        thoughts = ""
        for action, observation in intermediate_steps:
            thoughts += action.log
            thoughts += f"\nObservation: {observation}\nThought: "
        # Set the agent_scratchpad variable to that value
        kwargs["agent_scratchpad"] = thoughts
        # Create a tools variable from the list of tools provided
        kwargs["tools"] = "\n".join(
            [f"{tool.name}: {tool.description}" for tool in self.tools])
        # Create a list of tool names for the tools provided
        kwargs["tool_names"] = ", ".join([tool.name for tool in self.tools])
        return self.template.format(**kwargs)
```

The problem is this. This is very inefficient way to prompt things. 

1) Better way to put tried actions.
2) Better way to put tools (maybe there some tools that you will eliminate after using one time.)
3) Overall better usage of the prompt. For example it cannot handle these kind of problems:

One of the steps, the agent chooses to use GetCurrentWeather but instead of that it takes Use GetCurrentWeather to get the current weather in Istanbul.' as the name of the tool.
```
input: "{'requested_tool_name': 'Use GetCurrentWeather to get the current weather in Istanbul.', 'available_tool_names': ['DuckDuckSearch', 'Search', 'Wikipedia', 'GetCurrentWeather', 'GetTickerName', 'GetTickerPrice']}"
```

the result:
```
Use GetCurrentWeather to get the current weather in Istanbul. is not a valid tool, try one of [DuckDuckSearch, Search, Wikipedia, GetCurrentWeather, GetTickerName, GetTickerPrice].
```

Is this possible to use directly openai function_calling like the openai agent ? Maybe that fixes the last problem.

### Motivation

The REACT way is too dirty and inefficient to use, it should be much more structured way to use CustomPromptTemplate and the CustomOutputParser

### Your contribution

If people comments on it, I can help with this.
Error with db = Chroma.from_documents(documents=chunks, embedding=embeddings, persist_directory=persist_directory)
Labels: name: area: embeddings, name: auto:bug, name: auto:question, name: integration: chroma

### Issue you'd like to raise.

When I run the following line:
db = Chroma.from_documents(documents=chunks, embedding=embeddings, persist_directory=persist_directory)

I get the error "You are using a deprecated configuration of Chroma."

I need to know what the new form of Chroma.from_documents() function is so that I can specify a persistent directory.

### Suggestion:

_No response_
Add to_dict() implementation to Memory classes which are non-serializable
Labels: name: area: memory, name: auto:improvement

### Feature request

Add to_dict() function implementation to Memory classes such as BaseMemory, ConversationBufferMemory, etc.

### Motivation

Currently, we cannot use the JSON library to JSONIFY or Serialize the contents of the memory classes. This may be important if we want to persist the memory in DB for reference, logging, and tracking. 

Doing `json.dumps(memory)` throws error:
`TypeError: Object of type ConversationBufferMemory is not JSON serializable` 

### Your contribution

Adding the implementation for to_dict() function to memory classes
DOC: HuggingFacePipeline.from_model_id throws Value error
Labels: name: auto:bug, name: auto:documentation, name: area: models

### Issue with current documentation:

I tried to load LLama2-7b model from huggingface using HuggingFacePipeline.from_model_id but throws a value error: ValueError: The model has been loaded with `accelerate` and therefore cannot be moved to a specific device. Please discard the `device` argument when creating your pipeline object.

Here is my Code:
compute_dtype=getattr(torch,"float16")
bnb_config=BitsAndBytesConfig(
      load_in_4bit=True,
      bnb_4bit_quant_type="nf4",
      bnb_4bit_compute_dtype=compute_dtype,
      bnb_4bit_use_double_quant=True,
  )

 model_name="meta-llama/Llama-2-7b-hf"

llm = HuggingFacePipeline.from_model_id(
      model_id=model_name,
      task="text-generation",
      pipeline_kwargs={
          "return_full_text":True,  # langchain expects the full text
          "temperature":0.5,  # 'randomness' of outputs, 0.0 is the min and 1.0 the max
          "max_new_tokens":512,  # max number of tokens to generate in the output
          "repetition_penalty":1.1, 
          "return_dict_in_generate":True,
          },
      model_kwargs={
          "quantization_config":bnb_config,
      }
  )


Logs:

WARNING:langchain.llms.huggingface_pipeline:Device has 1 GPUs available. Provide device={deviceId} to `from_model_id` to use availableGPUs for execution. deviceId is -1 (default) for CPU and can be a positive integer associated with CUDA device id.
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
[<ipython-input-9-14e8e7a13801>](https://localhost:8080/#) in <cell line: 1>()
----> 1 llm=prepare_model()


[/usr/local/lib/python3.10/dist-packages/transformers/pipelines/base.py](https://localhost:8080/#) in __init__(self, model, tokenizer, feature_extractor, image_processor, modelcard, framework, task, args_parser, device, torch_dtype, binary_output, **kwargs)
    788 
    789         if hf_device_map is not None and device is not None:
--> 790             raise ValueError(
    791                 "The model has been loaded with `accelerate` and therefore cannot be moved to a specific device. Please "
    792                 "discard the `device` argument when creating your pipeline object."

ValueError: The model has been loaded with `accelerate` and therefore cannot be moved to a specific device. Please discard the `device` argument when creating your pipeline object.


Is it problem with Langchain?

### Idea or request for content:

_No response_
speed up enforce_stop_tokens helper function
Labels: name: lgtm, name: auto:improvement, name: area: models

**Description:**

As long as `enforce_stop_tokens` returns a first occurrence, we can speed up the execution by setting the optional `maxsplit` parameter to 1.

Tag maintainer:
@agola11
@hwchase17

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
I want to use LangChain in Vercel's ai-chatbot template 
Labels: name: auto:question, name: stale, name: area: models

Hi. I'm using [Vercel's ai-chatbot template](https://github.com/vercel-labs/ai-chatbot) to prototype my application. I was wondering how I can convert the chat api call👇🏾  to use LangChain. I've been able to find examples that work, but I don't know how to have it so that I still use Vercel's KV database with LangChain.

Here's the code I would like to modify. Thanks.

```Javascript
// app/api/chat/route.ts

import { kv } from '@vercel/kv'
import { OpenAIStream, StreamingTextResponse } from 'ai'
import { Configuration, OpenAIApi } from 'openai-edge'

import { auth } from '@/auth'
import { nanoid } from '@/lib/utils'

export const runtime = 'edge'

const configuration = new Configuration({
  apiKey: process.env.OPENAI_API_KEY
})

const openai = new OpenAIApi(configuration)

export async function POST(req: Request) {
  const json = await req.json()
  const { messages, previewToken } = json
  const userId = (await auth())?.user.id

  if (!userId) {
    return new Response('Unauthorized', {
      status: 401
    })
  }

  if (previewToken) {
    configuration.apiKey = previewToken
  }

  const res = await openai.createChatCompletion({
    model: 'gpt-3.5-turbo',
    messages,
    temperature: 0.7,
    stream: true
  })

  const stream = OpenAIStream(res, {
    async onCompletion(completion) {
      const title = json.messages[0].content.substring(0, 100)
      const id = json.id ?? nanoid()
      const createdAt = Date.now()
      const path = `/chat/${id}`
      const payload = {
        id,
        title,
        userId,
        createdAt,
        path,
        messages: [
          ...messages,
          {
            content: completion,
            role: 'assistant'
          }
        ]
      }
      await kv.hmset(`chat:${id}`, payload)
      await kv.zadd(`user:chat:${userId}`, {
        score: createdAt,
        member: `chat:${id}`
      })
    }
  })

  return new StreamingTextResponse(stream)
}

```
Issue: I want to load SitemapLoader but only 2023 lastmod pages.
Labels: name: area: doc loader, name: auto:question, name: stale

### Issue you'd like to raise.

Hi , I'm using the SiteMap Loader as a document Loader , and I want to filter the lastmod metadata for the 2023 year only and nothing else, is it possible? 

Here is my code:
`# fixes a bug with asyncio and jupyter
import nest_asyncio
nest_asyncio.apply()

from langchain.document_loaders.sitemap import SitemapLoader

loader = SitemapLoader(
    "https:www.example.ca.sitemap.xml",
    filter_urls=["https://www.example.ca/en/"]
)
docs = loader.load()`

### Suggestion:

_No response_
10674
Labels: name: area: doc loader, name: auto:improvement

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Extended hugging face inference with text classification
Labels: name: auto:improvement, name: area: models

Extended with the current HuggingFaceEndpoint to support "text-classification" tasks. At the moment this doesn't support batch inference.

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Support for computing embeddings using ONNX 
Labels: name: needs documentation, name: area: embeddings, name: auto:enhancement

- **Description:** This PR introduces the inclusion of `OnnxEmbeddings` in the LangChain library. It allows users to compute embeddings for documents and queries using ONNX models. The change involves the creation of a new file `onnx.py` to house the `OnnxEmbeddings` class and the modification of the `__init__.py` file to include `OnnxEmbeddings` in the `__all__` list, making it accessible as part of the module's public API.
    
-   **Issue:** N/A (New Feature)
    
- **Dependencies:** This change requires the `onnx`, `onnxruntime`, `optimum`, and `transformers` packages. Users who wish to utilize `OnnxEmbeddings` should ensure that these packages are installed. If not, they can install them using the following command:
```bash    
pip install optimum transformers onnxruntime onnx
```
- **Tag Maintainer:** @baskaryan, @eyurtsev, @hwchase17
    
- **Twitter handle:** https://twitter.com/malteH_B
- **LinkedIn handle:** https://www.linkedin.com/in/maltehb/
    

### Implementation Details

The `OnnxEmbeddings` class inherits from `BaseModel` and `Embeddings`, and it overrides the `embed_documents` and `embed_query` methods to compute embeddings using ONNX models. It utilizes the `AutoTokenizer` from the `transformers` library for tokenization and `ORTModelForFeatureExtraction` from the `optimum.onnxruntime` package for feature extraction.

Currently, the model loaded from the `transformers` hub is just exported to ONNX format in-memory. One could imagine a flag for writing the ONNX model to disk, and then loading it. 

The users have the flexibility to pass keyword arguments to the model and during tokenization through `model_kwargs` and `encode_kwargs` respectively. They can also set a `query_instruction`, which will prepend the query text during the embedding of queries. This is, of course, important if using models such as BGE or Instruct.

### Testing

Since this is a new feature, it is crucial to include tests for this integration. Tests should focus on the functionality of `OnnxEmbeddings`, ensuring that it computes the embeddings correctly and handles edge cases well. Preferably, unit tests that do not rely on network access should be written to maintain the reliability of the test suite.

### Examples

An example notebook showcasing the usage of `OnnxEmbeddings` should be included in the `docs/extras` directory to aid users in understanding how to use this new feature.

### Final Notes

This feature is an enhancement for LangChain, providing users with a significantly faster option for computing embeddings, compared to using `SentenceTransformers` and other similar libraries.

### Checklist

- I have run `make format` and `make lint` locally.
- The  `make test` command did not work, due to another issue, which should be fixed in another PR. 

I would appreciate it if @baskaryan, @eyurtsev, or @hwchase17 could review this PR and provide their valuable feedback.
Extend HuggingFaceEndpoint Class to Support Text Classification Endpoints
Labels: name: auto:enhancement, name: area: models

### Feature request

The current HuggingFaceEndpoint class supports several tasks like text generation and summarization but lacks support for text classification. This feature request aims to extend the class to handle text classification tasks.



### Motivation

I am working on a project and I find myself having to trying to use langchain but it seems to be missing a text-classifcation is missing. I think it will be a great add as it allows us to chain text-classification tasks as well.

### Your contribution

I will make changes to the HuggingFaceEndpoint to allow anyone else to make a chain with text-classification task involved. At the moment this will not be for batch classification.
Support for `ONNX` embeddings
Labels: name: area: embeddings, name: auto:enhancement

-   **Description:** This PR introduces the addition of a new file, `onnx.py`, in the `langchain/langchain/embeddings/` directory. The newly added file implements the `OnnxEmbeddings` class, which serves to compute document and query embeddings using an ONNX model. This class allows the user to initialize ONNX models and compute embeddings for given texts with the help of the ONNX Runtime. If the required dependencies are not met, the class will raise an informative ImportError, prompting the user to install the necessary packages.
    
-   **Issue:** There is no issue to fix, other than supporting a significant speedup in embedding performance time.
    
-   **Dependencies:**
    -   optimum.onnxruntime: This is required for the `ORTModelForFeatureExtraction` class used in initializing the ONNX model.
    -   transformers: Required for the `AutoTokenizer` used for tokenizing the input text before passing it to the model.
-   **Tag Maintainer:** @hwchase17 
    
-   **Twitter Handle:** https://twitter.com/malteH_B

### Contribution Guidelines Compliance

-   Code has been formatted using `make format`.
-   Code is lint-free with `make lint`.
-   Tests pass with `make test`.
[OpenSearch] Pass ids using from_texts and indexname in add_texts and search
Labels: name: area: vector store, name: auto:improvement


### Description
This PR makes the following changes to OpenSearch:
1. Pass optional ids with `from_texts`
2. Pass an optional index name with `add_texts` and `search` instead of using the same index name that was used during `from_texts`

### Issue
https://github.com/langchain-ai/langchain/issues/10967

### Maintainers
@rlancemartin, @eyurtsev, @navneet1v