delayedkarma/langchain-issues huggingface dataset:
Issue: Want to get this to run, im suspecting that the csv. file is causing the problem
Labels: name: area: doc loader, name: auto:bug

### Issue you'd like to raise.

imports
import langchain
import os
from apikey import apikey
import openai
from langchain.embeddings import OpenAIEmbeddings
from langchain.vectorstores import Chroma
from langchain import OpenAI
from langchain.document_loaders import UnstructuredFileLoader
from langchain.text_splitter import CharacterTextSplitter
from langchain.chains import RetrievalQA

import streamlit as st
from langchain.document_loaders import TextLoader
from langchain.embeddings.openai import OpenAIEmbeddings
from langchain.llms import OpenAI
from langchain.text_splitter import CharacterTextSplitter



import nltk
nltk.download("punkt")
#loading file

loader = UnstructuredFileLoader("aisc-shapes-database-v16.0.csv","aisc-shapes-database-v16.0_a1085.pdf")              
documents = loader.load()
len(documents)

text_splitter = CharacterTextSplitter(chunk_size =1000000000, chunk_overlap = 0)
text = text_splitter.split_documents(documents)

embeddings = OpenAIEmbeddings()#(openai_api_key = os.environ['OPENAI_API_KEY'])


doc_search = Chroma.from_documents(text, embeddings)
chain = RetrievalQA.from_chain_type(llm =OpenAI(), chain_type="stuff", retriever=doc_search.as_retriever(search_kwargs={"k":1}))


query = "What is the area of wide flange W44X408"

result = chain.run(query)
print(result)

model.save('CIVEN-GPT')

### Suggestion:

It runs without the csv. file so im assuming it is that however I would like to be able to include the data in the file in the training.
openai_functions_multi_agent: solved the case when the "arguments" is valid JSON but it does not contain `actions` key
Labels: name: area: agent, name: lgtm, name: auto:improvement

Description: There are cases when the output from the LLM comes fine (i.e. function_call["arguments"] is a valid JSON object), but it does not contain the key "actions". So I split the validation in 2 steps: loading arguments as JSON and then checking for "actions" in it.
Get input prompt in a callback handler
Labels: name: auto:improvement, name: area: models

### Feature request

I want to intercept the input prompt and the output of a chain, so I added a custom callback to the chain (derived from _BaseCallbackHandler_), but the input prompt seems quite tricky to retrieve.

The _on_chain_start_ method has the information hidden in the "serialized" variable, but accessing it is quite cumbersome. I let you judge by yourself:

```
def on_chain_start(self, serialized: Dict[str, Any], inputs: Dict[str, Any], **kwargs: Any) -> Any:
    """Run when chain starts running."""
    print(serialized["kwargs"]["prompt"]["kwargs"]["messages"][0]["kwargs"]["prompt"]["kwargs"]["template"])
```

Note that the format of _serialized_ changes from time to time for a reason I ignore, and it doesn't seem to be documented. This makes it unusable. Moreover, the "template" value is not the final prompt passed to the LLM after replacement of variables.

As for the _on_text_ method, it contains a formatted and colored text:

> Prompt after formatting:
> Human: prompt in green

Are there simpler ways to retrieve the input prompt from a callback handler?

### Motivation

Showing both input and output could help debugging and it may be desirable to customize the outputs given by the _verbose_ mode.

### Your contribution

Maybe simply add the input message in the parameters of the _on_chain_start_ method, regardless of the way it has been generated.
Issue: Agents using GPTQ models from huggingface is really slow. 
Labels: name: area: agent, name: auto:bug, name: auto:question, name: area: models

### Issue you'd like to raise.

While trying to load a GPTQ model through a HuggingFace Pipeline and then run an agent on it, the inference time is really slow. 

```
# Load configuration from the model to avoid warnings
generation_config = GenerationConfig.from_pretrained(model_name_or_path)

# Create a pipeline for text generation
pipe = pipeline(
    task="text-generation",
    model=model,
    tokenizer=tokenizer,
    max_new_tokens=1024,
    do_sample=True,
    repetition_penalty=1.15,
    generation_config=generation_config,
    use_cache=False
)

local_llm = HuggingFacePipeline(pipeline=pipe)
logging.info("Local LLM Loaded")

```

The model is getting loaded on GPU

![image](https://github.com/langchain-ai/langchain/assets/20572827/3df28d2b-527c-492f-9a01-b6bf9f7b8933)

However the inference is really slow. I am waiting around 10 minutes for one iteration to complete. 

`agent = create_csv_agent(
    local_llm,
    "titanic.csv",
    verbose=True,
    agent_type=AgentType.ZERO_SHOT_REACT_DESCRIPTION,
    handle_parsing_errors=True
)`

`agent.run("What is the total number of rows in titanic.csv")
`

Also, I get an error message -` Observation: len() is not a valid tool, try one of [python_repl_ast].` How to enable all tools so that the agent can use them? 

### Suggestion:

No suggestion, require help.
Add language detection example
Labels: name: auto:enhancement

### Description

Adds language detection examples based on [langdetect](https://github.com/Mimino666/langdetect/tree/master/langdetect) and [fasttext](https://github.com/facebookresearch/fastText/) libraries. These frameworks can be especially useful together with components that require selection of the language (e.g. data-anonymizer)

### Twitter handle

@deepsense_ai, @matt_wosinski
Will langchain be able to support Baidu Qianfan in the future?
Labels: name: auto:enhancement, name: area: models

### Feature request

New features to support Baudu's Qianfan

### Motivation

I believe that Baidu's recently launched LLM platform, Qianfan, which offers a range of APIs, will soon become widely adopted. It would be beneficial to consider incorporating features that facilitate seamless integration between Langchain and Qianfan, making it easier for developers to build applications.

### Your contribution

https://github.com/langchain-ai/langchain/pull/10496
SimpleJsonOutputParser not working
Labels: name: auto:bug, name: auto:documentation, name: area: models

### System Info

Langchain 0.0.287

In output_parsers there is a `SimpleJsonOutputParser` defined (json.py). This looks very reasonable for easily getting answers back in structured a format. However, the class does not work as it does not specify the method `get_format_instructions`and thus calling the parse method raises a `NotImplementedError`. In addition, there is no documentation and the class is not imported into the `__init__.py` of the directory.

Is this intended behavior ? I am ok to submit a small patch - for my case the class comes very handy and has less complexity than the approach via Pydantic.

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [X] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Steps to reproduce

1. from langchain.output_parsers.json import SimpleJsonOutputParser
2. output_parser = SimpleJsonOutputParser()
3. format_instructions = output_parser.get_format_instructions()

### Expected behavior

SimpleJsonOutputParser works like any other output parser.
Added Streaming Capability to SageMaker LLMs
Labels: name: lgtm, name: auto:enhancement, name: area: models, name: integration: aws

This PR adds the ability to declare a Streaming response in the SageMaker LLM by leveraging the `invoke_endpoint_with_response_stream` capability in `boto3`. It is heavily based on the AWS Blog Post announcement linked [here](https://aws.amazon.com/blogs/machine-learning/elevating-the-generative-ai-experience-introducing-streaming-support-in-amazon-sagemaker-hosting/).

It does not add any additional dependencies since it uses the existing `boto3` version.
Fix SingleStoreDB
Labels: name: area: vector store, name: auto:bug

After the refactoring #6570, the DistanceStrategy class was moved to another module and this introduced a bug into the SingleStoreDB vector store, as the `DistanceStrategy.EUCLEDIAN_DISTANCE` started to convert into the 'DistanceStrategy.EUCLEDIAN_DISTANCE' string, instead of just 'EUCLEDIAN_DISTANCE' (same for 'DOT_PRODUCT'). 

In this change, I check the type of the parameter and use `.name` attribute to get the correct object's name.
Issue: Is LangSmith playground compatible with Azure OpenAI?
Labels: name: auto:bug, name: area: models

### Issue you'd like to raise.

1. I have downloaded original LangSmith walkthrough notebook and modified it to run AzureOpenAI llm instead of OpenAI
2. After successful run of the first example I went to Langsmith, selected first LLM call and opened it in the Playground.
3. I have filled up OpenAI key and hit 'Start'

Here is the error I get:

Error: Invalid namespace: $ -> {"id":["langchain","chat_models","azure_openai","AzureChatOpenAI"],"lc":1,"type":"constructor","kwargs":{"temperature":0,"openai_api_key":{"id":["xxx"],"lc":1,"type":"secret"},"deployment_name":"chat-gpt","openai_api_base":"yyy","openai_api_type":"azure","openai_api_version":"2023-03-15-preview"}}

I have played with different ways of setting OPEN_API_KEY but none of them works, the same error is consistently displayed.
So it is a bug or Azure Open AI does not work by design in the Playground?


### Suggestion:

_No response_
Handle by default `ToolException`
Labels: name: area: agent, name: auto:improvement

### Feature request

As of today, if a tool crashes, the whole agent or chain crashes. From a user point-of-view, it is understandable that a specific tool is not available.

### Motivation

The user experience should be maintained if a dependency is broken. Plus, catching by default tool error can enhance the software reliability.

### Relates

- https://github.com/langchain-ai/langchain/issues/8348
Bagatur/elevenlabs
Labels: name: lgtm, name: auto:enhancement, name: area: models

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. These live is docs/extras directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17, @rlancemartin.
 -->
Opensearch metadata_field and metadata_filter not working
Labels: name: area: embeddings, name: area: vector store, name: auto:bug, name: area: models

### System Info

langchain version = 0.0.281 
python =  3.11
opensearch-py = 2.3.0

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [X] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction


I am trying to do metadata based filtering alongside the query execution using `OpensearchVectorSearch.similarity_search()`. But when I use `metadata_field` and `metadata_filter`, the search doesn't seems to take that into account and still returns results outside of those filters.

Here is myr code:

`response = es.similarity_search( query = "<sample query text>", K =4, metadata_field = "title", metadata_filter = {"match":{"title": "<sample doc title>}}, )`

Here `es` is the `OpenSearchVectorSearch` object for `index1`

The output structure is like this:
`[Document(page_content = ' ', metadata={'vector_field' : [], 'text' : ' ', 'metadata' : {'source' : ' ', 'title' : ' ' }})]`

Here the title I see is not the title I specified in my query.

Steps to reproduce:

1. Create an Opensearch index with multiple documents.
2. Run similarity_search() query with a metadata_field and/or metadata_filter


### Expected behavior

The query should be run against the specified `metadata_field` and `metadata_filter` and in output, I should only see the correct document name I specified in `metadata_field` and `metadata_filter`
fix: unify generation outputs on newer openllm release
Labels: name: auto:improvement, name: area: models

update newer generation format from OpenLLm where it returns a dictionary for one shot generation

cc @baskaryan 

Signed-off-by: Aaron <29749331+aarnphm@users.noreply.github.com>

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. These live is docs/extras directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17, @rlancemartin.
 -->
Additional Weaviate Filter Comparators
Labels: name: auto:improvement

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - Description: a description of the change, 
  - Issue: the issue # it fixes (if applicable),
  - Dependencies: any dependencies required for this change,
  - Tag maintainer: for a quicker response, tag the relevant maintainer (see below),
  - Twitter handle: we announce bigger features on Twitter. If your PR gets announced and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/hwchase17/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. These live is docs/extras directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17, @rlancemartin.
 -->

### Description
When using Weaviate Self-Retrievers, certain common filter comparators generated by user queries were unimplemented, resulting in errors. This PR implements some of them. All linting and format commands have been run and tests passed.
### Issue
#10474
### Dependencies
timestamp module
only use past x messages 
Labels: name: auto:enhancement, name: area: models

### Feature request

An input for conversational chains to be able to limit their context to a set number of chats

### Motivation

I am in the process of building a document analysis tool using langchain but when the chat chain becomes too long, I just get an error stating that the limit for the no of openai tokens has been reached because the context keeps becoming longer and longer. is there some way i could limit the context to only a certain no of messages and not take all of them in.

### Your contribution

No I am very new to  using langchain and having a hard time understanding the codebase. so i am afraid their is nothing i could do to help.
Reduce azure openai token usage
Labels: name: area: agent, name: auto:question, name: auto:improvement, name: area: models

hi team,

I  am using the Azure openai gpt4-32k as llm in langchain. I implemented openai plugin by agent, but the cost is increasing at an incredible rate. I think the agent would ask gpt4 modal to understand the plugin openapi json that make the token usage increasing. any recommendations to reduce the token usage in agent?

thanks
Support jinja2 template format when using ChatPromptTemplate.from_template_file
Labels: name: area: doc loader, name: auto:improvement

### Feature request

BaseStringMessagePromptTemplate.from_template supports the template_format variable, while BaseStringMessagePromptTemplate.from_template_file does not.

### Motivation

All supported template formats (including Jinja2) should be supported by all template loaders equally.

### Your contribution

I'm not experienced enough with the Langchain codebase to submit PRs at this time.
fix: Update Google Cloud Enterprise Search to Vertex AI Search
Labels: name: auto:documentation

- Description: Google Cloud Enterprise Search was renamed to Vertex AI Search
  - https://cloud.google.com/blog/products/ai-machine-learning/vertex-ai-search-and-conversation-is-now-generally-available
  - This PR updates the documentation and Retriever class to use the new terminology.
  - Changed retriever class from `GoogleCloudEnterpriseSearchRetriever` to `GoogleVertexAISearchRetriever`
  - Updated documentation to specify that `extractive_segments` requires the new [Enterprise edition](https://cloud.google.com/generative-ai-app-builder/docs/about-advanced-features#enterprise-features) to be enabled.
  - Fixed spelling errors in documentation.
  - Change parameter for Retriever from `search_engine_id` to `data_store_id`
    - When this retriever was originally implemented, there was no distinction between a data store and search engine, but now these have been split.
  - Fixed an issue blocking some users where the api_endpoint can't be set
Fix fine-tuned replicate models with faster cold boot
Labels: name: auto:bug, name: area: models

With the latest support for faster cold boot in replicate https://replicate.com/blog/fine-tune-cold-boots it looks like the replicate LLM support in langchain is broken since some internal replicate inputs are being returned. 

Screenshot below illustrates the problem:

<img width="1917" alt="image" src="https://github.com/langchain-ai/langchain/assets/749277/d28c27cc-40fb-4258-8710-844c00d3c2b0">

As you can see, the new replicate_weights param is being sent down with x-order = 0 (which is causing langchain to use that param instead of prompt which is x-order = 1)

FYI @baskaryan this requires a fix otherwise replicate is broken for these models. I have pinged replicate whether they want to fix it on their end by changing the x-order returned by them.

Update: per suggestion I updated the PR to just allow manually setting the prompt_key which can be set to "prompt" in this case by callers... I think this is going to be faster anyway than trying to dynamically query the model every time if you know the prompt key for your model.
Elasticsearch ELSER Timeout
Labels: name: area: vector store, name: auto:bug, name: stale

### System Info

Python 3.10.12
Google Colab
Elasticsearch Cloud 8.9.2
Langchain - latest

### Who can help?

@hwchase17 @agola11

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Steps:

1. Load list of documents
2. Setup ElasticsearchStore of Langchain, with appropriate ES cloud credentials
3. Successfully create index with custom embedding model (HF embedding model, deployed on colab)
4. Deploy ELSER model and run it (with default model id).
5. Try creating index with SparseVectorRetrievalStrategy (ELSER) over the same list of documents.
6. Tried to change timeout, but didn't effect the outcome.
7. NOTE: It does start uploading docs and docs count is increasing, but it stops after about 10 sec. I tried to run the ELSER model on 3 nodes, but nothing changed.

### Expected behavior

WARNING:elastic_transport.node_pool:Node <Urllib3HttpNode([https://-------.us-central1.gcp.cloud.es.io:443](https:/------.us-central1.gcp.cloud.es.io/))> has failed for 1 times in a row, putting on 1 second timeout
---------------------------------------------------------------------------
ConnectionTimeout                         Traceback (most recent call last)
[<ipython-input-92----------cb>](https://-------colab.googleusercontent.com/outputframe.html?vrz=colab_20230911-060143_RC00_564310758#) in <cell line: 1>()
----> 1 elastic_elser_search = ElasticsearchStore.from_documents(
      2     documents=split_texts,
      3     es_cloud_id="cloudid",
      4     index_name="search-tmd-elser",
      5     es_user="elastic",

10 frames
[/usr/local/lib/python3.10/dist-packages/elastic_transport/_node/_http_urllib3.py](https://---XXXX-----0-colab.googleusercontent.com/outputframe.html?vrz=colab_---XXXX-----#) in perform_request(self, method, target, body, headers, request_timeout)
    197                 exception=err,
    198             )
--> 199             raise err from None
    200 
    201         meta = ApiResponseMeta(

ConnectionTimeout: Connection timed out
JSONLoader Documentation Fix
Labels: name: lgtm, name: area: doc loader, name: auto:documentation

- Description: 
Updated JSONLoader usage documentation which was making it unusable
- Issue: JSONLoader if used with the documented arguments was failing on various JSON documents. 
- Dependencies: 
no dependencies
- Twitter handle: @TheSlnArchitect

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17, @rlancemartin.
 -->
Update SQLDatabaseChain for intermediate SQLResults
Labels: name: auto:improvement

Description: 
In the call method within the SQLDatabaseChain class:
Amend the intermediate steps dictionary to include a new key, say sql_result, where the SQL results at different stages would be saved.
During the SQL execution step, save the SQL result into the sql_result key in the intermediate steps dictionary, similar to how sql_cmd is being saved currently.

Issue: #10500 
Tag maintainer: zzfab
docs: `self-query` consistency
Labels: name: area: doc loader, name: auto:documentation

The `self-que[ring` navbar](https://python.langchain.com/docs/modules/data_connection/retrievers/self_query/) has repeated `self-quering` repeated in each menu item. I've simplified it to be more readable
- removed `self-quering` from a title of each page;
- added description to the vector stores
- added description and link to the Integration Card (`integrations/providers`) of the vector stores when they are missed.