delayedkarma/langchain-issues huggingface dataset:
Add Connery Tool and Toolkit
Labels: name: area: agent, name: lgtm, name: auto:enhancement, name: size:XL

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->

## Summary

This PR implements the "Connery Action Tool" and "Connery Toolkit". Using them, you can integrate Connery actions into your LangChain agents and chains.

Connery is an open-source plugin infrastructure for AI.

With Connery, you can easily create a custom plugin with a set of actions and seamlessly integrate them into your LangChain agents and chains. Connery will handle the rest: runtime, authorization, secret management, access management, audit logs, and other vital features. Additionally, Connery and our community offer a wide range of ready-to-use open-source plugins for your convenience.

Learn more about Connery:

- GitHub: https://github.com/connery-io/connery-platform
- Documentation: https://docs.connery.io
- Twitter: https://twitter.com/connery_io

## TODOs

- [x] API wrapper
   - [x] Integration tests
- [x] Connery Action Tool
   - [x] Docs
   - [x] Example
   - [x] Integration tests
- [x] Connery Toolkit
  - [x] Docs
  - [x] Example
- [x] Formatting (`make format`)
- [x] Linting (`make lint`)
- [x] Testing (`make test`)
DOC: LLM Caching example using in-memory cache is unclear
Labels: name: area: doc loader, name: auto:documentation

### Issue with current documentation:

The in-memory cache section of the LLM Caching documentation shows uncached and subsequent cached responses that differ. The later examples show cached responses that are the same as the uncached response, which makes more sense.  

### Idea or request for content:

The cached response under LLM Caching: In-memory  should be "\n\nWhy couldn't the bicycle stand up by itself? It was...two tired!"
Support weight only quantization with intel-extension-for-transformers.
Labels: name: auto:enhancement, name: area: models, name: size:XL

Support weight only quantization with intel-extension-for-transformers.
[Intel® Extension for Transformers](https://github.com/intel/intel-extension-for-transformers) is an innovative toolkit to accelerate Transformer-based models on Intel platforms, in particular effective on 4th Intel Xeon Scalable processor [Sapphire Rapids](https://www.intel.com/content/www/us/en/products/docs/processors/xeon-accelerated/4th-gen-xeon-scalable-processors.html) (codenamed Sapphire Rapids). The toolkit provides the below key features:

* Seamless user experience of model compressions on Transformer-based models by extending [Hugging Face transformers](https://github.com/huggingface/transformers) APIs and leveraging [Intel® Neural Compressor](https://github.com/intel/neural-compressor)
* Advanced software optimizations and unique compression-aware runtime.
* Optimized Transformer-based model packages.
* [NeuralChat](https://github.com/intel/intel-extension-for-transformers/blob/main/intel_extension_for_transformers/neural_chat), a customizable chatbot framework to create your own chatbot within minutes by leveraging a rich set of plugins and SOTA optimizations.
* [Inference](https://github.com/intel/intel-extension-for-transformers/blob/main/intel_extension_for_transformers/llm/runtime/graph) of Large Language Model (LLM) in pure C/C++ with weight-only quantization kernels.
This PR is an integration of weight only quantization feature with intel-extension-for-transformers.

Unit test is in lib/langchain/tests/integration_tests/llm/test_weight_only_quantization.py
The notebook is in docs/docs/integrations/llms/weight_only_quantization.ipynb.
The document is in docs/docs/integrations/providers/weight_only_quantization.mdx.
docs[patch] Fix some typos in merger_retriever.ipynb
Labels: name: lgtm, name: area: doc loader, name: auto:documentation, name: size:XS

This patch fixes some typos.

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
Validate_template modification.
Labels: name: area: doc loader, name: auto:documentation, name: size:L

I have made changes to the validate_template Document.
The validate_template is now set to False by default while creating a PromptTemplate. So, I have made sure that those changes are now being reflected in this new validate.ipynb file where I have written codes by using both the cases, that are validate_template = True and validate_template = True.
feat: Yaml output parser
Labels: name: lgtm, name: auto:enhancement, name: area: models, name: size:L

## Description
New YAML output parser as a drop-in replacement for the Pydantic output parser. Yaml is a much more token-efficient format than JSON, proving to be **~35% faster and using the same percentage fewer completion tokens**.

☑️ Formatted
☑️ Linted
☑️ Tested (analogous to the existing`test_pydantic_parser.py`)

The YAML parser excels in situations where a list of objects is required, where the root object needs no key:
```python
class Products(BaseModel):
   __root__: list[Product]
```

I ran the prompt `Generate 10 healthy, organic products` 10 times on one chain using  the `PydanticOutputParser`, the other one using the`YamlOutputParser` with `Products` (see below) being the targeted model to be created. 

LLMs used were Fireworks' `lama-v2-34b-code-instruct` and OpenAI `gpt-3.5-turbo`.  All runs succeeded without validation errors.

```python
class Nutrition(BaseModel):
    sugar: int = Field(description="Sugar in grams")
    fat: float = Field(description="% of daily fat intake")

class Product(BaseModel):
    name: str = Field(description="Product name")
    stats: Nutrition

class Products(BaseModel):
    """A list of products"""

    products: list[Product] # Used `__root__` for the yaml chain
```
Stats after 10 runs reach were as follows:
### JSON
ø time: 7.75s
ø tokens: 380.8

### YAML
ø time: 5.12s
ø tokens: 242.2


Looking forward to feedback, tips and contributions!
feature: implement _acall method for GPT4All(LLM)
Labels: name: auto:improvement, name: area: models, name: size:M

 - **Description:** Implement the _acall method of the class langchain.llms.gpt4all.GPT4All  
 - **Issue:** GPT4All chat error with async calls #5210 
 - **Dependencies:** no dependency
 - **Tag maintainer:** @hwchase17
 - **Twitter handle:** None

Okey, I tested locally. 

I ran "make format". No issue.

I ran "make lint". I found this issue [How to fix error "cannot import name 'ModelMetaclass' from 'pydantic.main'"](https://stackoverflow.com/questions/76647518/how-to-fix-error-cannot-import-name-modelmetaclass-from-pydantic-main). I followed the answer pip install lightning==2.0.1 and the command ran without problem. I believe this is not related to the PR but I had to clarify this.

I ran "make test". No issue. 

This is more of a fix for the existing default implementation of the _acall method inherited by the GPT4All class.

This is my first contribution ever in the open source world, so let me know if there is anything wrong. 

1. Let me know if there is a need for adding unit tests and integration tests for LLMs or chains or callbacks. I read the existing tests and found out that they cover them in a generic manner.

2. I have a working example that showcases how to effectively use an asynchronous callback handler with the GPT4All and RetrievalQA. I did not find the directory docs/extra, should I put it in cookbooks?
AmazonKendraRetriever: Error The provided QueryText has a character count of 1020, which exceeds the limit.
Labels: name: area: memory, name: area: vector store, name: auto:bug, name: integration: aws

### System Info

Langchain version: 0.0.348
Python version: 3.10.6
Platform: Linux

The issue is that when using Conversational Chain for with Kendra Retriever with memory, on any followup questions it gives this error.
My guess is somehow QueryText is getting more than the {question} value asked.



![image](https://github.com/langchain-ai/langchain/assets/147480492/ddf6ce1f-8ca2-45da-842a-06d2232ba6aa)
It works for first hit but for any follow up question (while memory is not empty) it gives this error.



### Who can help?

@hwchase17 
@ago

### Information

- [X] The official example notebooks/scripts
- [ ] My own modified scripts

### Related Components

- [ ] LLMs/Chat Models
- [ ] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [X] Vector Stores / Retrievers
- [X] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

Steps to reproduce error:

Create the retriever object:
from langchain.retrievers import AmazonKendraRetriever

retriever = AmazonKendraRetriever(
    index_id=kendra_index_id,
    top_k = 5
)

Then use it in Conversational Chain:
chain = ConversationalRetrievalChain.from_llm(
    llm, 
    retriever=retriever,
    memory=memory,
    verbose=True,
    condense_question_prompt=chat_prompt,
    combine_docs_chain_kwargs=dict(prompt=rag_prompt))

chain.run({"question":question})

Run at least twice, to get the error.


### Expected behavior

There shouldn't be any such error.
Question or QueryText is always less than 1000 characters.
Fix token_usage None issue in ChatOpenAI with local Chatglm2-6B
Labels: name: lgtm, name: auto:bug, name: area: models, name: size:S

When using local Chatglm2-6B by changing OPENAI_BASE_URL to localhost, the token_usage in ChatOpenAI becomes None. This leads to an AttributeError when trying to access token_usage.items().

This commit adds a check to ensure token_usage is not None before accessing its items. This change prevents the AttributeError and allows ChatOpenAI to work seamlessly with a local Chatglm2-6B model, aligning with the way it operates with the OpenAI API.

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
refactor: Qdrant async improvements
Labels: name: lgtm, name: area: vector store, name: auto:refactor, name: size:L

Follow up on https://github.com/langchain-ai/langchain/pull/13048.
This PR intends to simplify the Qdrant async implementation by replacing the internal GRPC methods with the `QdrantAsyncClient` methods.
This is a backward compatible change with no additional steps required after merge.
Update OpenAI Default Model to 'gpt-3.5-turbo-instruct' 
Labels: name: auto:improvement, name: area: models, name: size:XS

As per the guidance provided on the OpenAI model documentation page (https://platform.openai.com/docs/models/gpt-3-5), the text-davinci-003 model is scheduled to be discontinued as of January 4, 2024. 

![image](https://github.com/langchain-ai/langchain/assets/375580/2f567596-b831-4e80-899f-c9726b772463)

In response to this upcoming change, I have updated the default model in the langchain.llms.openai.OpenAI class from 'text-davinci-003' to 'gpt-3.5-turbo-instruct', following the recommendations provided by OpenAI.

![image](https://github.com/langchain-ai/langchain/assets/375580/f48ff1b2-7347-4ac3-99f5-2fe65e57a1ff)

Additionally, I have also updated the modelname_to_contextsize() method to include the maximum token count for the new models(gpt-4-1106-preview, gpt-3.5-turbo-1106)
fix(embeddings): huggingface hub embeddings and TEI
Labels: name: lgtm, name: area: embeddings, name: auto:bug, name: auto:documentation, name: size:M

**Description:** This PR fixes `HuggingFaceHubEmbeddings` by making the API token optional (as in the client beneath). Most models don't require one. I also updated the notebook for TEI (text-embeddings-inference) accordingly as requested here #14288. In addition, I fixed a  mistake in the POST call parameters.

**Tag maintainers:** @baskaryan
Fixed issue with importing Row from sqlalchemy
Labels: name: area: memory, name: lgtm, name: auto:bug, name: size:XS

  - **Description:** Fixed import of Row in cache.py, 
  - **Issue:** the issue #  #13464 https://creditone.us.to/langchain-ai/langchain/issues/13464,
  - **Dependencies:** None,
  - **Twitter handle:** @frankybridman
why the sql langchain's custom prompt is not working?
Labels: name: area: agent, name: auto:bug, name: area: models

### Issue you'd like to raise.

I tried to work on SQL cutsom prompt, but it didn't work and is still giving the wrong sql queries . Here is the code : def process_user_input(user_input):
    create_db()
    input_db = SQLDatabase.from_uri('sqlite:///sample_db_2.sqlite')
    llm_1 = OpenAI(temperature=0)
    db_agent = SQLDatabaseChain.from_llm(llm = llm_1,
                            db = input_db,
                            verbose=True,)
    chain = create_sql_query_chain(ChatOpenAI(temperature=0), input_db)
    response = chain.invoke({"question": user_input})

### Suggestion:

_No response_
DOC: model update in 'Using OpenAI Functions' docs
Labels: name: lgtm, name: auto:documentation, name: area: models, name: size:XS

  - **Description:** : 
I just update the openai functions docs to use the latest model (ex. gpt-3.5-turbo-1106)
https://python.langchain.com/docs/modules/chains/how_to/openai_functions

The reason is as follow: 

After reviewing the OpenAI Function Calling official guide at https://platform.openai.com/docs/guides/function-calling,  the following information was noted:

> "The latest models (gpt-3.5-turbo-1106 and gpt-4-1106-preview) have been trained to both detect when a function should be called (depending on the input) and to respond with JSON that adheres to the function signature more closely than previous models. With this capability also comes potential risks. We strongly recommend building in user confirmation flows before taking actions that impact the world on behalf of users (sending an email, posting something online, making a purchase, etc)."

CC: @efriis
Retrievers don't seem to work properly with ollama
Labels: name: area: embeddings, name: area: vector store, name: auto:bug, name: area: models

### System Info

I have retriever implementation like this
```
def get_vector_store(options: StoreOptions) -> VectorStore:
    """Gets the vector store for the given options."""
    vector_store: VectorStore
    embedding = get_embeddings()

    store_type = os.environ.get("STORE")

    if store_type == StoreType.QDRANT.value:
        client = qdrant_client.QdrantClient(
            url=os.environ["QDRANT_URL"],
            prefer_grpc=True,
            api_key=os.getenv("QDRANT_API_KEY", None),
        )

        vector_store = Qdrant(
            client, collection_name=options.namespace, embeddings=embedding
        )

        # vector_store = Qdrant.from_documents([], embedding, url='http://localhost:6333', collection=options.namespace)

    else:
        raise ValueError("Invalid STORE environment variable value")

    return vector_store
 ```
 
 get-embeddings.py
```
         return OllamaEmbeddings(base_url=f"host.docker.internal:11434", model="mistral")
```


```
knowledgebase: VectorStore = get_vector_store(StoreOptions("knowledgebase"))
async def get_relevant_docs(text: str, bot_id: str) -> Optional[str]:
    try:
        kb_retriever = knowledgebase.as_retriever(
            search_kwargs={
                "k": 3,
                "score_threshold": vs_thresholds.get("kb_score_threshold"),
                "filter": {"bot_id": bot_id},
            },
        )

        result = kb_retriever.get_relevant_documents(text)

        if result and len(result) > 0:
            # Assuming result is a list of objects and each object has a page_content attribute
            all_page_content = "\n\n".join([item.page_content for item in result])

            return all_page_content

        return None

    except Exception as e:
        logger.error(
            "Error occurred while getting relevant docs",
            incident="get_relevant_docs",
            payload=text,
            error=str(e),
        )
        return None
 ```
 
 As long as i use chatgpt embeddings and chat models, i get the correct outputs. Once i switch to ollama, none of my retrievers are working.
 
 I see the documents being ingested to qdrant, which means embeddings are working, but retrievers fail to retrieve any document

### Who can help?

_No response_

### Information

- [ ] The official example notebooks/scripts
- [X] My own modified scripts

### Related Components

- [X] LLMs/Chat Models
- [X] Embedding Models
- [ ] Prompts / Prompt Templates / Prompt Selectors
- [ ] Output Parsers
- [ ] Document Loaders
- [ ] Vector Stores / Retrievers
- [ ] Memory
- [ ] Agents / Agent Executors
- [ ] Tools / Toolkits
- [ ] Chains
- [ ] Callbacks/Tracing
- [ ] Async

### Reproduction

ss

### Expected behavior

retrievers should be able to fetch the documents from qdrant irrespective of embedding models being used
fix dashvector endpoint params error
Labels: name: lgtm, name: area: vector store, name: auto:bug, name: size:S

<!-- Thank you for contributing to LangChain!

Replace this entire comment with:
  - **Description:** a description of the change, 
  - **Issue:** the issue # it fixes (if applicable),
  - **Dependencies:** any dependencies required for this change,
  - **Tag maintainer:** for a quicker response, tag the relevant maintainer (see below),
  - **Twitter handle:** we announce bigger features on Twitter. If your PR gets announced, and you'd like a mention, we'll gladly shout you out!

Please make sure your PR is passing linting and testing before submitting. Run `make format`, `make lint` and `make test` to check this locally.

See contribution guidelines for more information on how to write/run tests, lint, etc: 
https://github.com/langchain-ai/langchain/blob/master/.github/CONTRIBUTING.md

If you're adding a new integration, please include:
  1. a test for the integration, preferably unit tests that do not rely on network access,
  2. an example notebook showing its use. It lives in `docs/extras` directory.

If no one reviews your PR within a few days, please @-mention one of @baskaryan, @eyurtsev, @hwchase17.
 -->
generic codebase?
Labels: name: auto:question, name: auto:enhancement, name: area: models

### Feature request

It would be nice to write code for any generic LLM to use in a given chain, being able to specify its provider and parameters in a config file outside the source code.

Is there a built-in way to do so? What major roadblocks would you see in doing that?

### Motivation

Most examples that I see in the docs or over the Internet tend to specify from code whether the LLM to be used in a given chain has to be from openAI, anthropic, LLaMA, etc., resulting in different codebases, while it would be great to write one single unified codebase and compare the performance of different open-source or API LLMs by simply changing one line of config.

This would be especially relevant when the specific LLM provider is not set or known from the get-go and multiple ones should be compared to find the most suitable one in terms of perfomance, efficiency and cost for any given LLM-driven application.

### Your contribution

I don't have time for a PR now, but since I have been doing a similar thing for our private codebase at the laboratory, combining openAI, anthropic and LLaMA under a single unified spell without using the langchain framework, I may be interested in supporting that in the future as the framework already has lots of interesting features in terms of tooling, memory and storage.
feat: adding paygo api support
Labels: name: auto:enhancement, name: area: models, name: size:L

- **Description:** Introducing support for LLMs and Chat models running in Azure AI studio and Azure ML using the new deployment mode pay-as-you-go (model as a service).
- **Issue:** NA
- **Dependencies:** None.
- **Tag maintainer:** @prakharg-msft @gdyre 
- **Twitter handle:** @santiagofacundo

Examples added:
* [docs/docs/integrations/llms/azure_ml.ipynb](https://github.com/santiagxf/langchain/blob/santiaxf/azureml-endpoints-paygo/docs/docs/integrations/llms/azure_ml.ipynb)
* [docs/docs/integrations/chat/azureml_chat_endpoint.ipynb](https://github.com/santiagxf/langchain/blob/santiaxf/azureml-endpoints-paygo/docs/docs/integrations/chat/azureml_chat_endpoint.ipynb)