Repository: mindsdb/mindsdb
Pull request: Updates to LiteLLM integration
## Description

This PR fixes #9472 , https://github.com/mindsdb/mindsdb/security/dependabot/142 and https://github.com/mindsdb/mindsdb/security/dependabot/143 and also updates the README to follow up to the new structure.


## Type of change

(Please delete options that are not relevant)

- [X] 🐛 Bug fix (non-breaking change which fixes an issue)
- [ ] ⚡ New feature (non-breaking change which adds functionality)
- [ ] 📢 Breaking change (fix or feature that would cause existing functionality not to work as expected)
- [ ] 📄 This change requires a documentation update


## Checklist:

- [X] My code follows the style guidelines(PEP 8) of MindsDB.
- [X] I have appropriately commented on my code, especially in complex areas.
- [X] Necessary documentation updates are either made or tracked in issues.
- [ ] Relevant unit and integration tests are updated or added.





Problem statement:
[Bug]: LiteLLM integration issues when querying
When querying the LiteLLM integration, the following error is returned:
```
[litellm/litellm_handler_simple1]: BadRequestError: litellm.BadRequestError: BadRequestError: OpenAIException - Error code: 400 - {'error': {'message': "Invalid 'functions': empty array. Expected an array with minimum length 1, but got an empty array instead.", 'type': 'invalid_request_error', 'param': 'functions', 'code': 'empty_array'}}
```
Steps to reproduce:
1. Create new model with litellm engine:
```
create model litellm_handler_simple6
predict text
using
engine="litellm",
model="gpt-4",
api_key="sk-xxx";
```
2. Query the above model:
```
select * from litellm_handler_simple6 where text='Once upon a time';
```

I guess this is related to sending a few deprecated parameters as `functions` to the latest OpenAI API.

Hints from issue discussion:

Commit message: Update README

Patch to review:
diff --git a/mindsdb/integrations/handlers/litellm_handler/README.md b/mindsdb/integrations/handlers/litellm_handler/README.md
index 7fa5753442b..48898beea5e 100644
--- a/mindsdb/integrations/handlers/litellm_handler/README.md
+++ b/mindsdb/integrations/handlers/litellm_handler/README.md
@@ -1,120 +1,60 @@
-# Litellm Handler
+## LlamaIndex Handler
 
-## Briefly describe what ML framework does this handler integrate to MindsDB, and how?
+This documentation describes the integration of MindsDB with [LiteLLM](https://docs.litellm.ai/docs/), that allows you to call all LLM APIs using the OpenAI format.
 
-litellm (https://docs.litellm.ai/docs/) is a library that allows you to use popular LLM models such as OpenAI's GPT-3 and Anthropic claude-2. 
-This handler allows you to use litellm models in MindsDB.
+## Prerequisites
 
-## Why is this integration useful? What does the ideal predictive use case for this integration look like? When would you definitely not use this integration?
+Before proceeding, ensure the following prerequisites are met:
 
-It provides a simple interface to use LLM models in MindsDB, see https://docs.litellm.ai/docs/providers for a list of supported providers.
+1. Install MindsDB locally via [Docker](https://docs.mindsdb.com/setup/self-hosted/docker) or [Docker Desktop](https://docs.mindsdb.com/setup/self-hosted/docker-desktop).
+2. To use LiteLLM within MindsDB, install the required dependencies following [this instruction](https://docs.mindsdb.com/setup/self-hosted/docker#install-dependencies).
 
-## Are models created with this integration fast and scalable, in general?
 
-Yes, the models are hosted by the provider, so there is no need for any additional system specifications. The only real requirement is to have an internet connection.
+## Setup
 
-## What are the recommended system specifications for models created with this framework?
-
-Since we use hosted LLM APIs, there is no need for any additional system specifications. The only real requirement is to have an internet connection.
-
-## To what degree can users control the underlying framework by passing parameters via the USING syntax?
-
-Users are allowed complete control over the underlying framework by passing parameters via the USING syntax.
-
-## Does this integration offer model explainability or insights via the DESCRIBE syntax?
-
-No, this integration doesn't support DESCRIBE syntax.
-
-## Does this integration support fine-tuning pre-existing models (i.e. is the update() method implemented)? Are there any caveats?
-
-No, fine-tuning is not supported.
-
-## Are there any other noteworthy aspects to this handler?
-
-The handler has a number of default parameters set, the user only needs to pass (see settings.py for more details):
-
-- model: str - the name of the model to use
-- api_key: str - the api key for the provider of the model
-
-## Any directions for future work in subsequent versions of the handler?
-
-tbc
-
-
-## Please provide a minimal SQL example that uses this ML engine (pointers to integration tests in the PR also valid)
+Create an AI engine from the [LiteLLM handler](https://github.com/mindsdb/mindsdb/tree/main/mindsdb/integrations/handlers/litellm_handler).
 
 ```sql
--- Create LLM engine NB you can provide api keys here or at create
--- You only need to provide key of llm you intend to use
-
-create ML_Engine litellm from litellm;
-
---simple completion with openai
-
-create model litellm_handler_simple
-predict text
-using
-engine="litellm",
-model="gpt-3.5-turbo",
-api_key="openai-api-key";
-
-select * from information_schema.models where name ="litellm_handler_simple" ;
-
-select * from litellm_handler_simple where text='Once upon a time';
-
---simple completion with anthropic
-
-create model litellm_handler_simple_anthropic
-predict text
-using
-engine="litellm",
-model="claude-2",
-api_key="anthropic-api-key";
-
-select * from information_schema.models where name ="litellm_handler_simple_anthropic" ;
-
-select * from litellm_handler_simple_anthropic where text='Once upon a time';
+CREATE ML_ENGINE litellm
+FROM litellm;
+```
 
---completion using prompt_template on create
 
-create model litellm_handler_prompt_template
-predict text
-using
-engine="litellm",
-model="gpt-3.5-turbo",
-api_key="openai-api-key",
-prompt_template="write a story based on {{text}}";
+## Usage
 
-select * from information_schema.models where name ="litellm_handler_prompt_template" ;
+Here is how to create a model that answers questions:
 
-select * from litellm_handler_prompt_template where text='Once upon a time';
+```sql
+CREATE MOEL chat_model
+PREDICT text
+USING
+    engine="litellm",
+    model="gpt-4",
+    api_key="openai-api-key";
+```
 
---completion using prompt_template on predict
+Query the model to get answer:
 
-create model litellm_handler_prompt_template_predict
-predict text
-using
-engine="litellm",
-model="gpt-3.5-turbo",
-api_key="openai-api-key";
+```sql
+SELECT *
+FROM mindsdb.chat_model
+WHERE question = "How far is the moon?"
+```
 
-select * from information_schema.models where name ="litellm_handler_prompt_template_predict" ;
+Here is the output:
 
-select * from litellm_handler_prompt_template_predict 
-where text='Once upon a time' prompt_template="write a story based on {{text}}";
+```sql
++---------------------------+-------------------------------+
+|question                   |answer                         |
++---------------------------+-------------------------------+
+|How far is the moon?   |    384,400 km                     |
++---------------------------+-------------------------------+
 
+```
 
---completion using messages with batch_completion (multiple messages)
+<Tip>
 
-create model litellm_handler_messages
-predict text
-using
-engine="litellm",
-model="gpt-3.5-turbo",
-api_key="openai-api-key";
-       
-select * from information_schema.models where name ="litellm_handler_messages" ;
+**Next Steps**
+Go to the [Use Cases](https://docs.mindsdb.com/use-cases/overview) section to see more examples.
+</Tip>
 
-select * from litellm_handler_messages 
-where messages='[[{"content": "The sky is blue", "role": "user"}], [{"content": "The grass is green", "role": "user"}]]';
-```
diff --git a/mindsdb/integrations/handlers/litellm_handler/requirements.txt b/mindsdb/integrations/handlers/litellm_handler/requirements.txt
index 19c81b7d40a..3f5fad229eb 100644
--- a/mindsdb/integrations/handlers/litellm_handler/requirements.txt
+++ b/mindsdb/integrations/handlers/litellm_handler/requirements.txt
@@ -1 +1 @@
-litellm==1.35.0
+litellm==1.40.16
diff --git a/mindsdb/integrations/handlers/litellm_handler/settings.py b/mindsdb/integrations/handlers/litellm_handler/settings.py
index 25381746bc8..b83b331ac9d 100644
--- a/mindsdb/integrations/handlers/litellm_handler/settings.py
+++ b/mindsdb/integrations/handlers/litellm_handler/settings.py
@@ -6,9 +6,8 @@ class CompletionParameters(BaseModel):
     model: str  # The model to be used for the API call.
     prompt_template: Optional[str] = None  # Template for the prompt to be used.
 
-    # Optional OpenAI params: see https://platform.openai.com/docs/api-reference/chat/create
-    functions: List = []  # List of functions for advanced model operations.
-    function_call: str = ""  # String to specify a particular function call, if needed.
+    # Optional OpenAI params: see  https://platform.openai.com/docs/api-reference/runs
+    tool_choice: Optional[str] = None  # Specific tool to be used, if applicable.
 
     timeout: Optional[Union[float, int]] = None  # Timeout for the API request.
     temperature: Optional[float] = None  # Controls randomness: higher value means more random responses.
@@ -33,7 +32,6 @@ class CompletionParameters(BaseModel):
     base_url: Optional[str] = None  # Base URL of the API.
     api_version: Optional[str] = None  # Version of the API to be used.
     api_key: str  # API key for authentication.
-    llm_model_list: Optional[List] = None  # List of models, API bases, keys, etc., for dynamic selection.
 
     class Config:
         extra = Extra.forbid


Code review comments:
diff_hunk: @@ -1,120 +1,60 @@
-# Litellm Handler
+## LlamaIndex Handler
path: mindsdb/integrations/handlers/litellm_handler/README.md
text: @user1:
This should be LiteLLM.

diff_hunk: @@ -6,9 +6,8 @@ class CompletionParameters(BaseModel):
     model: str  # The model to be used for the API call.
     prompt_template: Optional[str] = None  # Template for the prompt to be used.
 
-    # Optional OpenAI params: see https://platform.openai.com/docs/api-reference/chat/create
-    functions: List = []  # List of functions for advanced model operations.
-    function_call: str = ""  # String to specify a particular function call, if needed.
+    # Optional OpenAI params: see  https://platform.openai.com/docs/api-reference/runs
path: mindsdb/integrations/handlers/litellm_handler/settings.py
text: @user1:
This link refers to the Assistants API, but I think we are calling the Chat Completions API within the handler. Should it be changed?

diff_hunk: @@ -1,120 +1,60 @@
-# Litellm Handler
+## LlamaIndex Handler
 
-## Briefly describe what ML framework does this handler integrate to MindsDB, and how?
+This documentation describes the integration of MindsDB with [LiteLLM](https://docs.litellm.ai/docs/), that allows you to call all LLM APIs using the OpenAI format.
 
-litellm (https://docs.litellm.ai/docs/) is a library that allows you to use popular LLM models such as OpenAI's GPT-3 and Anthropic claude-2. 
-This handler allows you to use litellm models in MindsDB.
+## Prerequisites
 
-## Why is this integration useful? What does the ideal predictive use case for this integration look like? When would you definitely not use this integration?
+Before proceeding, ensure the following prerequisites are met:
 
-It provides a simple interface to use LLM models in MindsDB, see https://docs.litellm.ai/docs/providers for a list of supported providers.
+1. Install MindsDB locally via [Docker](https://docs.mindsdb.com/setup/self-hosted/docker) or [Docker Desktop](https://docs.mindsdb.com/setup/self-hosted/docker-desktop).
+2. To use LiteLLM within MindsDB, install the required dependencies following [this instruction](https://docs.mindsdb.com/setup/self-hosted/docker#install-dependencies).
 
-## Are models created with this integration fast and scalable, in general?
 
-Yes, the models are hosted by the provider, so there is no need for any additional system specifications. The only real requirement is to have an internet connection.
+## Setup
 
-## What are the recommended system specifications for models created with this framework?
-
-Since we use hosted LLM APIs, there is no need for any additional system specifications. The only real requirement is to have an internet connection.
-
-## To what degree can users control the underlying framework by passing parameters via the USING syntax?
-
-Users are allowed complete control over the underlying framework by passing parameters via the USING syntax.
-
-## Does this integration offer model explainability or insights via the DESCRIBE syntax?
-
-No, this integration doesn't support DESCRIBE syntax.
-
-## Does this integration support fine-tuning pre-existing models (i.e. is the update() method implemented)? Are there any caveats?
-
-No, fine-tuning is not supported.
-
-## Are there any other noteworthy aspects to this handler?
-
-The handler has a number of default parameters set, the user only needs to pass (see settings.py for more details):
-
-- model: str - the name of the model to use
-- api_key: str - the api key for the provider of the model
-
-## Any directions for future work in subsequent versions of the handler?
-
-tbc
-
-
-## Please provide a minimal SQL example that uses this ML engine (pointers to integration tests in the PR also valid)
+Create an AI engine from the [LiteLLM handler](https://github.com/mindsdb/mindsdb/tree/main/mindsdb/integrations/handlers/litellm_handler).
 
 ```sql
--- Create LLM engine NB you can provide api keys here or at create
--- You only need to provide key of llm you intend to use
-
-create ML_Engine litellm from litellm;
-
---simple completion with openai
-
-create model litellm_handler_simple
-predict text
-using
-engine="litellm",
-model="gpt-3.5-turbo",
-api_key="openai-api-key";
-
-select * from information_schema.models where name ="litellm_handler_simple" ;
-
-select * from litellm_handler_simple where text='Once upon a time';
-
---simple completion with anthropic
-
-create model litellm_handler_simple_anthropic
-predict text
-using
-engine="litellm",
-model="claude-2",
-api_key="anthropic-api-key";
-
-select * from information_schema.models where name ="litellm_handler_simple_anthropic" ;
-
-select * from litellm_handler_simple_anthropic where text='Once upon a time';
+CREATE ML_ENGINE litellm
+FROM litellm;
+```
 
---completion using prompt_template on create
 
-create model litellm_handler_prompt_template
-predict text
-using
-engine="litellm",
-model="gpt-3.5-turbo",
-api_key="openai-api-key",
-prompt_template="write a story based on {{text}}";
+## Usage
 
-select * from information_schema.models where name ="litellm_handler_prompt_template" ;
+Here is how to create a model that answers questions:
 
-select * from litellm_handler_prompt_template where text='Once upon a time';
+```sql
+CREATE MOEL chat_model
path: mindsdb/integrations/handlers/litellm_handler/README.md
text: @user1:
This should be `CREATE MODEL` (the 'D' is missing).