Repository: mindsdb/mindsdb
Pull request: 💎💎GEMini updated
## Description

Please include a summary of the change and the issue it solves. 

Fixes #8699 



## Type of change

(Please delete options that are not relevant)

- [ ] 🐛 Bug fix (non-breaking change which fixes an issue)
- [x] ⚡ New feature (non-breaking change which adds functionality)
- [ ] 📢 Breaking change (fix or feature that would cause existing functionality not to work as expected)
- [x] 📄 This change requires a documentation update

## Verification Process

To ensure the changes are working as expected:

 - [ ]   Test Location: Specify the URL or path for testing.
 - [ ]   Verification Steps: Outline the steps or queries needed to validate the change. Include any data, configurations, or actions required to reproduce or see the new functionality.

## Additional Media:

- [x] I have attached a brief loom video or screenshots showcasing the new functionality or change.


https://github.com/mindsdb/mindsdb/assets/75653580/fe1af932-dcc5-4819-9d7c-912ab8bb8f1a



## Checklist:

- [x] My code follows the style guidelines(PEP 8) of MindsDB.
- [x] Necessary documentation updates are either made or tracked in issues.
- [ ] I have appropriately commented on my code, especially in complex areas.
- [ ] Relevant unit and integration tests are updated or added.





Problem statement:
Gemini Handler Updates
### Short description and motivation for the proposed feature

Add following features to gemini:
- vision mode
- prompt template mode
- embedding mode
- question with context
- question without context
- json struct


### Video or screenshots

_No response_

### Describe some possible solutions

_No response_

### Anything else?

_No response_

Hints from issue discussion:

Commit message: flake8 done

Patch to review:
diff --git a/mindsdb/integrations/handlers/google_gemini_handler/README.md b/mindsdb/integrations/handlers/google_gemini_handler/README.md
index 0944b2b829b..4c0a61846a0 100644
--- a/mindsdb/integrations/handlers/google_gemini_handler/README.md
+++ b/mindsdb/integrations/handlers/google_gemini_handler/README.md
@@ -6,35 +6,105 @@ Google Generative AI is a library that provides access to powerful language mode
 
 *Note:* Ensure you have the necessary API key for accessing the Google gen AI library. You can get your API key at https://makersuite.google.com/. 
 
+>> This Handler requires python>=3.9 to work
+>> (Default model_name is **gemini-pro** )
+
 # Example Usage
 
-Create a ML Engine with the new `google_gemini` engine.
+#### Create Gemini ML Engine
+```sql
+CREATE ML_ENGINE g
+FROM gemini
+USING
+    api_key = 'AI-i5-c001';
+```
 
+
+#### Create Gemini Pro Model (Prompt-Template)
 ```sql
-CREATE ML_ENGINE Gemini_ML_Engine
-FROM google_gemini
+CREATE MODEL gem_p
+PREDICT answer
 USING
-  api_key = 'cloud_api_key';
+    engine = 'g',
+    prompt_template = 'Product Description: {{description}}. Question: {{question}}. Answer:',
+    model_name = 'gemini-pro';
+```
+
+```sql
+SELECT answer
+FROM gem_p
+WHERE description = "
+What is Rabbit R1?
+The Rabbit R1 is a pocket-sized AI device that promises a simpler and more intuitive way to interact with technology. Instead of being app-driven, the device relies on an AI model called LAMB (large action model) to understand your instructions and complete tasks autonomously.
+The device has a bright orange body, and is small and lightweight with a touchscreen, scroll wheel, and a talk button. There is also a rotating camera that functions as eyes of the device.
+
+The Rabbit R1 runs on its own operating system, called the Rabbit OS, that eliminates the need for app stores and downloads, requiring only natural language voice input to navigate. The initial version supports integration with the likes of Uber, Spotify, and Amazon, with the AI able to train and learn using other apps in the future.
+"
+AND question = 'Given me bullet pointed features of product ?';
 ```
 
+
+
+#### Create Gemini Contextual Model (Column-based)
 ```sql
-CREATE MODEL mindsdb.Google_Gemini_test
+CREATE MODEL gemini_c
 PREDICT answer
 USING
-  column = 'question',
-  engine = 'Gemini_ML_Engine',
-  api_key = 'cloud_api_key',
-  model = 'gemini-pro'
+    engine = 'g',
+    question_column = 'question',
+    context_column = 'context',
+    model_name="gemini-pro";
+```
+```sql
+SELECT answer
+FROM gem_qc
+WHERE context = "Ashoka the Great was an Indian emperor of the Maurya Dynasty who ruled from 268 to 232 BCE. He is regarded as one of India's greatest emperors, known for his extensive empire, his efforts to spread Buddhism, and his commitment to non-violence and peaceful coexistence."
+AND question = 'Ashoka was from which dynasty?';
 ```
 
+
+#### Vision Mode Query
 ```sql
-SELECT question, answer
-FROM mindsdb.Google_Gemini_test
-WHERE question = 'What is the meaning of life?';
+CREATE MODEL gem_v
+PREDICT answer
+USING
+  engine = 'g',
+  mode = 'vision',
+  img_url = 'url',
+  ctx_column = 'context';
 ```
 
 ```sql
-SELECT t.question, m.answer
-FROM mindsdb.Google_Gemini_test as m
-JOIN files.question_table as t;
+SELECT *
+FROM gem_v
+WHERE  url = 'https://images.unsplash.com/photo-1589762738975-a6773160c7d7?q=80&w=1374&auto=format&fit=crop&ixlib=rb-4.0.3&ixid=M3wxMjA3fDB8MHxwaG90by1wYWdlfHx8fGVufDB8fHx8fA%3D%3D'
+AND context='Does this man is superhuman?';
 ```
+
+#### Embedding Mode 
+```sql
+CREATE MODEL gem_e
+PREDICT answer
+USING
+  engine = 'g',
+  mode = 'embedding',
+  model_name = 'models/embedding-001',    --- default for embedding mode
+  question_column = 'question',
+  context_column = 'context',
+  title_column = 'title';     --OPTIONAL
+```
+>> (**title_column** is optional and default model_name for embedding mode is **models/embedding-001'**)
+
+```sql
+SELECT  question, answer
+FROM gem_e
+WHERE  question = 'How many moon exist in the solar system?'
+USING 
+  type='document';
+```
+>>(Use ** type='document'** for storing vectors for later searching and **type='query'** for searching in already created vectors.)
+
+#### Describe Gemini Pro Model Metadata
+```sql
+DESCRIBE MODEL `MODEL_NAME`.metadata;
+```
\ No newline at end of file
diff --git a/mindsdb/integrations/handlers/google_gemini_handler/google_gemini_handler.py b/mindsdb/integrations/handlers/google_gemini_handler/google_gemini_handler.py
index d76994401a3..0db425e5229 100644
--- a/mindsdb/integrations/handlers/google_gemini_handler/google_gemini_handler.py
+++ b/mindsdb/integrations/handlers/google_gemini_handler/google_gemini_handler.py
@@ -1,11 +1,19 @@
 import os
 from typing import Dict, Optional
 
+from PIL import Image
+import requests
+import numpy as np
+from io import BytesIO
+import json
+import textwrap
 import google.generativeai as genai
 import pandas as pd
 from mindsdb.integrations.libs.base import BaseMLEngine
 from mindsdb.utilities import log
 from mindsdb.utilities.config import Config
+from mindsdb.integrations.libs.llm_utils import get_completed_prompts
+
 
 logger = log.getLogger(__name__)
 
@@ -19,55 +27,196 @@ class GoogleGeminiHandler(BaseMLEngine):
 
     def __init__(self, *args, **kwargs):
         super().__init__(*args, **kwargs)
-        self.default_chat_model = "gemini-pro"
-        self.supported_chat_models = ["gemini-pro"]
+        self.default_model = "gemini-pro"
+        self.default_embedding_model = 'models/embedding-001'
         self.generative = True
-        self.connection = None
-
-    def create(
-        self,
-        target: str,
-        df: Optional[pd.DataFrame] = None,
-        args: Optional[Dict] = None,
-    ) -> None:
-        if "model" not in args["using"]:
-            args["using"]["model"] = self.default_chat_model
-        elif args["using"]["model"] not in self.supported_chat_models:
+        self.mode = 'default'
+
+    @staticmethod
+    def create_validation(target, args=None, **kwargs):
+        if 'using' not in args:
             raise Exception(
-                f"Invalid chat model. Please use one of {self.supported_chat_models}"
+                "Gemini engine requires a USING clause! Refer to its documentation for more details."
             )
+        else:
+            args = args['using']
 
-        api_key = self._get_google_gemini_api_key(args)
-
-        try:
-            genai.configure(api_key=api_key)
-            model = genai.GenerativeModel(args["using"]["model"])
-            model.generate_content("test")
-        except Exception as e:
+        if (len(set(args.keys()) & {'img_url', 'question_column', 'prompt_template', 'json_struct', 'prompt'}) == 0):
             raise Exception(
-                f"{e}: Invalid api key please check your api key"
+                'One of `question_column`, `prompt_template` or `json_struct` is required for this engine.'
             )
 
-        args["using"]["api_key"] = api_key
+        keys_collection = [
+            ['prompt_template'],
+            ['question_column', 'context_column'],
+            ['prompt', 'user_column', 'assistant_column'],
+            ['json_struct'],
+            ['img_url', 'ctx_column']
+        ]
+        for keys in keys_collection:
+            if keys[0] in args and any(
+                x[0] in args for x in keys_collection if x != keys
+            ):
+                raise Exception(
+                    textwrap.dedent(
+                        '''\
+                    Please provide one of
+                        1) a `prompt_template`
+                        2) a `question_column` and an optional `context_column`
+                        3) a `json_struct`
+                        4) a `prompt' and 'user_column' and 'assistant_column`
+                        5) a `img_url` and optional `ctx_column` for mode=`vision`
+                '''
+                    )
+                )
+
+        # for all args that are not expected, raise an error
+        known_args = set()
+        # flatten of keys_collection
+        for keys in keys_collection:
+            known_args = known_args.union(set(keys))
+
+        # TODO: need a systematic way to maintain a list of known args
+        known_args = known_args.union(
+            {
+                "target",
+                "model_name",
+                "mode",
+                'title_column',
+                "predict_params",
+                "type",
+                "max_tokens",
+                "temperature",
+                "api_key",
+            }
+        )
+
+        unknown_args = set(args.keys()) - known_args
+        if unknown_args:
+            # return a list of unknown args as a string
+            raise Exception(
+                f"Unknown arguments: {', '.join(unknown_args)}.\n Known arguments are: {', '.join(known_args)}"
+            )
 
-        self.model_storage.json_set("args", args)
+    def create(self, target, args=None, **kwargs):
+        args = args['using']
+        args['target'] = target
+        self.model_storage.json_set('args', args)
 
     def predict(
         self, df: Optional[pd.DataFrame] = None, args: Optional[Dict] = None
     ) -> pd.DataFrame:
-        args = self.model_storage.json_get("args")
-        api_key = args["using"]["api_key"]
+        pred_args = args['predict_params'] if args else {}
+        args = self.model_storage.json_get('args')
+        df = df.reset_index(drop=True)
+
+        if pred_args.get('prompt_template', False):
+            base_template = pred_args[
+                'prompt_template'
+            ]  # override with predict-time template if available
+        elif args.get('prompt_template', False):
+            base_template = args['prompt_template']
+        else:
+            base_template = None
+
+        # Embedding Mode
+        if args.get('mode') == 'embedding':
+            args['type'] = pred_args.get('type', 'query')
+            return self.embedding_worker(args, df)
+
+        elif args.get('mode') == 'vision':
+            return self.vision_worker(args, df)
+
+        elif args.get('mode') == 'conversational':
+            # Enable chat mode using
+            # https://ai.google.dev/tutorials/python_quickstart#chat_conversations
+            # OR
+            # https://github.com/google/generative-ai-python?tab=readme-ov-file#developers-who-use-the-palm-api
+            pass
+
+        else:
+            if args.get('prompt_template', False):
+                prompts, empty_prompt_ids = get_completed_prompts(
+                                                    base_template,
+                                                    df)
+
+            elif args.get('context_column', False):
+                empty_prompt_ids = np.where(
+                    df[[args['context_column'], args['question_column']]]
+                    .isna()
+                    .all(axis=1)
+                    .values
+                )[0]
+                contexts = list(df[args['context_column']].apply(lambda x: str(x)))
+                questions = list(df[args['question_column']].apply(lambda x: str(x)))
+                prompts = [
+                    f'Give only answer for: \nContext: {c}\nQuestion: {q}\nAnswer: '
+                    for c, q in zip(contexts, questions)
+                ]
+
+            elif args.get('json_struct', False):
+                empty_prompt_ids = np.where(
+                    df[[args['input_text']]].isna().all(axis=1).values
+                )[0]
+                prompts = []
+                for i in df.index:
+                    if 'json_struct' in df.columns:
+                        if isinstance(df['json_struct'][i], str):
+                            df['json_struct'][i] = json.loads(df['json_struct'][i])
+                        json_struct = ''
+                        for ind, val in enumerate(df['json_struct'][i].values()):
+                            json_struct = json_struct + f'{ind}. {val}\n'
+                    else:
+                        json_struct = ''
+                        for ind, val in enumerate(args['json_struct'].values()):
+                            json_struct = json_struct + f'{ind + 1}. {val}\n'
+
+                    p = textwrap.dedent(
+                        f'''\
+                        Using text starting after 'The text is:', give exactly {len(args['json_struct'])} answers to the questions:
+                        {{{{json_struct}}}}
+
+                        Answers should be in the same order as the questions.
+                        Answer should be in form of one JSON Object eg. {"{'key':'value',..}"} where key=question and value=answer.
+                        If there is no answer to the question in the text, put a -.
+                        Answers should be as short as possible, ideally 1-2 words (unless otherwise specified).
+
+                        The text is:
+                        {{{{{args['input_text']}}}}}
+                    '''
+                    )
+                    p = p.replace('{{json_struct}}', json_struct)
+                    for column in df.columns:
+                        if column == 'json_struct':
+                            continue
+                        p = p.replace(f'{{{{{column}}}}}', str(df[column][i]))
+                    prompts.append(p)
+            elif 'prompt' in args:
+                empty_prompt_ids = []
+                prompts = list(df[args['user_column']])
+            else:
+                empty_prompt_ids = np.where(
+                    df[[args['question_column']]].isna().all(axis=1).values
+                )[0]
+                prompts = list(df[args['question_column']].apply(lambda x: str(x)))
+
+        # remove prompts without signal from completion queue
+        prompts = [
+            j for i, j in enumerate(prompts) if i not in empty_prompt_ids
+        ]
+
+        api_key = self._get_google_gemini_api_key(args)
         genai.configure(api_key=api_key)
 
-        input_column = args["using"]["column"]
-        if input_column not in df.columns:
-            raise RuntimeError(f'Column "{input_column}" not found in input data')
+        model = genai.GenerativeModel(
+            args.get('model_name', self.default_model)
+        )
+        results = []
+        for m in prompts:
+            results.append(model.generate_content(m).text)
 
-        self.connection = genai.GenerativeModel(args["using"]["model"])
-        result_df = pd.DataFrame()
-        result_df["predictions"] = df[input_column].apply(self.predict_answer)
-        result_df = result_df.rename(columns={"predictions": args["target"]})
-        return result_df
+        pred_df = pd.DataFrame(results, columns=[args['target']])
+        return pred_df
 
     def _get_google_gemini_api_key(self, args, strict=True):
         """
@@ -78,8 +227,8 @@ def _get_google_gemini_api_key(self, args, strict=True):
             4. google_gemini.api_key setting in config.json
         """
 
-        if "api_key" in args["using"]:
-            return args["using"]["api_key"]
+        if "api_key" in args:
+            return args["api_key"]
         # 2
         connection_args = self.engine_storage.get_connection_args()
         if "api_key" in connection_args:
@@ -100,14 +249,69 @@ def _get_google_gemini_api_key(self, args, strict=True):
                  or re-create this model and pass the API key with `USING` syntax.'
             )
 
-    def predict_answer(self, text):
-        """
-        connects with google generative AI api to predict the answer for the particular question
+    def embedding_worker(self, args: Dict, df: pd.DataFrame):
+        if args.get('question_column'):
+            prompts = list(df[args['question_column']].apply(lambda x: str(x)))
+            if args.get('title_column', None):
+                titles = list(df[args['title_column']].apply(lambda x: str(x)))
+            else:
+                titles = None
 
-        """
+            api_key = self._get_google_gemini_api_key(args)
+            genai.configure(api_key=api_key)
+            model_name = args.get('model_name', self.default_embedding_model)
+            task_type = args.get('type')
+            task_type = f'retrieval_{task_type}'
 
-        completion = self.connection.generate_content(
-            text
-        )
+            if task_type == 'retrieval_query':
+                results = [str(genai.embed_content(model=model_name, content=query, task_type=task_type)['embedding']) for query in prompts]
+            elif titles:
+                results = [str(genai.embed_content(model=model_name, content=doc, task_type=task_type, title=title)['embedding']) for title, doc in zip(titles, prompts)]
+            else:
+                results = [str(genai.embed_content(model=model_name, content=doc, task_type=task_type)['embedding']) for doc in prompts]
+
+            pred_df = pd.DataFrame(results, columns=[args['target']])
+            return pred_df
+        else:
+            raise Exception('Embedding mode needs a question_column')
+
+    def vision_worker(self, args: Dict, df: pd.DataFrame):
+        if args.get('img_url'):
+            urls = list(df[args['img_url']].apply(lambda x: str(x)))
+
+        else:
+            raise Exception('Vision mode needs a img_url')
+
+        prompts = None
+        if args.get('ctx_column'):
+            prompts = list(df[args['ctx_column']].apply(lambda x: str(x)))
+
+        api_key = self._get_google_gemini_api_key(args)
+        genai.configure(api_key=api_key)
+        model = genai.GenerativeModel('gemini-pro-vision')
+        imgs = [Image.open(BytesIO(requests.get(url).content)) for url in urls]
+        if prompts:
+            results = [model.generate_content([img, text]).text for img, text in zip(imgs, prompts)]
+        else:
+            results = [model.generate_content(img).text for img in imgs]
+
+        pred_df = pd.DataFrame(results, columns=[args['target']])
+
+        return pred_df
+
+    def describe(self, attribute: Optional[str] = None) -> pd.DataFrame:
+
+        args = self.model_storage.json_get('args')
+
+        if attribute == 'args':
+            return pd.DataFrame(args.items(), columns=['key', 'value'])
+        elif attribute == 'metadata':
+            api_key = self._get_google_gemini_api_key(args)
+            genai.configure(api_key=api_key)
+            model_name = args.get('model_name', self.default_model)
 
-        return completion.text
+            meta = genai.get_model(f'models/{model_name}').__dict__
+            return pd.DataFrame(meta.items(), columns=['key', 'value'])
+        else:
+            tables = ['args', 'metadata']
+            return pd.DataFrame(tables, columns=['tables'])
diff --git a/mindsdb/integrations/handlers/google_gemini_handler/requirements.txt b/mindsdb/integrations/handlers/google_gemini_handler/requirements.txt
index a7af1ddb0b0..9ab505e6a5b 100644
--- a/mindsdb/integrations/handlers/google_gemini_handler/requirements.txt
+++ b/mindsdb/integrations/handlers/google_gemini_handler/requirements.txt
@@ -1 +1,2 @@
-google-generativeai==0.3.1
\ No newline at end of file
+google-generativeai==0.3.2
+Pillow==10.1.0


Code review comments:
diff_hunk: @@ -6,35 +6,105 @@ Google Generative AI is a library that provides access to powerful language mode
 
 *Note:* Ensure you have the necessary API key for accessing the Google gen AI library. You can get your API key at https://makersuite.google.com/. 
 
+>> This Handler requires python>=3.9 to work
+>> (Default model_name is **gemini-pro** )
+
 # Example Usage
 
-Create a ML Engine with the new `google_gemini` engine.
+#### Create Gemini ML Engine
+```sql
+CREATE ML_ENGINE g
+FROM gemini
+USING
+    api_key = 'AI-i5-c001';
+```
 
+
+#### Create Gemini Pro Model (Prompt-Template)
 ```sql
-CREATE ML_ENGINE Gemini_ML_Engine
-FROM google_gemini
+CREATE MODEL gem_p
+PREDICT answer
 USING
-  api_key = 'cloud_api_key';
+    engine = 'g',
+    prompt_template = 'Product Description: {{description}}. Question: {{question}}. Answer:',
+    model_name = 'gemini-pro';
+```
+
+```sql
+SELECT answer
+FROM gem_p
+WHERE description = "
+What is Rabbit R1?
+The Rabbit R1 is a pocket-sized AI device that promises a simpler and more intuitive way to interact with technology. Instead of being app-driven, the device relies on an AI model called LAMB (large action model) to understand your instructions and complete tasks autonomously.
+The device has a bright orange body, and is small and lightweight with a touchscreen, scroll wheel, and a talk button. There is also a rotating camera that functions as eyes of the device.
+
+The Rabbit R1 runs on its own operating system, called the Rabbit OS, that eliminates the need for app stores and downloads, requiring only natural language voice input to navigate. The initial version supports integration with the likes of Uber, Spotify, and Amazon, with the AI able to train and learn using other apps in the future.
+"
+AND question = 'Given me bullet pointed features of product ?';
 ```
 
+
+
+#### Create Gemini Contextual Model (Column-based)
 ```sql
-CREATE MODEL mindsdb.Google_Gemini_test
+CREATE MODEL gemini_c
 PREDICT answer
 USING
-  column = 'question',
-  engine = 'Gemini_ML_Engine',
-  api_key = 'cloud_api_key',
-  model = 'gemini-pro'
+    engine = 'g',
+    question_column = 'question',
+    context_column = 'context',
+    model_name="gemini-pro";
+```
+```sql
+SELECT answer
+FROM gem_qc
+WHERE context = "Ashoka the Great was an Indian emperor of the Maurya Dynasty who ruled from 268 to 232 BCE. He is regarded as one of India's greatest emperors, known for his extensive empire, his efforts to spread Buddhism, and his commitment to non-violence and peaceful coexistence."
+AND question = 'Ashoka was from which dynasty?';
 ```
 
+
+#### Vision Mode Query
 ```sql
-SELECT question, answer
-FROM mindsdb.Google_Gemini_test
-WHERE question = 'What is the meaning of life?';
+CREATE MODEL gem_v
+PREDICT answer
+USING
+  engine = 'g',
+  mode = 'vision',
+  img_url = 'url',
+  ctx_column = 'context';
 ```
 
 ```sql
-SELECT t.question, m.answer
-FROM mindsdb.Google_Gemini_test as m
-JOIN files.question_table as t;
+SELECT *
+FROM gem_v
+WHERE  url = 'https://images.unsplash.com/photo-1589762738975-a6773160c7d7?q=80&w=1374&auto=format&fit=crop&ixlib=rb-4.0.3&ixid=M3wxMjA3fDB8MHxwaG90by1wYWdlfHx8fGVufDB8fHx8fA%3D%3D'
+AND context='Does this man is superhuman?';
path: mindsdb/integrations/handlers/google_gemini_handler/README.md
text: @user1:
Let's rephrase this question as `"Is this man a superhuman?'`

diff_hunk: @@ -6,35 +6,105 @@ Google Generative AI is a library that provides access to powerful language mode
 
 *Note:* Ensure you have the necessary API key for accessing the Google gen AI library. You can get your API key at https://makersuite.google.com/. 
 
+>> This Handler requires python>=3.9 to work
+>> (Default model_name is **gemini-pro** )
+
 # Example Usage
 
-Create a ML Engine with the new `google_gemini` engine.
+#### Create Gemini ML Engine
+```sql
+CREATE ML_ENGINE g
+FROM gemini
+USING
+    api_key = 'AI-i5-c001';
+```
 
+
+#### Create Gemini Pro Model (Prompt-Template)
 ```sql
-CREATE ML_ENGINE Gemini_ML_Engine
-FROM google_gemini
+CREATE MODEL gem_p
+PREDICT answer
 USING
-  api_key = 'cloud_api_key';
+    engine = 'g',
+    prompt_template = 'Product Description: {{description}}. Question: {{question}}. Answer:',
+    model_name = 'gemini-pro';
+```
+
+```sql
+SELECT answer
+FROM gem_p
+WHERE description = "
+What is Rabbit R1?
+The Rabbit R1 is a pocket-sized AI device that promises a simpler and more intuitive way to interact with technology. Instead of being app-driven, the device relies on an AI model called LAMB (large action model) to understand your instructions and complete tasks autonomously.
+The device has a bright orange body, and is small and lightweight with a touchscreen, scroll wheel, and a talk button. There is also a rotating camera that functions as eyes of the device.
+
+The Rabbit R1 runs on its own operating system, called the Rabbit OS, that eliminates the need for app stores and downloads, requiring only natural language voice input to navigate. The initial version supports integration with the likes of Uber, Spotify, and Amazon, with the AI able to train and learn using other apps in the future.
+"
+AND question = 'Given me bullet pointed features of product ?';
 ```
 
+
+
+#### Create Gemini Contextual Model (Column-based)
 ```sql
-CREATE MODEL mindsdb.Google_Gemini_test
+CREATE MODEL gemini_c
 PREDICT answer
 USING
-  column = 'question',
-  engine = 'Gemini_ML_Engine',
-  api_key = 'cloud_api_key',
-  model = 'gemini-pro'
+    engine = 'g',
+    question_column = 'question',
+    context_column = 'context',
+    model_name="gemini-pro";
+```
+```sql
+SELECT answer
+FROM gem_qc
+WHERE context = "Ashoka the Great was an Indian emperor of the Maurya Dynasty who ruled from 268 to 232 BCE. He is regarded as one of India's greatest emperors, known for his extensive empire, his efforts to spread Buddhism, and his commitment to non-violence and peaceful coexistence."
+AND question = 'Ashoka was from which dynasty?';
 ```
 
+
+#### Vision Mode Query
 ```sql
-SELECT question, answer
-FROM mindsdb.Google_Gemini_test
-WHERE question = 'What is the meaning of life?';
+CREATE MODEL gem_v
+PREDICT answer
+USING
+  engine = 'g',
+  mode = 'vision',
+  img_url = 'url',
+  ctx_column = 'context';
 ```
 
 ```sql
-SELECT t.question, m.answer
-FROM mindsdb.Google_Gemini_test as m
-JOIN files.question_table as t;
+SELECT *
+FROM gem_v
+WHERE  url = 'https://images.unsplash.com/photo-1589762738975-a6773160c7d7?q=80&w=1374&auto=format&fit=crop&ixlib=rb-4.0.3&ixid=M3wxMjA3fDB8MHxwaG90by1wYWdlfHx8fGVufDB8fHx8fA%3D%3D'
+AND context='Does this man is superhuman?';
 ```
+
+#### Embedding Mode 
+```sql
+CREATE MODEL gem_e
+PREDICT answer
+USING
+  engine = 'g',
+  mode = 'embedding',
+  model_name = 'models/embedding-001',    --- default for embedding mode
+  question_column = 'question',
+  context_column = 'context',
+  title_column = 'title';     --OPTIONAL
+```
+>> (**title_column** is optional and default model_name for embedding mode is **models/embedding-001'**)
+
+```sql
+SELECT  question, answer
+FROM gem_e
+WHERE  question = 'How many moon exist in the solar system?'
path: mindsdb/integrations/handlers/google_gemini_handler/README.md
text: @user1:
Let's rephrase this question as `"How many moons are there in the solar system?"`

diff_hunk: @@ -6,35 +6,105 @@ Google Generative AI is a library that provides access to powerful language mode
 
 *Note:* Ensure you have the necessary API key for accessing the Google gen AI library. You can get your API key at https://makersuite.google.com/. 
 
+>> This Handler requires python>=3.9 to work
+>> (Default model_name is **gemini-pro** )
+
 # Example Usage
 
-Create a ML Engine with the new `google_gemini` engine.
+#### Create Gemini ML Engine
+```sql
+CREATE ML_ENGINE g
+FROM gemini
+USING
+    api_key = 'AI-i5-c001';
+```
 
+
+#### Create Gemini Pro Model (Prompt-Template)
 ```sql
-CREATE ML_ENGINE Gemini_ML_Engine
-FROM google_gemini
+CREATE MODEL gem_p
+PREDICT answer
 USING
-  api_key = 'cloud_api_key';
+    engine = 'g',
+    prompt_template = 'Product Description: {{description}}. Question: {{question}}. Answer:',
+    model_name = 'gemini-pro';
+```
+
+```sql
+SELECT answer
+FROM gem_p
+WHERE description = "
+What is Rabbit R1?
+The Rabbit R1 is a pocket-sized AI device that promises a simpler and more intuitive way to interact with technology. Instead of being app-driven, the device relies on an AI model called LAMB (large action model) to understand your instructions and complete tasks autonomously.
+The device has a bright orange body, and is small and lightweight with a touchscreen, scroll wheel, and a talk button. There is also a rotating camera that functions as eyes of the device.
+
+The Rabbit R1 runs on its own operating system, called the Rabbit OS, that eliminates the need for app stores and downloads, requiring only natural language voice input to navigate. The initial version supports integration with the likes of Uber, Spotify, and Amazon, with the AI able to train and learn using other apps in the future.
+"
+AND question = 'Given me bullet pointed features of product ?';
path: mindsdb/integrations/handlers/google_gemini_handler/README.md
text: @user1:
Let's rephrase this question as `"What are some key feature bullet points of this product?"`

diff_hunk: @@ -1 +1,2 @@
-google-generativeai==0.3.1
\ No newline at end of file
+google-generativeai==0.3.2
+Pillow==10.1.0
path: mindsdb/integrations/handlers/google_gemini_handler/requirements.txt
text: @user1:
For my understanding, is this version pin necessary for `Pillow`?

@author:
no i think it not necessary, therefore i am removing it