based on the documentation, add file uploading to the html page

Call media.upload to upload a file using the File API. The following code uploads a document file and then uses the file in a call to models.generateContent.


NUM_BYTES=$(wc -c < "${PDF_PATH}")
DISPLAY_NAME=TEXT
tmp_header_file=upload-header.tmp

# Initial resumable request defining metadata.
# The upload url is in the response headers dump them to a file.
curl "${BASE_URL}/upload/v1beta/files?key=${GOOGLE_API_KEY}" \
  -D upload-header.tmp \
  -H "X-Goog-Upload-Protocol: resumable" \
  -H "X-Goog-Upload-Command: start" \
  -H "X-Goog-Upload-Header-Content-Length: ${NUM_BYTES}" \
  -H "X-Goog-Upload-Header-Content-Type: application/pdf" \
  -H "Content-Type: application/json" \
  -d "{'file': {'display_name': '${DISPLAY_NAME}'}}" 2> /dev/null

upload_url=$(grep -i "x-goog-upload-url: " "${tmp_header_file}" | cut -d" " -f2 | tr -d "\r")
rm "${tmp_header_file}"

# Upload the actual bytes.
curl "${upload_url}" \
  -H "Content-Length: ${NUM_BYTES}" \
  -H "X-Goog-Upload-Offset: 0" \
  -H "X-Goog-Upload-Command: upload, finalize" \
  --data-binary "@${PDF_PATH}" 2> /dev/null > file_info.json

file_uri=$(jq ".file.uri" file_info.json)
echo file_uri=$file_uri

# Now generate content using that file
curl "https://generativelanguage.googleapis.com/v1beta/models/gemini-1.5-flash:generateContent?key=$GOOGLE_API_KEY" \
    -H 'Content-Type: application/json' \
    -X POST \
    -d '{
      "contents": [{
        "parts":[
          {"text": "Can you add a few more lines to this poem?"},
          {"file_data":{"mime_type": "application/pdf", "file_uri": '$file_uri'}}]
        }]
       }' 2> /dev/null > response.json

cat response.json
echo

jq ".candidates[].content.parts[].text" response.json

Get metadata for a file
You can verify the API successfully stored the uploaded file and get its metadata by calling files.get. Only the name (and by extension, the uri) are unique.


name=$(jq ".file.name" file_info.json)
# Get the file of interest to check state
curl https://generativelanguage.googleapis.com/v1beta/files/$name > file_info.json
# Print some information about the file you got
name=$(jq ".file.name" file_info.json)
echo name=$name
file_uri=$(jq ".file.uri" file_info.json)
echo file_uri=$file_uri


Upload an image and generate content
Use the media.upload method of the File API to upload an image of any size. (Always use the File API when the combination of files and system instructions that you intend to send is larger than 20 MB.)

Note: The File API lets you store up to 20 GB of files per project, with a per-file maximum size of 2 GB. Files are stored for 48 hours. They can be accessed in that period with your API key, but cannot be downloaded from the API. It is available at no cost in all regions where the Gemini API is available.
After uploading the file, you can make GenerateContent requests that reference the File API URI. Select the generative model and provide it with a text prompt and the uploaded image.


MIME_TYPE=$(file -b --mime-type "${IMG_PATH_2}")
NUM_BYTES=$(wc -c < "${IMG_PATH_2}")
DISPLAY_NAME=TEXT

tmp_header_file=upload-header.tmp

# Initial resumable request defining metadata.
# The upload url is in the response headers dump them to a file.
curl "${BASE_URL}/upload/v1beta/files?key=${GOOGLE_API_KEY}" \
  -D upload-header.tmp \
  -H "X-Goog-Upload-Protocol: resumable" \
  -H "X-Goog-Upload-Command: start" \
  -H "X-Goog-Upload-Header-Content-Length: ${NUM_BYTES}" \
  -H "X-Goog-Upload-Header-Content-Type: ${MIME_TYPE}" \
  -H "Content-Type: application/json" \
  -d "{'file': {'display_name': '${DISPLAY_NAME}'}}" 2> /dev/null

upload_url=$(grep -i "x-goog-upload-url: " "${tmp_header_file}" | cut -d" " -f2 | tr -d "\r")
rm "${tmp_header_file}"

# Upload the actual bytes.
curl "${upload_url}" \
  -H "Content-Length: ${NUM_BYTES}" \
  -H "X-Goog-Upload-Offset: 0" \
  -H "X-Goog-Upload-Command: upload, finalize" \
  --data-binary "@${IMG_PATH_2}" 2> /dev/null > file_info.json

file_uri=$(jq ".file.uri" file_info.json)
echo file_uri=$file_uri

# Now generate content using that file
curl "https://generativelanguage.googleapis.com/v1beta/models/gemini-1.5-flash:generateContent?key=$GOOGLE_API_KEY" \
    -H 'Content-Type: application/json' \
    -X POST \
    -d '{
      "contents": [{
        "parts":[
          {"text": "Can you tell me about the instruments in this photo?"},
          {"file_data":
            {"mime_type": "image/jpeg", 
            "file_uri": '$file_uri'}
        }]
        }]
       }' 2> /dev/null > response.json

cat response.json
echo

jq ".candidates[].content.parts[].text" response.json

Verify image file upload and get metadata
You can verify the API successfully stored the uploaded file and get its metadata by calling files.get. Only the name (and by extension, the uri) are unique.


name=$(jq ".file.name" file_info.json)
# Get the file of interest to check state
curl https://generativelanguage.googleapis.com/v1beta/files/$name > file_info.json
# Print some information about the file you got
name=$(jq ".file.name" file_info.json)
echo name=$name
file_uri=$(jq ".file.uri" file_info.json)
echo file_uri=$file_uri

Prompting with video
In this tutorial, you will upload a video using the File API and generate content based on those images.

Note: The File API is required to upload video files, due to their size. However, the File API is only available for Python, Node.js, Go, and REST.
Technical details (video)
Gemini 1.5 Pro and Flash support up to approximately an hour of video data.

Video must be in one of the following video format MIME types:

video/mp4
video/mpeg
video/mov
video/avi
video/x-flv
video/mpg
video/webm
video/wmv
video/3gpp
The File API service extracts image frames from videos at 1 frame per second (FPS) and audio at 1Kbps, single channel, adding timestamps every second. These rates are subject to change in the future for improvements in inference.

Note: The details of fast action sequences may be lost at the 1 FPS frame sampling rate. Consider slowing down high-speed clips for improved inference quality.
Individual frames are 258 tokens, and audio is 32 tokens per second. With metadata, each second of video becomes ~300 tokens, which means a 1M context window can fit slightly less than an hour of video.

To ask questions about time-stamped locations, use the format MM:SS, where the first two digits represent minutes and the last two digits represent seconds.

For best results:

Use one video per prompt.
If using a single video, place the text prompt after the video.
Upload a video and generate content
Note: The File API lets you store up to 20 GB of files per project, with a per-file maximum size of 2 GB. Files are stored for 48 hours. They can be accessed in that period with your API key, but they cannot be downloaded using any API. It is available at no cost in all regions where the Gemini API is available.

MIME_TYPE=$(file -b --mime-type "${VIDEO_PATH}")
NUM_BYTES=$(wc -c < "${VIDEO_PATH}")
DISPLAY_NAME=VIDEO_PATH

# Initial resumable request defining metadata.
# The upload url is in the response headers dump them to a file.
curl "${BASE_URL}/upload/v1beta/files?key=${GOOGLE_API_KEY}" \
  -D upload-header.tmp \
  -H "X-Goog-Upload-Protocol: resumable" \
  -H "X-Goog-Upload-Command: start" \
  -H "X-Goog-Upload-Header-Content-Length: ${NUM_BYTES}" \
  -H "X-Goog-Upload-Header-Content-Type: ${MIME_TYPE}" \
  -H "Content-Type: application/json" \
  -d "{'file': {'display_name': '${DISPLAY_NAME}'}}" 2> /dev/null

upload_url=$(grep -i "x-goog-upload-url: " "${tmp_header_file}" | cut -d" " -f2 | tr -d "\r")
rm "${tmp_header_file}"

# Upload the actual bytes.
curl "${upload_url}" \
  -H "Content-Length: ${NUM_BYTES}" \
  -H "X-Goog-Upload-Offset: 0" \
  -H "X-Goog-Upload-Command: upload, finalize" \
  --data-binary "@${VIDEO_PATH}" 2> /dev/null > file_info.json

file_uri=$(jq ".file.uri" file_info.json)
echo file_uri=$file_uri

state=$(jq ".file.state" file_info.json)
echo state=$state

# Ensure the state of the video is 'ACTIVE'
while [[ "($state)" = *"PROCESSING"* ]];
do
  echo "Processing video..."
  sleep 5
  # Get the file of interest to check state
  curl https://generativelanguage.googleapis.com/v1beta/files/$name > file_info.json
  state=$(jq ".file.state" file_info.json)
done

# Now generate content using that file
curl "https://generativelanguage.googleapis.com/v1beta/models/gemini-1.5-flash:generateContent?key=$GOOGLE_API_KEY" \
    -H 'Content-Type: application/json' \
    -X POST \
    -d '{
      "contents": [{
        "parts":[
          {"text": "Describe this video clip"},
          {"file_data":{"mime_type": "video/mp4", "file_uri": '$file_uri'}}]
        }]
       }' 2> /dev/null > response.json

cat response.json
echo

jq ".candidates[].content.parts[].text" response.json

Upload an audio file and generate content
You can use the File API to upload an audio file of any size. Always use the File API when the total request size (including the files, text prompt, system instructions, etc.) is larger than 20 MB.

Note: The File API lets you store up to 20 GB of files per project, with a per-file maximum size of 2 GB. Files are stored for 48 hours. They can be accessed in that period with your API key, but cannot be downloaded from the API. The File API is available at no cost in all regions where the Gemini API is available.
Call media.upload to upload a file using the File API. The following code uploads an audio file and then uses the file in a call to models.generateContent.


MIME_TYPE=$(file -b --mime-type "${AUDIO_PATH}")
NUM_BYTES=$(wc -c < "${AUDIO_PATH}")
DISPLAY_NAME=AUDIO

tmp_header_file=upload-header.tmp

# Initial resumable request defining metadata.
# The upload url is in the response headers dump them to a file.
curl "${BASE_URL}/upload/v1beta/files?key=${GOOGLE_API_KEY}" \
  -D upload-header.tmp \
  -H "X-Goog-Upload-Protocol: resumable" \
  -H "X-Goog-Upload-Command: start" \
  -H "X-Goog-Upload-Header-Content-Length: ${NUM_BYTES}" \
  -H "X-Goog-Upload-Header-Content-Type: ${MIME_TYPE}" \
  -H "Content-Type: application/json" \
  -d "{'file': {'display_name': '${DISPLAY_NAME}'}}" 2> /dev/null

upload_url=$(grep -i "x-goog-upload-url: " "${tmp_header_file}" | cut -d" " -f2 | tr -d "\r")
rm "${tmp_header_file}"

# Upload the actual bytes.
curl "${upload_url}" \
  -H "Content-Length: ${NUM_BYTES}" \
  -H "X-Goog-Upload-Offset: 0" \
  -H "X-Goog-Upload-Command: upload, finalize" \
  --data-binary "@${AUDIO_PATH}" 2> /dev/null > file_info.json

file_uri=$(jq ".file.uri" file_info.json)
echo file_uri=$file_uri

# Now generate content using that file
curl "https://generativelanguage.googleapis.com/v1beta/models/gemini-1.5-flash:generateContent?key=$GOOGLE_API_KEY" \
    -H 'Content-Type: application/json' \
    -X POST \
    -d '{
      "contents": [{
        "parts":[
          {"text": "Describe this audio clip"},
          {"file_data":{"mime_type": "audio/mp3", "file_uri": '$file_uri'}}]
        }]
       }' 2> /dev/null > response.json

cat response.json
echo

jq ".candidates[].content.parts[].text" response.json

Get metadata for a file
You can verify the API successfully stored the uploaded file and get its metadata by calling files.get.


name=$(jq ".file.name" file_info.json)
# Get the file of interest to check state
curl https://generativelanguage.googleapis.com/v1beta/files/$name > file_info.json
# Print some information about the file you got
name=$(jq ".file.name" file_info.json)
echo name=$name
file_uri=$(jq ".file.uri" file_info.json)
echo file_uri=$file_uri


<!DOCTYPE html>
<html>
<head>
  <title>Google</title>
  <style>
    body {
      font-family: sans-serif;
      margin: 0;
      padding: 0;
      display: flex;
      justify-content: center;
      align-items: center;
      min-height: 100vh;
      background-color: #f0f0f0;
    }

    .container {
      display: flex;
      flex-direction: column;
      width: 90%;
      max-width: 600px;
      height: 80vh;
      background-color: #fff;
      border: 1px solid #ccc;
      border-radius: 8px;
      box-shadow: 0 2px 10px rgba(0,0,0,0.1);
      overflow: hidden;
    }

    /* Header for Toggle Buttons */
    .header {
      padding: 10px 20px;
      background-color: #f9f9f9;
      border-bottom: 1px solid #ddd;
      display: flex;
      flex-wrap: wrap;
      justify-content: space-between;
      align-items: center;
    }

    .toggle-container {
      display: flex;
      align-items: center;
      margin: 5px 0;
    }

    .toggle-container label {
      margin-right: 10px;
      font-size: 14px;
    }

    /* Toggle switch styles */
    .switch {
      position: relative;
      display: inline-block;
      width: 50px;
      height: 28px;
      margin-left: 10px;
    }

    .switch input {
      opacity: 0;
      width: 0;
      height: 0;
    }

    .slider {
      position: absolute;
      cursor: pointer;
      top: 0;
      left: 0;
      right: 0;
      bottom: 0;
      background-color: #ccc;
      transition: .4s;
      border-radius: 28px;
    }

    .slider:before {
      position: absolute;
      content: "";
      height: 22px;
      width: 22px;
      left: 3px;
      bottom: 3px;
      background-color: white;
      transition: .4s;
      border-radius: 50%;
    }

    input:checked + .slider {
      background-color: #2196F3;
    }

    input:checked + .slider:before {
      transform: translateX(22px);
    }

    /* Chat Container */
    #chat-container {
      flex: 1;
      padding: 15px;
      overflow-y: auto;
      background-color: #fafafa;
    }

    .message {
      padding: 10px 15px;
      margin-bottom: 10px;
      border-radius: 18px;
      max-width: 80%;
      word-wrap: break-word;
      position: relative;
      clear: both;
    }

    .user-message {
      background-color: #e0f7fa;
      align-self: flex-end;
      margin-left: auto;
      text-align: right;
    }

    .bot-message {
      background-color: #f1f8e9;
      align-self: flex-start;
      margin-right: auto;
    }

    /* Input Area */
    #input-area {
      display: flex;
      padding: 10px 20px;
      border-top: 1px solid #ddd;
      background-color: #f9f9f9;
    }

    #input-box {
      flex-grow: 1;
      padding: 10px;
      border: 1px solid #ccc;
      border-radius: 20px;
      resize: none;
      font-size: 14px;
      outline: none;
    }

    #send-button {
      padding: 10px 20px;
      margin-left: 10px;
      background-color: #007bff;
      color: #fff;
      border: none;
      border-radius: 20px;
      cursor: pointer;
      font-size: 14px;
      transition: background-color 0.3s;
    }

    #send-button:hover {
      background-color: #0056b3;
    }

    /* Typing Indicator */
    #typing-indicator {
      padding: 5px 20px;
      font-style: italic;
      color: #999;
      background-color: #f9f9f9;
      border-top: 1px solid #ddd;
      display: none;
    }

    /* Responsive Adjustments */
    @media (max-width: 768px) {
      .container {
        width: 95%;
        height: 90vh;
      }
    }

    @media (max-width: 480px) {
      .header {
        flex-direction: column;
        align-items: flex-start;
      }

      .toggle-container {
        width: 100%;
        justify-content: space-between;
      }

      #send-button {
        padding: 10px;
        margin-left: 5px;
      }
    }
  </style>
</head>

<body>
  <div class="container">
    <!-- Header with Toggle Buttons -->
    <div class="header">
      <!-- Toggle Button for Super Smart Mode -->
      <div class="toggle-container">
        <label for="superSmartToggle">Super Smart Mode</label>
        <label class="switch">
          <input type="checkbox" id="superSmartToggle">
          <span class="slider"></span>
        </label>
      </div>

      <!-- Toggle Button for Fast/Smart Model Switching -->
      <div class="toggle-container">
        <label for="modelToggle">Fast Mode</label>
        <label class="switch">
          <input type="checkbox" id="modelToggle">
          <span class="slider"></span>
        </label>
      </div>
    </div>

    <!-- Chat Messages -->
    <div id="chat-container"></div>

    <!-- Typing Indicator -->
    <div id="typing-indicator">Gemini is typing...</div> 

    <!-- Input Area -->
    <div id="input-area">
      <textarea id="input-box" placeholder="Enter your message..." rows="1"></textarea>
      <button id="send-button">Send</button>
    </div>
  </div>

  <script>
    // List of API keys. Replace these with your actual API keys.
    const API_KEYS = [
      'AIzaSyBdc_A8VxExwO6-0LY1--r2-Ym73lMg-h4',
      'AIzaSyBydeCpaNOIPSk794CpTGnfKrxbjCgJLu4',
      'AIzaSyCQ7tBHtctxlNs61Zm2Y__7KYeDLfz5dms',
      'AIzaSyAip_DFfKNC35MFIorEoNK4y6czP9P65-c'
    ];

    const chatContainer = document.getElementById('chat-container');
    const inputBox = document.getElementById('input-box');
    const sendButton = document.getElementById('send-button');
    const typingIndicator = document.getElementById('typing-indicator');
    const superSmartToggle = document.getElementById('superSmartToggle');
    const modelToggle = document.getElementById('modelToggle');

    // Initialize messageHistory based on toggles' initial state
    let messageHistory = "";

    // Current model variable
    let currentModel = 'gemini-1.5-pro-exp-0827'; // Default model

    // Event listener for the Super Smart Mode toggle switch
    superSmartToggle.addEventListener('change', () => {
      if (superSmartToggle.checked) {
        messageHistory = "bock bock" + messageHistory;
      } else {
        // If disabling, remove "Tt" from the history
        messageHistory = messageHistory.replace("bock bock", "");
      }
    });

    // Event listener for the Fast/Smart Model toggle switch
    modelToggle.addEventListener('change', () => {
      if (modelToggle.checked) {
        currentModel = 'gemini-1.5-flash';
      } else {
        currentModel = 'gemini-1.5-pro-exp-0827';
      }
      console.log(`Model switched to: ${currentModel}`);
    });

    sendButton.addEventListener('click', sendMessage);
    inputBox.addEventListener('keydown', (event) => {
      if (event.key === 'Enter' && !event.shiftKey) {
        event.preventDefault();
        sendMessage();
      }
    });

    /**
     * Attempts to fetch a response using the provided API key.
     * @param {string} apiKey - The API key to use for the request.
     * @param {string} prompt - The user message to send.
     * @param {string} model - The model to use for generating the response.
     * @returns {Promise<Object>} - The response data from the API.
     */
    async function fetchWithKey(apiKey, prompt, model) {
      const response = await fetch(`https://generativelanguage.googleapis.com/v1beta/models/${model}:generateContent?key=${apiKey}`, {
        method: 'POST',
        headers: {
          'Content-Type': 'application/json'
        },
        body: JSON.stringify({
          contents: [
            {
              role: "user",
              parts: [
                { text: prompt } // Include history in the prompt
              ]
            }
          ],
          "generationConfig": {
            "temperature": 0.5,
            "topK": 64,
            "topP": 0.95,
            "maxOutputTokens": 8192,
            "responseMimeType": "text/plain"
          }
        })
      });

      if (!response.ok) {
        // Extract error message from response
        const errorData = await response.json();
        throw new Error(errorData.error?.message || 'Unknown API error');
      }

      const data = await response.json();
      return data;
    }

    /**
     * Sends the user's message and handles the bot's response using multiple API keys.
     */
    async function sendMessage() {
      const userMessageText = inputBox.value.trim();
      if (userMessageText === '') return;

      // Display user's message
      const userMessage = document.createElement('div');
      userMessage.classList.add('message', 'user-message');
      userMessage.textContent = userMessageText;
      chatContainer.appendChild(userMessage);
      chatContainer.scrollTop = chatContainer.scrollHeight;      
      
      inputBox.value = '';
      typingIndicator.style.display = 'block';

      let botMessageText = '';
      let success = false;

      // Prepare the prompt based on the current messageHistory and Super smart mode
      const currentMessageHistory = messageHistory + `User: ${userMessageText}\n`;

      for (const apiKey of API_KEYS) {  // Simplified loop
        try {
          const data = await fetchWithKey(apiKey, currentMessageHistory, currentModel); // Add user message here

          const fullResponse = data?.candidates[0]?.content?.parts[0]?.text; // Optional chaining

          if (!fullResponse) {
            throw new Error("Invalid response format: Missing content."); // Better error handling
          }

          if (superSmartToggle.checked) {
            // Parse response between <final_answer> and </final_answer>
            const finalAnswerStartIndex = fullResponse.indexOf("<final_answer>");
            const finalAnswerEndIndex = fullResponse.indexOf("</final_answer>");

            if (finalAnswerStartIndex !== -1 && finalAnswerEndIndex !== -1) {
              botMessageText = fullResponse.substring(finalAnswerStartIndex + "<final_answer>".length, finalAnswerEndIndex).trim();
            } else {
              botMessageText = "Error: Could not extract final answer. Invalid response format.";
              console.error("Final answer tags not found in response:", fullResponse);
            }
          } else {
            // When Super smart mode is off, use the entire response
            botMessageText = fullResponse.trim();
          }

          if (botMessageText) {
            // Update history *after* successful response
            messageHistory += `Gemini (${currentModel}): ${botMessageText}\n`;
          }

          success = true;
          break; 
        } catch (error) {
          console.error(`Error with API key:`, error.message);
        }
      }

      const botMessage = document.createElement('div'); // Create element *outside* the loop
      botMessage.classList.add('message', 'bot-message');
      botMessage.textContent = success ? botMessageText : "I'm sorry, I'm currently unable to process your request. Please try again later."; // Display error or bot message
      chatContainer.appendChild(botMessage);
      chatContainer.scrollTop = chatContainer.scrollHeight;

      typingIndicator.style.display = 'none';
    }
  </script>
</body>
</html>