Upload folder using huggingface_hub

.ipynb_checkpoints/model fine-tunning-checkpoint.ipynb

@@ -0,0 +1,923 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "96d10e25-bc25-4058-baaa-a74ab15af266",
+   "metadata": {},
+   "source": [
+    "# Kaggle API Setup for TensorFlow and Keras NLP Project\n",
+    "Gemma is a family of lightweight, state-of-the-art open models built from the same research and technology used to create the Gemini models.\n",
+    "\n",
+    "## Introduction\n",
+    "In this notebook, we will prepare the environment to download and utilize the Gemma model in a TensorFlow and Keras NLP project. We will begin by importing the necessary libraries, followed by configuring the Kaggle API credentials to enable seamless access to the required datasets.\n",
+    "\n",
+    "## 1. Importing Required Libraries\n",
+    "To start, we will import the essential libraries for this project, including TensorFlow, Keras, and Keras NLP, which are crucial for building and deploying NLP models.\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "996572a0-edb7-49c9-89cc-27ad9b5a42d2",
+   "metadata": {
+    "scrolled": true
+   },
+   "outputs": [],
+   "source": [
+    "# Import required libraries\n",
+    "\n",
+    "import os\n",
+    "import tensorflow as tf\n",
+    "import keras_nlp\n",
+    "import keras\n",
+    "import json\n",
+    "\n",
+    "# Ignore Warnings\n",
+    "from silence_tensorflow import silence_tensorflow\n",
+    "silence_tensorflow()\n",
+    "import warnings\n",
+    "warnings.filterwarnings(\"ignore\")\n",
+    "os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3' "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "29ec8457-c4b5-4017-b190-812e0b2428f5",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Set Kaggle API credentials\n",
+    "\n",
+    "os.environ[\"KAGGLE_USERNAME\"] = \"rogerkorantenng\"\n",
+    "os.environ[\"KAGGLE_KEY\"] = \"9a33b6e88bcb6058b1281d777fa6808d\""
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "0145d6ca-9424-4c32-9e87-7cbac86cf65f",
+   "metadata": {},
+   "source": [
+    "## 2. Building and Compiling the Gemma Model\n",
+    "\n",
+    "In this section, we will build and compile the Gemma model, a language model designed for natural language processing (NLP) tasks. The process involves several key steps: loading the pre-trained model, enabling fine-tuning with LoRA (Low-Rank Adaptation), configuring the input sequence length, setting up the optimizer, and compiling the model for training.\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "e8045579-6e77-4879-b630-d1a9d2550a4d",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def get_compiled_model():\n",
+    "    gemma_lm = keras_nlp.models.GemmaCausalLM.from_preset(\"gemma_2b_en\")\n",
+    "    gemma_lm.summary()\n",
+    "\n",
+    "    gemma_lm.backbone.enable_lora(rank=4)\n",
+    "    gemma_lm.summary()\n",
+    "    \n",
+    "    # Set the sequence length to 128 before using the model.\n",
+    "    gemma_lm.preprocessor.sequence_length = 256\n",
+    "    \n",
+    "    # Use AdamW (a common optimizer for transformer models).\n",
+    "    optimizer = keras.optimizers.AdamW(\n",
+    "        learning_rate=5e-5,\n",
+    "        weight_decay=0.01,\n",
+    "    )\n",
+    "    \n",
+    "    # Exclude layernorm and bias terms from decay.\n",
+    "    optimizer.exclude_from_weight_decay(var_names=[\"bias\", \"scale\"])\n",
+    "    \n",
+    "    gemma_lm.compile(\n",
+    "        loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),\n",
+    "        optimizer=optimizer,\n",
+    "        weighted_metrics=[keras.metrics.SparseCategoricalAccuracy()],\n",
+    "    )\n",
+    "\n",
+    "    \n",
+    "    return gemma_lm"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "6475baba-ce54-40ec-a484-22629936c845",
+   "metadata": {},
+   "source": [
+    "## 3. Loading and Processing the Dataset\n",
+    "\n",
+    "In this section, we will define a function to load and process a JSON dataset. The dataset is read line by line, and each line is parsed and formatted according to the required structure. The function returns a list of formatted examples that can be used for training or analysis.\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "238f65b1-75c2-42a1-abd3-b4d8c7c36799",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def get_dataset():\n",
+    "    # Initialize an empty list to hold the processed data.\n",
+    "    data = []\n",
+    "    \n",
+    "    # Open and read the JSON file line by line.\n",
+    "    with open('/project/data/combined_dataset.json') as file:\n",
+    "        for line in file:\n",
+    "            features = json.loads(line)\n",
+    "            \n",
+    "            # Filter out examples without \"Context\".\n",
+    "            if not features.get(\"Context\"):\n",
+    "                continue\n",
+    "            \n",
+    "            # Format the example as a string.\n",
+    "            template = \"Instruction:\\n{Context}\\n\\nResponse:\\n{Response}\"\n",
+    "            formatted_example = template.format(**features)\n",
+    "            \n",
+    "            # Append the formatted example to the data list.\n",
+    "            data.append(formatted_example)\n",
+    "    \n",
+    "    return data "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "id": "d589e374-cc82-4132-9613-23ee064aa10e",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "2024-08-30 16:13:39.953235: I tensorflow/core/common_runtime/gpu/gpu_device.cc:2021] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 13775 MB memory:  -> device: 0, name: Tesla T4, pci bus id: 0000:18:00.0, compute capability: 7.5\n",
+      "2024-08-30 16:13:39.954747: I tensorflow/core/common_runtime/gpu/gpu_device.cc:2021] Created device /job:localhost/replica:0/task:0/device:GPU:1 with 13775 MB memory:  -> device: 1, name: Tesla T4, pci bus id: 0000:19:00.0, compute capability: 7.5\n",
+      "2024-08-30 16:13:39.956103: I tensorflow/core/common_runtime/gpu/gpu_device.cc:2021] Created device /job:localhost/replica:0/task:0/device:GPU:2 with 13775 MB memory:  -> device: 2, name: Tesla T4, pci bus id: 0000:35:00.0, compute capability: 7.5\n",
+      "2024-08-30 16:13:39.957459: I tensorflow/core/common_runtime/gpu/gpu_device.cc:2021] Created device /job:localhost/replica:0/task:0/device:GPU:3 with 13775 MB memory:  -> device: 3, name: Tesla T4, pci bus id: 0000:36:00.0, compute capability: 7.5\n",
+      "2024-08-30 16:13:39.958812: I tensorflow/core/common_runtime/gpu/gpu_device.cc:2021] Created device /job:localhost/replica:0/task:0/device:GPU:4 with 13775 MB memory:  -> device: 4, name: Tesla T4, pci bus id: 0000:e7:00.0, compute capability: 7.5\n",
+      "2024-08-30 16:13:39.960166: I tensorflow/core/common_runtime/gpu/gpu_device.cc:2021] Created device /job:localhost/replica:0/task:0/device:GPU:5 with 13775 MB memory:  -> device: 5, name: Tesla T4, pci bus id: 0000:e8:00.0, compute capability: 7.5\n",
+      "2024-08-30 16:13:39.961521: I tensorflow/core/common_runtime/gpu/gpu_device.cc:2021] Created device /job:localhost/replica:0/task:0/device:GPU:6 with 13775 MB memory:  -> device: 6, name: Tesla T4, pci bus id: 0000:f4:00.0, compute capability: 7.5\n",
+      "2024-08-30 16:13:39.962850: I tensorflow/core/common_runtime/gpu/gpu_device.cc:2021] Created device /job:localhost/replica:0/task:0/device:GPU:7 with 13775 MB memory:  -> device: 7, name: Tesla T4, pci bus id: 0000:f5:00.0, compute capability: 7.5\n",
+      "normalizer.cc(51) LOG(INFO) precompiled_charsmap is empty. use identity normalization.\n"
+     ]
+    },
+    {
+     "data": {
+      "text/html": [
+       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\">Preprocessor: \"gemma_causal_lm_preprocessor\"</span>\n",
+       "</pre>\n"
+      ],
+      "text/plain": [
+       "\u001b[1mPreprocessor: \"gemma_causal_lm_preprocessor\"\u001b[0m\n"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n",
+       "┃<span style=\"font-weight: bold\"> Tokenizer (type)                                   </span>┃<span style=\"font-weight: bold\">                                             Vocab # </span>┃\n",
+       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n",
+       "│ gemma_tokenizer (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">GemmaTokenizer</span>)                   │                                             <span style=\"color: #00af00; text-decoration-color: #00af00\">256,000</span> │\n",
+       "└────────────────────────────────────────────────────┴─────────────────────────────────────────────────────┘\n",
+       "</pre>\n"
+      ],
+      "text/plain": [
+       "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n",
+       "┃\u001b[1m \u001b[0m\u001b[1mTokenizer (type)                                  \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1m                                            Vocab #\u001b[0m\u001b[1m \u001b[0m┃\n",
+       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n",
+       "│ gemma_tokenizer (\u001b[38;5;33mGemmaTokenizer\u001b[0m)                   │                                             \u001b[38;5;34m256,000\u001b[0m │\n",
+       "└────────────────────────────────────────────────────┴─────────────────────────────────────────────────────┘\n"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\">Model: \"gemma_causal_lm\"</span>\n",
+       "</pre>\n"
+      ],
+      "text/plain": [
+       "\u001b[1mModel: \"gemma_causal_lm\"\u001b[0m\n"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n",
+       "┃<span style=\"font-weight: bold\"> Layer (type)                  </span>┃<span style=\"font-weight: bold\"> Output Shape              </span>┃<span style=\"font-weight: bold\">         Param # </span>┃<span style=\"font-weight: bold\"> Connected to               </span>┃\n",
+       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n",
+       "│ padding_mask (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">InputLayer</span>)     │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>)              │               <span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> │ -                          │\n",
+       "├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤\n",
+       "│ token_ids (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">InputLayer</span>)        │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>)              │               <span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> │ -                          │\n",
+       "├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤\n",
+       "│ gemma_backbone                │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">2048</span>)        │   <span style=\"color: #00af00; text-decoration-color: #00af00\">2,506,172,416</span> │ padding_mask[<span style=\"color: #00af00; text-decoration-color: #00af00\">0</span>][<span style=\"color: #00af00; text-decoration-color: #00af00\">0</span>],        │\n",
+       "│ (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">GemmaBackbone</span>)               │                           │                 │ token_ids[<span style=\"color: #00af00; text-decoration-color: #00af00\">0</span>][<span style=\"color: #00af00; text-decoration-color: #00af00\">0</span>]            │\n",
+       "├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤\n",
+       "│ token_embedding               │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">256000</span>)      │     <span style=\"color: #00af00; text-decoration-color: #00af00\">524,288,000</span> │ gemma_backbone[<span style=\"color: #00af00; text-decoration-color: #00af00\">0</span>][<span style=\"color: #00af00; text-decoration-color: #00af00\">0</span>]       │\n",
+       "│ (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">ReversibleEmbedding</span>)         │                           │                 │                            │\n",
+       "└───────────────────────────────┴───────────────────────────┴─────────────────┴────────────────────────────┘\n",
+       "</pre>\n"
+      ],
+      "text/plain": [
+       "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n",
+       "┃\u001b[1m \u001b[0m\u001b[1mLayer (type)                 \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1mOutput Shape             \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1m        Param #\u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1mConnected to              \u001b[0m\u001b[1m \u001b[0m┃\n",
+       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n",
+       "│ padding_mask (\u001b[38;5;33mInputLayer\u001b[0m)     │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;45mNone\u001b[0m)              │               \u001b[38;5;34m0\u001b[0m │ -                          │\n",
+       "├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤\n",
+       "│ token_ids (\u001b[38;5;33mInputLayer\u001b[0m)        │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;45mNone\u001b[0m)              │               \u001b[38;5;34m0\u001b[0m │ -                          │\n",
+       "├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤\n",
+       "│ gemma_backbone                │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m2048\u001b[0m)        │   \u001b[38;5;34m2,506,172,416\u001b[0m │ padding_mask[\u001b[38;5;34m0\u001b[0m][\u001b[38;5;34m0\u001b[0m],        │\n",
+       "│ (\u001b[38;5;33mGemmaBackbone\u001b[0m)               │                           │                 │ token_ids[\u001b[38;5;34m0\u001b[0m][\u001b[38;5;34m0\u001b[0m]            │\n",
+       "├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤\n",
+       "│ token_embedding               │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m256000\u001b[0m)      │     \u001b[38;5;34m524,288,000\u001b[0m │ gemma_backbone[\u001b[38;5;34m0\u001b[0m][\u001b[38;5;34m0\u001b[0m]       │\n",
+       "│ (\u001b[38;5;33mReversibleEmbedding\u001b[0m)         │                           │                 │                            │\n",
+       "└───────────────────────────────┴───────────────────────────┴─────────────────┴────────────────────────────┘\n"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Total params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">2,506,172,416</span> (9.34 GB)\n",
+       "</pre>\n"
+      ],
+      "text/plain": [
+       "\u001b[1m Total params: \u001b[0m\u001b[38;5;34m2,506,172,416\u001b[0m (9.34 GB)\n"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Trainable params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">2,506,172,416</span> (9.34 GB)\n",
+       "</pre>\n"
+      ],
+      "text/plain": [
+       "\u001b[1m Trainable params: \u001b[0m\u001b[38;5;34m2,506,172,416\u001b[0m (9.34 GB)\n"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Non-trainable params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> (0.00 B)\n",
+       "</pre>\n"
+      ],
+      "text/plain": [
+       "\u001b[1m Non-trainable params: \u001b[0m\u001b[38;5;34m0\u001b[0m (0.00 B)\n"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\">Preprocessor: \"gemma_causal_lm_preprocessor\"</span>\n",
+       "</pre>\n"
+      ],
+      "text/plain": [
+       "\u001b[1mPreprocessor: \"gemma_causal_lm_preprocessor\"\u001b[0m\n"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n",
+       "┃<span style=\"font-weight: bold\"> Tokenizer (type)                                   </span>┃<span style=\"font-weight: bold\">                                             Vocab # </span>┃\n",
+       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n",
+       "│ gemma_tokenizer (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">GemmaTokenizer</span>)                   │                                             <span style=\"color: #00af00; text-decoration-color: #00af00\">256,000</span> │\n",
+       "└────────────────────────────────────────────────────┴─────────────────────────────────────────────────────┘\n",
+       "</pre>\n"
+      ],
+      "text/plain": [
+       "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n",
+       "┃\u001b[1m \u001b[0m\u001b[1mTokenizer (type)                                  \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1m                                            Vocab #\u001b[0m\u001b[1m \u001b[0m┃\n",
+       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━��━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n",
+       "│ gemma_tokenizer (\u001b[38;5;33mGemmaTokenizer\u001b[0m)                   │                                             \u001b[38;5;34m256,000\u001b[0m │\n",
+       "└────────────────────────────────────────────────────┴─────────────────────────────────────────────────────┘\n"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\">Model: \"gemma_causal_lm\"</span>\n",
+       "</pre>\n"
+      ],
+      "text/plain": [
+       "\u001b[1mModel: \"gemma_causal_lm\"\u001b[0m\n"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n",
+       "┃<span style=\"font-weight: bold\"> Layer (type)                  </span>┃<span style=\"font-weight: bold\"> Output Shape              </span>┃<span style=\"font-weight: bold\">         Param # </span>┃<span style=\"font-weight: bold\"> Connected to               </span>┃\n",
+       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n",
+       "│ padding_mask (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">InputLayer</span>)     │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>)              │               <span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> │ -                          │\n",
+       "├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤\n",
+       "│ token_ids (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">InputLayer</span>)        │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>)              │               <span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> │ -                          │\n",
+       "├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤\n",
+       "│ gemma_backbone                │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">2048</span>)        │   <span style=\"color: #00af00; text-decoration-color: #00af00\">2,507,536,384</span> │ padding_mask[<span style=\"color: #00af00; text-decoration-color: #00af00\">0</span>][<span style=\"color: #00af00; text-decoration-color: #00af00\">0</span>],        │\n",
+       "│ (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">GemmaBackbone</span>)               │                           │                 │ token_ids[<span style=\"color: #00af00; text-decoration-color: #00af00\">0</span>][<span style=\"color: #00af00; text-decoration-color: #00af00\">0</span>]            │\n",
+       "├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤\n",
+       "│ token_embedding               │ (<span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00d7ff; text-decoration-color: #00d7ff\">None</span>, <span style=\"color: #00af00; text-decoration-color: #00af00\">256000</span>)      │     <span style=\"color: #00af00; text-decoration-color: #00af00\">524,288,000</span> │ gemma_backbone[<span style=\"color: #00af00; text-decoration-color: #00af00\">0</span>][<span style=\"color: #00af00; text-decoration-color: #00af00\">0</span>]       │\n",
+       "│ (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">ReversibleEmbedding</span>)         │                           │                 │                            │\n",
+       "└───────────────────────────────┴───────────────────────────┴─────────────────┴────────────────────────────┘\n",
+       "</pre>\n"
+      ],
+      "text/plain": [
+       "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n",
+       "┃\u001b[1m \u001b[0m\u001b[1mLayer (type)                 \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1mOutput Shape             \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1m        Param #\u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1mConnected to              \u001b[0m\u001b[1m \u001b[0m┃\n",
+       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n",
+       "│ padding_mask (\u001b[38;5;33mInputLayer\u001b[0m)     │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;45mNone\u001b[0m)              │               \u001b[38;5;34m0\u001b[0m │ -                          │\n",
+       "├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤\n",
+       "│ token_ids (\u001b[38;5;33mInputLayer\u001b[0m)        │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;45mNone\u001b[0m)              │               \u001b[38;5;34m0\u001b[0m │ -                          │\n",
+       "├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤\n",
+       "│ gemma_backbone                │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m2048\u001b[0m)        │   \u001b[38;5;34m2,507,536,384\u001b[0m │ padding_mask[\u001b[38;5;34m0\u001b[0m][\u001b[38;5;34m0\u001b[0m],        │\n",
+       "│ (\u001b[38;5;33mGemmaBackbone\u001b[0m)               │                           │                 │ token_ids[\u001b[38;5;34m0\u001b[0m][\u001b[38;5;34m0\u001b[0m]            │\n",
+       "├───────────────────────────────┼───────────────────────────┼─────────────────┼────────────────────────────┤\n",
+       "│ token_embedding               │ (\u001b[38;5;45mNone\u001b[0m, \u001b[38;5;45mNone\u001b[0m, \u001b[38;5;34m256000\u001b[0m)      │     \u001b[38;5;34m524,288,000\u001b[0m │ gemma_backbone[\u001b[38;5;34m0\u001b[0m][\u001b[38;5;34m0\u001b[0m]       │\n",
+       "│ (\u001b[38;5;33mReversibleEmbedding\u001b[0m)         │                           │                 │                            │\n",
+       "└───────────────────────────────┴───────────────────────────┴─────────────────┴────────────────────────────┘\n"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Total params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">2,507,536,384</span> (9.34 GB)\n",
+       "</pre>\n"
+      ],
+      "text/plain": [
+       "\u001b[1m Total params: \u001b[0m\u001b[38;5;34m2,507,536,384\u001b[0m (9.34 GB)\n"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Trainable params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">1,363,968</span> (5.20 MB)\n",
+       "</pre>\n"
+      ],
+      "text/plain": [
+       "\u001b[1m Trainable params: \u001b[0m\u001b[38;5;34m1,363,968\u001b[0m (5.20 MB)\n"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Non-trainable params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">2,506,172,416</span> (9.34 GB)\n",
+       "</pre>\n"
+      ],
+      "text/plain": [
+       "\u001b[1m Non-trainable params: \u001b[0m\u001b[38;5;34m2,506,172,416\u001b[0m (9.34 GB)\n"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "# Get Model and Compile\n",
+    "model  = get_compiled_model()\n",
+    "\n",
+    "# Get the dataset outside the strategy scope.\n",
+    "data = get_dataset()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "50106081-cd9e-4246-974e-a4e7db99be97",
+   "metadata": {},
+   "source": [
+    "## 4. Defining the Prompt Template and Generating Responses\n",
+    "\n",
+    "In this section, we define a template for creating prompts that the language model will use to generate responses. The template includes placeholders for an 'instruction' and a 'response'. We then format this template with actual data to create a complete prompt. Finally, the prompt is passed to the language model to generate a response.\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "id": "47a8c8da-b73a-4684-9199-176b4fa9bd3d",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "2024-08-30 16:13:57.828298: E tensorflow/core/util/util.cc:131] oneDNN supports DT_INT64 only on platforms with AVX-512. Falling back to the default Eigen-based implementation if present.\n",
+      "WARNING: All log messages before absl::InitializeLog() is called are written to STDERR\n",
+      "I0000 00:00:1725034446.797146   18073 service.cc:146] XLA service 0x5a7be0cb31d0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:\n",
+      "I0000 00:00:1725034446.797176   18073 service.cc:154]   StreamExecutor device (0): Tesla T4, Compute Capability 7.5\n",
+      "I0000 00:00:1725034446.797180   18073 service.cc:154]   StreamExecutor device (1): Tesla T4, Compute Capability 7.5\n",
+      "I0000 00:00:1725034446.797182   18073 service.cc:154]   StreamExecutor device (2): Tesla T4, Compute Capability 7.5\n",
+      "I0000 00:00:1725034446.797185   18073 service.cc:154]   StreamExecutor device (3): Tesla T4, Compute Capability 7.5\n",
+      "I0000 00:00:1725034446.797188   18073 service.cc:154]   StreamExecutor device (4): Tesla T4, Compute Capability 7.5\n",
+      "I0000 00:00:1725034446.797190   18073 service.cc:154]   StreamExecutor device (5): Tesla T4, Compute Capability 7.5\n",
+      "I0000 00:00:1725034446.797193   18073 service.cc:154]   StreamExecutor device (6): Tesla T4, Compute Capability 7.5\n",
+      "I0000 00:00:1725034446.797196   18073 service.cc:154]   StreamExecutor device (7): Tesla T4, Compute Capability 7.5\n",
+      "2024-08-30 16:14:07.351952: I tensorflow/compiler/mlir/tensorflow/utils/dump_mlir_util.cc:268] disabling MLIR crash reproducer, set env var `MLIR_CRASH_REPRODUCER_DIRECTORY` to enable.\n",
+      "2024-08-30 16:14:09.094695: I external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:531] Loaded cuDNN version 8905\n",
+      "I0000 00:00:1725034456.429832   18073 device_compiler.h:188] Compiled cluster using XLA!  This line is logged at most once for the lifetime of the process.\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Instruction:\n",
+      "I'm going through some things with my feelings and myself. I barely sleep and I do nothing but think about how I'm worthless and how I shouldn't be here.\n",
+      "   I've never tried or contemplated suicide. I've always wanted to fix my issues, but I never get around to it.\n",
+      "   How can I change my feeling of being worthless to everyone?\n",
+      "\n",
+      "Response:\n",
+      "I'm sorry to hear that you're going through some things. I'm not sure what you mean by \"I barely sleep and I do nothing but think about how I'm worthless and how I shouldn't be here.\" I'm not sure what you mean by \"I've never tried or contemplated suicide.\" I'm not sure what you mean by \"I've always wanted to fix my issues, but I never get around to it.\" I'm not sure what you mean by \"How can I change my feeling of being worthless to everyone?\"\n",
+      "\n",
+      "I'm not sure what you mean by \"I'm sorry to hear that you're going through some things.\"\n",
+      "\n",
+      "I'm not sure what you mean by \"I'm not sure what you mean by 'I barely sleep\n"
+     ]
+    }
+   ],
+   "source": [
+    "# Define the template with placeholders for 'instruction' and 'response'\n",
+    "template = \"Instruction:\\n{instruction}\\n\\nResponse:\\n{response}\"\n",
+    "\n",
+    "# Create the prompt by formatting the template with actual data\n",
+    "prompt = template.format(\n",
+    "    instruction=\"I'm going through some things with my feelings and myself. I barely sleep and I do nothing but think about how I'm worthless and how I shouldn't be here.\\n   I've never tried or contemplated suicide. I've always wanted to fix my issues, but I never get around to it.\\n   How can I change my feeling of being worthless to everyone?\",\n",
+    "    response=\"\",\n",
+    ")\n",
+    "\n",
+    "# Assuming gemma_lm is a language model that you're using to generate text\n",
+    "print(model.generate(prompt, max_length=256))\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "9784a448-b7a2-40e7-8162-3caa3ac64d22",
+   "metadata": {},
+   "source": [
+    "## 5. Model Fine Tunning\n",
+    "\n",
+    "In this section, we compile the model, prepare the dataset, and then train the model using the data. We will walk through the steps of obtaining the compiled model, loading the dataset, and fitting the model to the data.\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "id": "9355a347-5893-4f94-bff8-09cb126d818d",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 1/40\n"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "W0000 00:00:1725034506.217331   18546 assert_op.cc:38] Ignoring Assert operator compile_loss/sparse_categorical_crossentropy/SparseSoftmaxCrossEntropyWithLogits/assert_equal_1/Assert/Assert\n",
+      "2024-08-30 16:15:23.759806: I external/local_xla/xla/stream_executor/cuda/cuda_asm_compiler.cc:393] ptxas warning : Registers are spilled to local memory in function 'loop_add_subtract_fusion_2', 220 bytes spill stores, 220 bytes spill loads\n",
+      "\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2455s\u001b[0m 682ms/step - loss: 2.0513 - sparse_categorical_accuracy: 0.4572\n",
+      "Epoch 2/40\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2398s\u001b[0m 683ms/step - loss: 1.9421 - sparse_categorical_accuracy: 0.4760\n",
+      "Epoch 3/40\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2398s\u001b[0m 683ms/step - loss: 1.8639 - sparse_categorical_accuracy: 0.4929\n",
+      "Epoch 4/40\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2396s\u001b[0m 682ms/step - loss: 1.7974 - sparse_categorical_accuracy: 0.5083\n",
+      "Epoch 5/40\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2397s\u001b[0m 683ms/step - loss: 1.7301 - sparse_categorical_accuracy: 0.5239\n",
+      "Epoch 6/40\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2396s\u001b[0m 682ms/step - loss: 1.6699 - sparse_categorical_accuracy: 0.5380\n",
+      "Epoch 7/40\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2397s\u001b[0m 682ms/step - loss: 1.6151 - sparse_categorical_accuracy: 0.5512\n",
+      "Epoch 8/40\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2399s\u001b[0m 683ms/step - loss: 1.5641 - sparse_categorical_accuracy: 0.5633\n",
+      "Epoch 9/40\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2393s\u001b[0m 681ms/step - loss: 1.5194 - sparse_categorical_accuracy: 0.5776\n",
+      "Epoch 10/40\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━��\u001b[0m\u001b[37m\u001b[0m \u001b[1m2392s\u001b[0m 681ms/step - loss: 1.4795 - sparse_categorical_accuracy: 0.5869\n",
+      "Epoch 11/40\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2393s\u001b[0m 681ms/step - loss: 1.4477 - sparse_categorical_accuracy: 0.5949\n",
+      "Epoch 12/40\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2395s\u001b[0m 682ms/step - loss: 1.4191 - sparse_categorical_accuracy: 0.6025\n",
+      "Epoch 13/40\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2397s\u001b[0m 683ms/step - loss: 1.3948 - sparse_categorical_accuracy: 0.6080\n",
+      "Epoch 14/40\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2397s\u001b[0m 682ms/step - loss: 1.3707 - sparse_categorical_accuracy: 0.6142\n",
+      "Epoch 15/40\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2398s\u001b[0m 683ms/step - loss: 1.3508 - sparse_categorical_accuracy: 0.6195\n",
+      "Epoch 16/40\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2397s\u001b[0m 682ms/step - loss: 1.3308 - sparse_categorical_accuracy: 0.6236\n",
+      "Epoch 17/40\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2400s\u001b[0m 683ms/step - loss: 1.3068 - sparse_categorical_accuracy: 0.6303\n",
+      "Epoch 18/40\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2400s\u001b[0m 683ms/step - loss: 1.2879 - sparse_categorical_accuracy: 0.6350\n",
+      "Epoch 19/40\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2401s\u001b[0m 683ms/step - loss: 1.2676 - sparse_categorical_accuracy: 0.6395\n",
+      "Epoch 20/40\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2396s\u001b[0m 682ms/step - loss: 1.2474 - sparse_categorical_accuracy: 0.6444\n",
+      "Epoch 21/40\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2395s\u001b[0m 682ms/step - loss: 1.2283 - sparse_categorical_accuracy: 0.6491\n",
+      "Epoch 22/40\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2398s\u001b[0m 683ms/step - loss: 1.2086 - sparse_categorical_accuracy: 0.6543\n",
+      "Epoch 23/40\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2399s\u001b[0m 683ms/step - loss: 1.1896 - sparse_categorical_accuracy: 0.6593\n",
+      "Epoch 24/40\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2400s\u001b[0m 683ms/step - loss: 1.1706 - sparse_categorical_accuracy: 0.6644\n",
+      "Epoch 25/40\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2399s\u001b[0m 683ms/step - loss: 1.1508 - sparse_categorical_accuracy: 0.6695\n",
+      "Epoch 26/40\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2401s\u001b[0m 683ms/step - loss: 1.1322 - sparse_categorical_accuracy: 0.6744\n",
+      "Epoch 27/40\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2400s\u001b[0m 683ms/step - loss: 1.1152 - sparse_categorical_accuracy: 0.6789\n",
+      "Epoch 28/40\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2397s\u001b[0m 682ms/step - loss: 1.0921 - sparse_categorical_accuracy: 0.6851\n",
+      "Epoch 29/40\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2396s\u001b[0m 682ms/step - loss: 1.0791 - sparse_categorical_accuracy: 0.6881\n",
+      "Epoch 30/40\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2398s\u001b[0m 683ms/step - loss: 1.0581 - sparse_categorical_accuracy: 0.6941\n",
+      "Epoch 31/40\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2399s\u001b[0m 683ms/step - loss: 1.0382 - sparse_categorical_accuracy: 0.6994\n",
+      "Epoch 32/40\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2401s\u001b[0m 684ms/step - loss: 1.0208 - sparse_categorical_accuracy: 0.7045\n",
+      "Epoch 33/40\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2402s\u001b[0m 684ms/step - loss: 1.0037 - sparse_categorical_accuracy: 0.7089\n",
+      "Epoch 34/40\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2402s\u001b[0m 684ms/step - loss: 0.9862 - sparse_categorical_accuracy: 0.7137\n",
+      "Epoch 35/40\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2405s\u001b[0m 685ms/step - loss: 0.9688 - sparse_categorical_accuracy: 0.7183\n",
+      "Epoch 36/40\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2402s\u001b[0m 684ms/step - loss: 0.9554 - sparse_categorical_accuracy: 0.7219\n",
+      "Epoch 37/40\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2402s\u001b[0m 684ms/step - loss: 0.9479 - sparse_categorical_accuracy: 0.7239\n",
+      "Epoch 38/40\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2404s\u001b[0m 685ms/step - loss: 0.9224 - sparse_categorical_accuracy: 0.7313\n",
+      "Epoch 39/40\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2408s\u001b[0m 686ms/step - loss: 0.9132 - sparse_categorical_accuracy: 0.7335\n",
+      "Epoch 40/40\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2410s\u001b[0m 686ms/step - loss: 0.8930 - sparse_categorical_accuracy: 0.7399\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "<keras.src.callbacks.history.History at 0x7b0d2c394a00>"
+      ]
+     },
+     "execution_count": 7,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "# Fit the model using the data.\n",
+    "model.fit(data, epochs=40, batch_size=1, verbose=1)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "8ca51424-9aa2-4933-9f97-106eec5347c5",
+   "metadata": {},
+   "source": [
+    "## 6. Generating a Response from the Fine-Tuned Language Model\n",
+    "\n",
+    "In this section, we will define a template for generating prompts and use the language model `gemma_lm` to generate a response based on the provided instruction. This process involves creating a formatted prompt and then using the model to produce a response.\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "id": "9199e8bd-3269-4b4a-bb8b-78399965883f",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Instruction:\n",
+      "I'm going through some things with my feelings and myself. I barely sleep and I do nothing but think about how I'm worthless and how I shouldn't be here.\n",
+      "   I've never tried or contemplated suicide. I've always wanted to fix my issues, but I never get around to it.\n",
+      "   How can I change my feeling of being worthless to everyone?\n",
+      "\n",
+      "Response:\n",
+      "It sounds like you are having a really tough time with feeling this way.  Feeling this way is not normal and it is important to talk about these feelings with someone.  You are not worthless, you are a wonderful person who is going through some tough times.  Therapy can help you to work through these issues and come to a place of self-love and acceptance.  You can do this!  \n"
+     ]
+    }
+   ],
+   "source": [
+    "# Define the template with placeholders for 'instruction' and 'response'\n",
+    "template = \"Instruction:\\n{instruction}\\n\\nResponse:\\n{response}\"\n",
+    "\n",
+    "# Create the prompt by formatting the template with actual data\n",
+    "prompt = template.format(\n",
+    "    instruction=\"I'm going through some things with my feelings and myself. I barely sleep and I do nothing but think about how I'm worthless and how I shouldn't be here.\\n   I've never tried or contemplated suicide. I've always wanted to fix my issues, but I never get around to it.\\n   How can I change my feeling of being worthless to everyone?\",\n",
+    "    response=\"\",\n",
+    ")\n",
+    "\n",
+    "# Assuming gemma_lm is a language model that you're using to generate text\n",
+    "print(model.generate(prompt, max_length=256))\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "b418c2e7-af66-4625-887d-0c1e5d1f4464",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 1/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2388s\u001b[0m 680ms/step - loss: 0.8785 - sparse_categorical_accuracy: 0.7438\n",
+      "Epoch 2/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2388s\u001b[0m 680ms/step - loss: 0.8661 - sparse_categorical_accuracy: 0.7473\n",
+      "Epoch 3/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2384s\u001b[0m 679ms/step - loss: 0.8522 - sparse_categorical_accuracy: 0.7514\n",
+      "Epoch 4/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2380s\u001b[0m 678ms/step - loss: 0.8408 - sparse_categorical_accuracy: 0.7547\n",
+      "Epoch 5/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2379s\u001b[0m 677ms/step - loss: 0.8266 - sparse_categorical_accuracy: 0.7585\n",
+      "Epoch 6/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2380s\u001b[0m 678ms/step - loss: 0.8117 - sparse_categorical_accuracy: 0.7635\n",
+      "Epoch 7/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2379s\u001b[0m 677ms/step - loss: 0.8048 - sparse_categorical_accuracy: 0.7653\n",
+      "Epoch 8/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2382s\u001b[0m 678ms/step - loss: 0.7892 - sparse_categorical_accuracy: 0.7698\n",
+      "Epoch 9/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2387s\u001b[0m 680ms/step - loss: 0.7790 - sparse_categorical_accuracy: 0.7724\n",
+      "Epoch 10/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2387s\u001b[0m 680ms/step - loss: 0.7680 - sparse_categorical_accuracy: 0.7754\n",
+      "Epoch 11/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2391s\u001b[0m 681ms/step - loss: 0.7569 - sparse_categorical_accuracy: 0.7786\n",
+      "Epoch 12/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2392s\u001b[0m 681ms/step - loss: 0.7476 - sparse_categorical_accuracy: 0.7821\n",
+      "Epoch 13/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2396s\u001b[0m 682ms/step - loss: 0.7373 - sparse_categorical_accuracy: 0.7844\n",
+      "Epoch 14/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2399s\u001b[0m 683ms/step - loss: 0.7335 - sparse_categorical_accuracy: 0.7855\n",
+      "Epoch 15/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2401s\u001b[0m 684ms/step - loss: 0.7246 - sparse_categorical_accuracy: 0.7883\n",
+      "Epoch 16/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2405s\u001b[0m 685ms/step - loss: 0.6987 - sparse_categorical_accuracy: 0.7965\n",
+      "Epoch 17/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2400s\u001b[0m 683ms/step - loss: 0.7001 - sparse_categorical_accuracy: 0.7956\n",
+      "Epoch 18/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2400s\u001b[0m 683ms/step - loss: 0.6918 - sparse_categorical_accuracy: 0.7976\n",
+      "Epoch 19/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2397s\u001b[0m 683ms/step - loss: 0.6806 - sparse_categorical_accuracy: 0.8007\n",
+      "Epoch 20/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2395s\u001b[0m 682ms/step - loss: 0.6768 - sparse_categorical_accuracy: 0.8020\n",
+      "Epoch 21/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2393s\u001b[0m 681ms/step - loss: 0.6644 - sparse_categorical_accuracy: 0.8058\n",
+      "Epoch 22/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2390s\u001b[0m 680ms/step - loss: 0.6582 - sparse_categorical_accuracy: 0.8080\n",
+      "Epoch 23/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2385s\u001b[0m 679ms/step - loss: 0.6479 - sparse_categorical_accuracy: 0.8107\n",
+      "Epoch 24/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2380s\u001b[0m 678ms/step - loss: 0.6394 - sparse_categorical_accuracy: 0.8131\n",
+      "Epoch 25/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2376s\u001b[0m 677ms/step - loss: 0.6351 - sparse_categorical_accuracy: 0.8141\n",
+      "Epoch 26/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2372s\u001b[0m 675ms/step - loss: 0.6264 - sparse_categorical_accuracy: 0.8169\n",
+      "Epoch 27/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2373s\u001b[0m 676ms/step - loss: 0.6198 - sparse_categorical_accuracy: 0.8193\n",
+      "Epoch 28/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2367s\u001b[0m 674ms/step - loss: 0.6127 - sparse_categorical_accuracy: 0.8213\n",
+      "Epoch 29/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2361s\u001b[0m 672ms/step - loss: 0.6048 - sparse_categorical_accuracy: 0.8236\n",
+      "Epoch 30/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2358s\u001b[0m 671ms/step - loss: 0.6168 - sparse_categorical_accuracy: 0.8185\n",
+      "Epoch 31/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2357s\u001b[0m 671ms/step - loss: 0.5947 - sparse_categorical_accuracy: 0.8266\n",
+      "Epoch 32/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2356s\u001b[0m 671ms/step - loss: 0.5878 - sparse_categorical_accuracy: 0.8279\n",
+      "Epoch 33/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2353s\u001b[0m 670ms/step - loss: 0.5781 - sparse_categorical_accuracy: 0.8315\n",
+      "Epoch 34/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2349s\u001b[0m 669ms/step - loss: 0.5729 - sparse_categorical_accuracy: 0.8335\n",
+      "Epoch 35/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2348s\u001b[0m 668ms/step - loss: 0.5684 - sparse_categorical_accuracy: 0.8344\n",
+      "Epoch 36/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2346s\u001b[0m 668ms/step - loss: 0.5684 - sparse_categorical_accuracy: 0.8339\n",
+      "Epoch 37/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2346s\u001b[0m 668ms/step - loss: 0.5550 - sparse_categorical_accuracy: 0.8389\n",
+      "Epoch 38/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2343s\u001b[0m 667ms/step - loss: 0.5544 - sparse_categorical_accuracy: 0.8384\n",
+      "Epoch 39/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2343s\u001b[0m 667ms/step - loss: 0.5454 - sparse_categorical_accuracy: 0.8413\n",
+      "Epoch 40/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2343s\u001b[0m 667ms/step - loss: 0.5389 - sparse_categorical_accuracy: 0.8433\n",
+      "Epoch 41/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2344s\u001b[0m 667ms/step - loss: 0.5456 - sparse_categorical_accuracy: 0.8406\n",
+      "Epoch 42/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2347s\u001b[0m 668ms/step - loss: 0.5289 - sparse_categorical_accuracy: 0.8462\n",
+      "Epoch 43/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2349s\u001b[0m 669ms/step - loss: 0.5264 - sparse_categorical_accuracy: 0.8465\n",
+      "Epoch 44/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2356s\u001b[0m 671ms/step - loss: 0.5192 - sparse_categorical_accuracy: 0.8489\n",
+      "Epoch 45/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2363s\u001b[0m 673ms/step - loss: 0.5127 - sparse_categorical_accuracy: 0.8513\n",
+      "Epoch 46/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2369s\u001b[0m 674ms/step - loss: 0.5090 - sparse_categorical_accuracy: 0.8522\n",
+      "Epoch 47/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2375s\u001b[0m 676ms/step - loss: 0.5033 - sparse_categorical_accuracy: 0.8538\n",
+      "Epoch 48/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2378s\u001b[0m 677ms/step - loss: 0.5023 - sparse_categorical_accuracy: 0.8541\n",
+      "Epoch 49/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2382s\u001b[0m 678ms/step - loss: 0.4946 - sparse_categorical_accuracy: 0.8565\n",
+      "Epoch 50/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2386s\u001b[0m 679ms/step - loss: 0.4915 - sparse_categorical_accuracy: 0.8567\n",
+      "Epoch 51/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2389s\u001b[0m 680ms/step - loss: 0.4842 - sparse_categorical_accuracy: 0.8597\n",
+      "Epoch 52/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2392s\u001b[0m 681ms/step - loss: 0.4836 - sparse_categorical_accuracy: 0.8599\n",
+      "Epoch 53/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2394s\u001b[0m 682ms/step - loss: 0.4772 - sparse_categorical_accuracy: 0.8611\n",
+      "Epoch 54/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2397s\u001b[0m 683ms/step - loss: 0.4749 - sparse_categorical_accuracy: 0.8617\n",
+      "Epoch 55/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2393s\u001b[0m 681ms/step - loss: 0.4785 - sparse_categorical_accuracy: 0.8603\n",
+      "Epoch 56/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2389s\u001b[0m 680ms/step - loss: 0.4587 - sparse_categorical_accuracy: 0.8680\n",
+      "Epoch 57/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2389s\u001b[0m 680ms/step - loss: 0.4649 - sparse_categorical_accuracy: 0.8655\n",
+      "Epoch 58/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2385s\u001b[0m 679ms/step - loss: 0.4573 - sparse_categorical_accuracy: 0.8675\n",
+      "Epoch 59/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2382s\u001b[0m 678ms/step - loss: 0.4545 - sparse_categorical_accuracy: 0.8689\n",
+      "Epoch 60/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2376s\u001b[0m 676ms/step - loss: 0.4499 - sparse_categorical_accuracy: 0.8697\n",
+      "Epoch 61/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2377s\u001b[0m 677ms/step - loss: 0.4482 - sparse_categorical_accuracy: 0.8698\n",
+      "Epoch 62/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2370s\u001b[0m 675ms/step - loss: 0.4421 - sparse_categorical_accuracy: 0.8722\n",
+      "Epoch 63/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2360s\u001b[0m 672ms/step - loss: 0.4370 - sparse_categorical_accuracy: 0.8739\n",
+      "Epoch 64/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2352s\u001b[0m 670ms/step - loss: 0.4315 - sparse_categorical_accuracy: 0.8753\n",
+      "Epoch 65/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2351s\u001b[0m 669ms/step - loss: 0.4342 - sparse_categorical_accuracy: 0.8747\n",
+      "Epoch 66/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2355s\u001b[0m 670ms/step - loss: 0.4305 - sparse_categorical_accuracy: 0.8758\n",
+      "Epoch 67/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2354s\u001b[0m 670ms/step - loss: 0.4266 - sparse_categorical_accuracy: 0.8771\n",
+      "Epoch 68/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2351s\u001b[0m 669ms/step - loss: 0.4204 - sparse_categorical_accuracy: 0.8790\n",
+      "Epoch 69/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2350s\u001b[0m 669ms/step - loss: 0.4176 - sparse_categorical_accuracy: 0.8795\n",
+      "Epoch 70/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2346s\u001b[0m 668ms/step - loss: 0.4132 - sparse_categorical_accuracy: 0.8811\n",
+      "Epoch 71/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2344s\u001b[0m 667ms/step - loss: 0.4143 - sparse_categorical_accuracy: 0.8804\n",
+      "Epoch 72/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2342s\u001b[0m 667ms/step - loss: 0.4078 - sparse_categorical_accuracy: 0.8828\n",
+      "Epoch 73/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2343s\u001b[0m 667ms/step - loss: 0.4041 - sparse_categorical_accuracy: 0.8837\n",
+      "Epoch 74/90\n",
+      "\u001b[1m3512/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m2341s\u001b[0m 666ms/step - loss: 0.4060 - sparse_categorical_accuracy: 0.8832\n",
+      "Epoch 75/90\n",
+      "\u001b[1m2888/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━━━━\u001b[0m \u001b[1m6:55\u001b[0m 667ms/step - loss: 0.4078 - sparse_categorical_accuracy: 0.8826"
+     ]
+    }
+   ],
+   "source": [
+    "# Fit the model using the data.\n",
+    "model.fit(data, epochs=90, batch_size=1, verbose=1)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 20,
+   "id": "f5d19bd9-72c8-4519-933b-c94a25b1ee3d",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "model.backbone.save_lora_weights(\"model2.lora.h5\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 23,
+   "id": "1fda3d9d-488d-46a2-8f4e-b0b83f175426",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Instruction:\n",
+      "I'm going through some things with my feelings and myself. I barely sleep and I do nothing but think about how I'm worthless and how I shouldn't be here.\n",
+      "   I've never tried or contemplated suicide. I've always wanted to fix my issues, but I never get around to it.\n",
+      "   How can I change my feeling of being worthless to everyone?\n",
+      "\n",
+      "Response:\n",
+      "I'm glad you're willing to slow down, but it's a complicated feeling that can be challenging to identify and work through. I've heard it described as feeling like a heavy cloud or rain cloud, and it's good to know where you are when you want to know what to do about it.It is possible to know more about this feeling and why it's arising from time to time like this. It helps to know that what you're experiencing is likely to do with your sense of who you are and your sense of how important you are in people's lives. It's also helpful to know that this feeling is something that's happening deep within you, not something outside of you.It can be valuable to look at the relationships in your life and consider your place in them.\n"
+     ]
+    }
+   ],
+   "source": [
+    "# Define the template with placeholders for 'instruction' and 'response'\n",
+    "template = \"Instruction:\\n{instruction}\\n\\nResponse:\\n{response}\"\n",
+    "\n",
+    "# Create the prompt by formatting the template with actual data\n",
+    "prompt = template.format(\n",
+    "    instruction=\"I'm going through some things with my feelings and myself. I barely sleep and I do nothing but think about how I'm worthless and how I shouldn't be here.\\n   I've never tried or contemplated suicide. I've always wanted to fix my issues, but I never get around to it.\\n   How can I change my feeling of being worthless to everyone?\",\n",
+    "    response=\"\",\n",
+    ")\n",
+    "\n",
+    "# Assuming gemma_lm is a language model that you're using to generate text\n",
+    "print(model.generate(prompt, max_length=256))\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "252c7c12-e763-4524-8cd4-7582431efb02",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\u001b[1m3136/3512\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m━━━\u001b[0m \u001b[1m4:11\u001b[0m 668ms/step - loss: 0.3683 - sparse_categorical_accuracy: 0.8940"
+     ]
+    }
+   ],
+   "source": [
+    "model.fit(data, epochs=1, batch_size=1, verbose=1)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "11bda333-9e55-4a62-9992-3d595d27ff54",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.12"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

README.md

@@ -1,12 +1,6 @@
 ---
-title: Mental Sage
-emoji: 🦀
-colorFrom: red
-colorTo: yellow
+title: Mental-Sage
+app_file: main.py
 sdk: gradio
 sdk_version: 5.5.0
-app_file: app.py
-pinned: false
 ---
-
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference

Untitled.ipynb

@@ -0,0 +1,138 @@
+{
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "fac5022c-df54-4824-8c57-98fe045372fd",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "2024-09-02 15:52:58.146880: I tensorflow/core/common_runtime/gpu/gpu_device.cc:2021] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 692 MB memory:  -> device: 0, name: Tesla T4, pci bus id: 0000:18:00.0, compute capability: 7.5\n",
+      "2024-09-02 15:52:58.148409: I tensorflow/core/common_runtime/gpu/gpu_device.cc:2021] Created device /job:localhost/replica:0/task:0/device:GPU:1 with 13673 MB memory:  -> device: 1, name: Tesla T4, pci bus id: 0000:19:00.0, compute capability: 7.5\n",
+      "2024-09-02 15:52:58.149801: I tensorflow/core/common_runtime/gpu/gpu_device.cc:2021] Created device /job:localhost/replica:0/task:0/device:GPU:2 with 13673 MB memory:  -> device: 2, name: Tesla T4, pci bus id: 0000:35:00.0, compute capability: 7.5\n",
+      "2024-09-02 15:52:58.151206: I tensorflow/core/common_runtime/gpu/gpu_device.cc:2021] Created device /job:localhost/replica:0/task:0/device:GPU:3 with 13673 MB memory:  -> device: 3, name: Tesla T4, pci bus id: 0000:36:00.0, compute capability: 7.5\n",
+      "2024-09-02 15:52:58.152545: I tensorflow/core/common_runtime/gpu/gpu_device.cc:2021] Created device /job:localhost/replica:0/task:0/device:GPU:4 with 13673 MB memory:  -> device: 4, name: Tesla T4, pci bus id: 0000:e7:00.0, compute capability: 7.5\n",
+      "2024-09-02 15:52:58.153906: I tensorflow/core/common_runtime/gpu/gpu_device.cc:2021] Created device /job:localhost/replica:0/task:0/device:GPU:5 with 13673 MB memory:  -> device: 5, name: Tesla T4, pci bus id: 0000:e8:00.0, compute capability: 7.5\n",
+      "2024-09-02 15:52:58.155220: I tensorflow/core/common_runtime/gpu/gpu_device.cc:2021] Created device /job:localhost/replica:0/task:0/device:GPU:6 with 13673 MB memory:  -> device: 6, name: Tesla T4, pci bus id: 0000:f4:00.0, compute capability: 7.5\n",
+      "2024-09-02 15:52:58.156627: I tensorflow/core/common_runtime/gpu/gpu_device.cc:2021] Created device /job:localhost/replica:0/task:0/device:GPU:7 with 13673 MB memory:  -> device: 7, name: Tesla T4, pci bus id: 0000:f5:00.0, compute capability: 7.5\n",
+      "2024-09-02 15:53:08.871520: W external/local_tsl/tsl/framework/bfc_allocator.cc:482] Allocator (GPU_0_bfc) ran out of memory trying to allocate 1.95GiB (rounded to 2097152000)requested by op StatelessRandomNormalV2\n",
+      "If the cause is memory fragmentation maybe the environment variable 'TF_GPU_ALLOCATOR=cuda_malloc_async' will improve the situation. \n",
+      "Current allocation summary follows.\n",
+      "Current allocation summary follows.\n",
+      "2024-09-02 15:53:08.871548: I external/local_tsl/tsl/framework/bfc_allocator.cc:1039] BFCAllocator dump for GPU_0_bfc\n",
+      "2024-09-02 15:53:08.871559: I external/local_tsl/tsl/framework/bfc_allocator.cc:1046] Bin (256): \tTotal Chunks: 6, Chunks in use: 6. 1.5KiB allocated for chunks. 1.5KiB in use in bin. 48B client-requested in use in bin.\n",
+      "2024-09-02 15:53:08.871566: I external/local_tsl/tsl/framework/bfc_allocator.cc:1046] Bin (512): \tTotal Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.\n",
+      "2024-09-02 15:53:08.871572: I external/local_tsl/tsl/framework/bfc_allocator.cc:1046] Bin (1024): \tTotal Chunks: 1, Chunks in use: 1. 1.2KiB allocated for chunks. 1.2KiB in use in bin. 1.0KiB client-requested in use in bin.\n",
+      "2024-09-02 15:53:08.871577: I external/local_tsl/tsl/framework/bfc_allocator.cc:1046] Bin (2048): \tTotal Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.\n",
+      "2024-09-02 15:53:08.871582: I external/local_tsl/tsl/framework/bfc_allocator.cc:1046] Bin (4096): \tTotal Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.\n",
+      "2024-09-02 15:53:08.871586: I external/local_tsl/tsl/framework/bfc_allocator.cc:1046] Bin (8192): \tTotal Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.\n",
+      "2024-09-02 15:53:08.871591: I external/local_tsl/tsl/framework/bfc_allocator.cc:1046] Bin (16384): \tTotal Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.\n",
+      "2024-09-02 15:53:08.871596: I external/local_tsl/tsl/framework/bfc_allocator.cc:1046] Bin (32768): \tTotal Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.\n",
+      "2024-09-02 15:53:08.871600: I external/local_tsl/tsl/framework/bfc_allocator.cc:1046] Bin (65536): \tTotal Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.\n",
+      "2024-09-02 15:53:08.871605: I external/local_tsl/tsl/framework/bfc_allocator.cc:1046] Bin (131072): \tTotal Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.\n",
+      "2024-09-02 15:53:08.871609: I external/local_tsl/tsl/framework/bfc_allocator.cc:1046] Bin (262144): \tTotal Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.\n",
+      "2024-09-02 15:53:08.871613: I external/local_tsl/tsl/framework/bfc_allocator.cc:1046] Bin (524288): \tTotal Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.\n",
+      "2024-09-02 15:53:08.871618: I external/local_tsl/tsl/framework/bfc_allocator.cc:1046] Bin (1048576): \tTotal Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.\n",
+      "2024-09-02 15:53:08.871622: I external/local_tsl/tsl/framework/bfc_allocator.cc:1046] Bin (2097152): \tTotal Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.\n",
+      "2024-09-02 15:53:08.871627: I external/local_tsl/tsl/framework/bfc_allocator.cc:1046] Bin (4194304): \tTotal Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.\n",
+      "2024-09-02 15:53:08.871631: I external/local_tsl/tsl/framework/bfc_allocator.cc:1046] Bin (8388608): \tTotal Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.\n",
+      "2024-09-02 15:53:08.871636: I external/local_tsl/tsl/framework/bfc_allocator.cc:1046] Bin (16777216): \tTotal Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.\n",
+      "2024-09-02 15:53:08.871640: I external/local_tsl/tsl/framework/bfc_allocator.cc:1046] Bin (33554432): \tTotal Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.\n",
+      "2024-09-02 15:53:08.871645: I external/local_tsl/tsl/framework/bfc_allocator.cc:1046] Bin (67108864): \tTotal Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.\n",
+      "2024-09-02 15:53:08.871650: I external/local_tsl/tsl/framework/bfc_allocator.cc:1046] Bin (134217728): \tTotal Chunks: 0, Chunks in use: 0. 0B allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.\n",
+      "2024-09-02 15:53:08.871656: I external/local_tsl/tsl/framework/bfc_allocator.cc:1046] Bin (268435456): \tTotal Chunks: 1, Chunks in use: 0. 692.62MiB allocated for chunks. 0B in use in bin. 0B client-requested in use in bin.\n",
+      "2024-09-02 15:53:08.871662: I external/local_tsl/tsl/framework/bfc_allocator.cc:1062] Bin for 1.95GiB was 256.00MiB, Chunk State: \n",
+      "2024-09-02 15:53:08.871670: I external/local_tsl/tsl/framework/bfc_allocator.cc:1068]   Size: 692.62MiB | Requested Size: 0B | in_use: 0 | bin_num: 20, prev:   Size: 256B | Requested Size: 16B | in_use: 1 | bin_num: -1\n",
+      "2024-09-02 15:53:08.871674: I external/local_tsl/tsl/framework/bfc_allocator.cc:1075] Next region of size 726269952\n",
+      "2024-09-02 15:53:08.871681: I external/local_tsl/tsl/framework/bfc_allocator.cc:1095] InUse at 7f7fac000000 of size 256 next 1\n",
+      "2024-09-02 15:53:08.871685: I external/local_tsl/tsl/framework/bfc_allocator.cc:1095] InUse at 7f7fac000100 of size 1280 next 2\n",
+      "2024-09-02 15:53:08.871689: I external/local_tsl/tsl/framework/bfc_allocator.cc:1095] InUse at 7f7fac000600 of size 256 next 3\n",
+      "2024-09-02 15:53:08.871692: I external/local_tsl/tsl/framework/bfc_allocator.cc:1095] InUse at 7f7fac000700 of size 256 next 4\n",
+      "2024-09-02 15:53:08.871695: I external/local_tsl/tsl/framework/bfc_allocator.cc:1095] InUse at 7f7fac000800 of size 256 next 5\n",
+      "2024-09-02 15:53:08.871700: I external/local_tsl/tsl/framework/bfc_allocator.cc:1095] InUse at 7f7fac000900 of size 256 next 6\n",
+      "2024-09-02 15:53:08.871703: I external/local_tsl/tsl/framework/bfc_allocator.cc:1095] InUse at 7f7fac000a00 of size 256 next 7\n",
+      "2024-09-02 15:53:08.871708: I external/local_tsl/tsl/framework/bfc_allocator.cc:1095] Free  at 7f7fac000b00 of size 726267136 next 18446744073709551615\n",
+      "2024-09-02 15:53:08.871713: I external/local_tsl/tsl/framework/bfc_allocator.cc:1100]      Summary of in-use Chunks by size: \n",
+      "2024-09-02 15:53:08.871718: I external/local_tsl/tsl/framework/bfc_allocator.cc:1103] 6 Chunks of size 256 totalling 1.5KiB\n",
+      "2024-09-02 15:53:08.871723: I external/local_tsl/tsl/framework/bfc_allocator.cc:1103] 1 Chunks of size 1280 totalling 1.2KiB\n",
+      "2024-09-02 15:53:08.871727: I external/local_tsl/tsl/framework/bfc_allocator.cc:1107] Sum Total of in-use chunks: 2.8KiB\n",
+      "2024-09-02 15:53:08.871732: I external/local_tsl/tsl/framework/bfc_allocator.cc:1109] Total bytes in pool: 726269952 memory_limit_: 726269952 available bytes: 0 curr_region_allocation_bytes_: 1452539904\n",
+      "2024-09-02 15:53:08.871740: I external/local_tsl/tsl/framework/bfc_allocator.cc:1114] Stats: \n",
+      "Limit:                       726269952\n",
+      "InUse:                            2816\n",
+      "MaxInUse:                         2816\n",
+      "NumAllocs:                           9\n",
+      "MaxAllocSize:                     1280\n",
+      "Reserved:                            0\n",
+      "PeakReserved:                        0\n",
+      "LargestFreeBlock:                    0\n",
+      "\n",
+      "2024-09-02 15:53:08.871746: W external/local_tsl/tsl/framework/bfc_allocator.cc:494] *___________________________________________________________________________________________________\n",
+      "2024-09-02 15:53:08.871766: W tensorflow/core/framework/op_kernel.cc:1840] OP_REQUIRES failed at stateless_random_ops_v2.cc:64 : RESOURCE_EXHAUSTED: OOM when allocating tensor with shape[256000,2048] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc\n",
+      "2024-09-02 15:53:08.871780: I tensorflow/core/framework/local_rendezvous.cc:404] Local rendezvous is aborting with status: RESOURCE_EXHAUSTED: OOM when allocating tensor with shape[256000,2048] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc\n"
+     ]
+    },
+    {
+     "ename": "ResourceExhaustedError",
+     "evalue": "{{function_node __wrapped__StatelessRandomNormalV2_device_/job:localhost/replica:0/task:0/device:GPU:0}} OOM when allocating tensor with shape[256000,2048] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc [Op:StatelessRandomNormalV2]",
+     "output_type": "error",
+     "traceback": [
+      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
+      "\u001b[0;31mResourceExhaustedError\u001b[0m                    Traceback (most recent call last)",
+      "Cell \u001b[0;32mIn[4], line 6\u001b[0m\n\u001b[1;32m      4\u001b[0m os\u001b[38;5;241m.\u001b[39menviron[\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mKAGGLE_KEY\u001b[39m\u001b[38;5;124m\"\u001b[39m] \u001b[38;5;241m=\u001b[39m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124m9a33b6e88bcb6058b1281d777fa6808d\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m      5\u001b[0m \u001b[38;5;28;01mimport\u001b[39;00m \u001b[38;5;21;01mkeras_nlp\u001b[39;00m\n\u001b[0;32m----> 6\u001b[0m gemma_lm \u001b[38;5;241m=\u001b[39m \u001b[43mkeras_nlp\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mmodels\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mGemmaCausalLM\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mfrom_preset\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mgemma_2b_en\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m)\u001b[49m\n\u001b[1;32m      7\u001b[0m gemma_lm\u001b[38;5;241m.\u001b[39mgenerate(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mKeras is a\u001b[39m\u001b[38;5;124m\"\u001b[39m, max_length\u001b[38;5;241m=\u001b[39m\u001b[38;5;241m30\u001b[39m)\n",
+      "File \u001b[0;32m~/.local/lib/python3.10/site-packages/keras_nlp/src/models/task.py:264\u001b[0m, in \u001b[0;36mTask.from_preset\u001b[0;34m(cls, preset, load_weights, **kwargs)\u001b[0m\n\u001b[1;32m    262\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mdtype\u001b[39m\u001b[38;5;124m\"\u001b[39m \u001b[38;5;129;01min\u001b[39;00m kwargs:\n\u001b[1;32m    263\u001b[0m     config_overrides[\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mdtype\u001b[39m\u001b[38;5;124m\"\u001b[39m] \u001b[38;5;241m=\u001b[39m kwargs\u001b[38;5;241m.\u001b[39mpop(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mdtype\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n\u001b[0;32m--> 264\u001b[0m backbone \u001b[38;5;241m=\u001b[39m \u001b[43mbackbone_preset_cls\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mfrom_preset\u001b[49m\u001b[43m(\u001b[49m\n\u001b[1;32m    265\u001b[0m \u001b[43m    \u001b[49m\u001b[43mpreset\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    266\u001b[0m \u001b[43m    \u001b[49m\u001b[43mload_weights\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mload_weights\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    267\u001b[0m \u001b[43m    \u001b[49m\u001b[43mconfig_overrides\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mconfig_overrides\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    268\u001b[0m \u001b[43m\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    269\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mpreprocessor\u001b[39m\u001b[38;5;124m\"\u001b[39m \u001b[38;5;129;01min\u001b[39;00m kwargs:\n\u001b[1;32m    270\u001b[0m     preprocessor \u001b[38;5;241m=\u001b[39m kwargs\u001b[38;5;241m.\u001b[39mpop(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mpreprocessor\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n",
+      "File \u001b[0;32m~/.local/lib/python3.10/site-packages/keras_nlp/src/models/backbone.py:190\u001b[0m, in \u001b[0;36mBackbone.from_preset\u001b[0;34m(cls, preset, load_weights, **kwargs)\u001b[0m\n\u001b[1;32m    183\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28missubclass\u001b[39m(preset_cls, \u001b[38;5;28mcls\u001b[39m):\n\u001b[1;32m    184\u001b[0m     \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mValueError\u001b[39;00m(\n\u001b[1;32m    185\u001b[0m         \u001b[38;5;124mf\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mPreset has type `\u001b[39m\u001b[38;5;132;01m{\u001b[39;00mpreset_cls\u001b[38;5;241m.\u001b[39m\u001b[38;5;18m__name__\u001b[39m\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m` which is not a \u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m    186\u001b[0m         \u001b[38;5;124mf\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124ma subclass of calling class `\u001b[39m\u001b[38;5;132;01m{\u001b[39;00m\u001b[38;5;28mcls\u001b[39m\u001b[38;5;241m.\u001b[39m\u001b[38;5;18m__name__\u001b[39m\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m`. Call \u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m    187\u001b[0m         \u001b[38;5;124mf\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124m`from_preset` directly on `\u001b[39m\u001b[38;5;132;01m{\u001b[39;00mpreset_cls\u001b[38;5;241m.\u001b[39m\u001b[38;5;18m__name__\u001b[39m\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m` instead.\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m    188\u001b[0m     )\n\u001b[0;32m--> 190\u001b[0m backbone \u001b[38;5;241m=\u001b[39m \u001b[43mload_serialized_object\u001b[49m\u001b[43m(\u001b[49m\u001b[43mpreset\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mCONFIG_FILE\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    191\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m load_weights:\n\u001b[1;32m    192\u001b[0m     jax_memory_cleanup(backbone)\n",
+      "File \u001b[0;32m~/.local/lib/python3.10/site-packages/keras_nlp/src/utils/preset_utils.py:569\u001b[0m, in \u001b[0;36mload_serialized_object\u001b[0;34m(preset, config_file, config_overrides)\u001b[0m\n\u001b[1;32m    567\u001b[0m config \u001b[38;5;241m=\u001b[39m load_config(preset, config_file)\n\u001b[1;32m    568\u001b[0m config[\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mconfig\u001b[39m\u001b[38;5;124m\"\u001b[39m] \u001b[38;5;241m=\u001b[39m {\u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mconfig[\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mconfig\u001b[39m\u001b[38;5;124m\"\u001b[39m], \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mconfig_overrides}\n\u001b[0;32m--> 569\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mkeras\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43msaving\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mdeserialize_keras_object\u001b[49m\u001b[43m(\u001b[49m\u001b[43mconfig\u001b[49m\u001b[43m)\u001b[49m\n",
+      "File \u001b[0;32m~/.local/lib/python3.10/site-packages/keras/src/saving/serialization_lib.py:718\u001b[0m, in \u001b[0;36mdeserialize_keras_object\u001b[0;34m(config, custom_objects, safe_mode, **kwargs)\u001b[0m\n\u001b[1;32m    716\u001b[0m \u001b[38;5;28;01mwith\u001b[39;00m custom_obj_scope, safe_mode_scope:\n\u001b[1;32m    717\u001b[0m     \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[0;32m--> 718\u001b[0m         instance \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mcls\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mfrom_config\u001b[49m\u001b[43m(\u001b[49m\u001b[43minner_config\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    719\u001b[0m     \u001b[38;5;28;01mexcept\u001b[39;00m \u001b[38;5;167;01mTypeError\u001b[39;00m \u001b[38;5;28;01mas\u001b[39;00m e:\n\u001b[1;32m    720\u001b[0m         \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mTypeError\u001b[39;00m(\n\u001b[1;32m    721\u001b[0m             \u001b[38;5;124mf\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;132;01m{\u001b[39;00m\u001b[38;5;28mcls\u001b[39m\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m could not be deserialized properly. Please\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m    722\u001b[0m             \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124m ensure that components that are Python object\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m    726\u001b[0m             \u001b[38;5;124mf\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;130;01m\\n\u001b[39;00m\u001b[38;5;130;01m\\n\u001b[39;00m\u001b[38;5;124mconfig=\u001b[39m\u001b[38;5;132;01m{\u001b[39;00mconfig\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m.\u001b[39m\u001b[38;5;130;01m\\n\u001b[39;00m\u001b[38;5;130;01m\\n\u001b[39;00m\u001b[38;5;124mException encountered: \u001b[39m\u001b[38;5;132;01m{\u001b[39;00me\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m    727\u001b[0m         )\n",
+      "File \u001b[0;32m~/.local/lib/python3.10/site-packages/keras_nlp/src/models/backbone.py:119\u001b[0m, in \u001b[0;36mBackbone.from_config\u001b[0;34m(cls, config)\u001b[0m\n\u001b[1;32m    115\u001b[0m \u001b[38;5;129m@classmethod\u001b[39m\n\u001b[1;32m    116\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mfrom_config\u001b[39m(\u001b[38;5;28mcls\u001b[39m, config):\n\u001b[1;32m    117\u001b[0m     \u001b[38;5;66;03m# The default `from_config()` for functional models will return a\u001b[39;00m\n\u001b[1;32m    118\u001b[0m     \u001b[38;5;66;03m# vanilla `keras.Model`. We override it to get a subclass instance back.\u001b[39;00m\n\u001b[0;32m--> 119\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43mcls\u001b[39;49m\u001b[43m(\u001b[49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[38;5;241;43m*\u001b[39;49m\u001b[43mconfig\u001b[49m\u001b[43m)\u001b[49m\n",
+      "File \u001b[0;32m~/.local/lib/python3.10/site-packages/keras_nlp/src/models/gemma/gemma_backbone.py:171\u001b[0m, in \u001b[0;36mGemmaBackbone.__init__\u001b[0;34m(self, vocabulary_size, num_layers, num_query_heads, num_key_value_heads, hidden_dim, intermediate_dim, head_dim, query_head_dim_normalize, use_post_ffw_norm, use_post_attention_norm, attention_logit_soft_cap, final_logit_soft_cap, use_sliding_window_attention, sliding_window_size, layer_norm_epsilon, dropout, dtype, **kwargs)\u001b[0m\n\u001b[1;32m    165\u001b[0m token_id_input \u001b[38;5;241m=\u001b[39m keras\u001b[38;5;241m.\u001b[39mInput(\n\u001b[1;32m    166\u001b[0m     shape\u001b[38;5;241m=\u001b[39m(\u001b[38;5;28;01mNone\u001b[39;00m,), dtype\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mfloat32\u001b[39m\u001b[38;5;124m\"\u001b[39m, name\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mtoken_ids\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m    167\u001b[0m )\n\u001b[1;32m    168\u001b[0m padding_mask_input \u001b[38;5;241m=\u001b[39m keras\u001b[38;5;241m.\u001b[39mInput(\n\u001b[1;32m    169\u001b[0m     shape\u001b[38;5;241m=\u001b[39m(\u001b[38;5;28;01mNone\u001b[39;00m,), dtype\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mfloat32\u001b[39m\u001b[38;5;124m\"\u001b[39m, name\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mpadding_mask\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m    170\u001b[0m )\n\u001b[0;32m--> 171\u001b[0m x \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mtoken_embedding\u001b[49m\u001b[43m(\u001b[49m\u001b[43mtoken_id_input\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    172\u001b[0m x \u001b[38;5;241m=\u001b[39m x \u001b[38;5;241m*\u001b[39m ops\u001b[38;5;241m.\u001b[39mcast(ops\u001b[38;5;241m.\u001b[39msqrt(hidden_dim), x\u001b[38;5;241m.\u001b[39mdtype)\n\u001b[1;32m    173\u001b[0m \u001b[38;5;28;01mfor\u001b[39;00m transformer_layer \u001b[38;5;129;01min\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mtransformer_layers:\n",
+      "File \u001b[0;32m~/.local/lib/python3.10/site-packages/keras/src/utils/traceback_utils.py:122\u001b[0m, in \u001b[0;36mfilter_traceback.<locals>.error_handler\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m    119\u001b[0m     filtered_tb \u001b[38;5;241m=\u001b[39m _process_traceback_frames(e\u001b[38;5;241m.\u001b[39m__traceback__)\n\u001b[1;32m    120\u001b[0m     \u001b[38;5;66;03m# To get the full stack trace, call:\u001b[39;00m\n\u001b[1;32m    121\u001b[0m     \u001b[38;5;66;03m# `keras.config.disable_traceback_filtering()`\u001b[39;00m\n\u001b[0;32m--> 122\u001b[0m     \u001b[38;5;28;01mraise\u001b[39;00m e\u001b[38;5;241m.\u001b[39mwith_traceback(filtered_tb) \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m\n\u001b[1;32m    123\u001b[0m \u001b[38;5;28;01mfinally\u001b[39;00m:\n\u001b[1;32m    124\u001b[0m     \u001b[38;5;28;01mdel\u001b[39;00m filtered_tb\n",
+      "File \u001b[0;32m~/.local/lib/python3.10/site-packages/keras_nlp/src/layers/modeling/reversible_embedding.py:115\u001b[0m, in \u001b[0;36mReversibleEmbedding.build\u001b[0;34m(self, inputs_shape)\u001b[0m\n\u001b[1;32m    114\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mbuild\u001b[39m(\u001b[38;5;28mself\u001b[39m, inputs_shape\u001b[38;5;241m=\u001b[39m\u001b[38;5;28;01mNone\u001b[39;00m):\n\u001b[0;32m--> 115\u001b[0m     \u001b[38;5;28;43msuper\u001b[39;49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mbuild\u001b[49m\u001b[43m(\u001b[49m\u001b[43minputs_shape\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m    117\u001b[0m     \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mtie_weights:\n\u001b[1;32m    118\u001b[0m         \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mreverse_embeddings \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39madd_weight(\n\u001b[1;32m    119\u001b[0m             name\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mreverse_embeddings\u001b[39m\u001b[38;5;124m\"\u001b[39m,\n\u001b[1;32m    120\u001b[0m             shape\u001b[38;5;241m=\u001b[39m(\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39moutput_dim, \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39minput_dim),\n\u001b[1;32m    121\u001b[0m             initializer\u001b[38;5;241m=\u001b[39m\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39membeddings_initializer,\n\u001b[1;32m    122\u001b[0m             dtype\u001b[38;5;241m=\u001b[39m\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mdtype,\n\u001b[1;32m    123\u001b[0m         )\n",
+      "\u001b[0;31mResourceExhaustedError\u001b[0m: {{function_node __wrapped__StatelessRandomNormalV2_device_/job:localhost/replica:0/task:0/device:GPU:0}} OOM when allocating tensor with shape[256000,2048] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc [Op:StatelessRandomNormalV2]"
+     ]
+    }
+   ],
+   "source": [
+    "# Set Kaggle API credentials\n",
+    "import os\n",
+    "os.environ[\"KAGGLE_USERNAME\"] = \"rogerkorantenng\"\n",
+    "os.environ[\"KAGGLE_KEY\"] = \"9a33b6e88bcb6058b1281d777fa6808d\"\n",
+    "import keras_nlp\n",
+    "gemma_lm = keras_nlp.models.GemmaCausalLM.from_preset(\"gemma_2b_en\")\n",
+    "gemma_lm.generate(\"Keras is a\", max_length=30)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "e5488db1-48b2-4d0c-a4cd-1329a333b3f9",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.12"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

Untitled1.ipynb

@@ -0,0 +1,94 @@
+{
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "be79942c-2080-452a-a349-ea19b7611c85",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "2024-09-03 17:04:36.485116: I tensorflow/core/util/port.cc:153] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.\n",
+      "2024-09-03 17:04:36.500874: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:485] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered\n",
+      "2024-09-03 17:04:36.519822: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:8454] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered\n",
+      "2024-09-03 17:04:36.525533: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1452] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered\n",
+      "2024-09-03 17:04:36.539736: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.\n",
+      "To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "'3.5.0'"
+      ]
+     },
+     "execution_count": 2,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "import keras\n",
+    "keras.__version__"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "00e538b8-91c1-401b-9cc9-320fc496a828",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Name: keras\n",
+      "Version: 3.5.0\n",
+      "Summary: Multi-backend Keras.\n",
+      "Home-page: https://github.com/keras-team/keras\n",
+      "Author: Keras team\n",
+      "Author-email: keras-users@googlegroups.com\n",
+      "License: Apache License 2.0\n",
+      "Location: /home/workbench/.local/lib/python3.10/site-packages\n",
+      "Requires: absl-py, h5py, ml-dtypes, namex, numpy, optree, packaging, rich\n",
+      "Required-by: tensorflow\n"
+     ]
+    }
+   ],
+   "source": [
+    "!pip show keras\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "85cff0f4-2cb7-4c20-ba0c-a1c6198bf8dd",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.12"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

chatbot/.env

@@ -0,0 +1,4 @@
+GPT4_API_KEY=XTtd4wdC0mlEdbQqp0KWOe9aWrlg4z4HurxXlgBf5ASUvrB9iPkwJQQJ99AKACHYHv6XJ3w3AAABACOGaeSg
+GPT4O_API_KEY=XTtd4wdC0mlEdbQqp0KWOe9aWrlg4z4HurxXlgBf5ASUvrB9iPkwJQQJ99AKACHYHv6XJ3w3AAABACOGaeSg
+GPT35_TURBO_API_KEY=cfe75d17d47c40318db6230d2677d84f
+GPT4_32K_API_KEY=AebO97c0w0lnrppFNpbuPVueKth7HfGOdMbKeaX5R2Uql6wvJA5qJQQJ99AKACL93NaXJ3w3AAABACOGh8MD

chatbot/.gradio/certificate.pem

@@ -0,0 +1,31 @@
+-----BEGIN CERTIFICATE-----
+MIIFazCCA1OgAwIBAgIRAIIQz7DSQONZRGPgu2OCiwAwDQYJKoZIhvcNAQELBQAw
+TzELMAkGA1UEBhMCVVMxKTAnBgNVBAoTIEludGVybmV0IFNlY3VyaXR5IFJlc2Vh
+cmNoIEdyb3VwMRUwEwYDVQQDEwxJU1JHIFJvb3QgWDEwHhcNMTUwNjA0MTEwNDM4
+WhcNMzUwNjA0MTEwNDM4WjBPMQswCQYDVQQGEwJVUzEpMCcGA1UEChMgSW50ZXJu
+ZXQgU2VjdXJpdHkgUmVzZWFyY2ggR3JvdXAxFTATBgNVBAMTDElTUkcgUm9vdCBY
+MTCCAiIwDQYJKoZIhvcNAQEBBQADggIPADCCAgoCggIBAK3oJHP0FDfzm54rVygc
+h77ct984kIxuPOZXoHj3dcKi/vVqbvYATyjb3miGbESTtrFj/RQSa78f0uoxmyF+
+0TM8ukj13Xnfs7j/EvEhmkvBioZxaUpmZmyPfjxwv60pIgbz5MDmgK7iS4+3mX6U
+A5/TR5d8mUgjU+g4rk8Kb4Mu0UlXjIB0ttov0DiNewNwIRt18jA8+o+u3dpjq+sW
+T8KOEUt+zwvo/7V3LvSye0rgTBIlDHCNAymg4VMk7BPZ7hm/ELNKjD+Jo2FR3qyH
+B5T0Y3HsLuJvW5iB4YlcNHlsdu87kGJ55tukmi8mxdAQ4Q7e2RCOFvu396j3x+UC
+B5iPNgiV5+I3lg02dZ77DnKxHZu8A/lJBdiB3QW0KtZB6awBdpUKD9jf1b0SHzUv
+KBds0pjBqAlkd25HN7rOrFleaJ1/ctaJxQZBKT5ZPt0m9STJEadao0xAH0ahmbWn
+OlFuhjuefXKnEgV4We0+UXgVCwOPjdAvBbI+e0ocS3MFEvzG6uBQE3xDk3SzynTn
+jh8BCNAw1FtxNrQHusEwMFxIt4I7mKZ9YIqioymCzLq9gwQbooMDQaHWBfEbwrbw
+qHyGO0aoSCqI3Haadr8faqU9GY/rOPNk3sgrDQoo//fb4hVC1CLQJ13hef4Y53CI
+rU7m2Ys6xt0nUW7/vGT1M0NPAgMBAAGjQjBAMA4GA1UdDwEB/wQEAwIBBjAPBgNV
+HRMBAf8EBTADAQH/MB0GA1UdDgQWBBR5tFnme7bl5AFzgAiIyBpY9umbbjANBgkq
+hkiG9w0BAQsFAAOCAgEAVR9YqbyyqFDQDLHYGmkgJykIrGF1XIpu+ILlaS/V9lZL
+ubhzEFnTIZd+50xx+7LSYK05qAvqFyFWhfFQDlnrzuBZ6brJFe+GnY+EgPbk6ZGQ
+3BebYhtF8GaV0nxvwuo77x/Py9auJ/GpsMiu/X1+mvoiBOv/2X/qkSsisRcOj/KK
+NFtY2PwByVS5uCbMiogziUwthDyC3+6WVwW6LLv3xLfHTjuCvjHIInNzktHCgKQ5
+ORAzI4JMPJ+GslWYHb4phowim57iaztXOoJwTdwJx4nLCgdNbOhdjsnvzqvHu7Ur
+TkXWStAmzOVyyghqpZXjFaH3pO3JLF+l+/+sKAIuvtd7u+Nxe5AW0wdeRlN8NwdC
+jNPElpzVmbUq4JUagEiuTDkHzsxHpFKVK7q4+63SM1N95R1NbdWhscdCb+ZAJzVc
+oyi3B43njTOQ5yOf+1CceWxG1bQVs5ZufpsMljq4Ui0/1lvh+wjChP4kqKOJ2qxq
+4RgqsahDYVvTH9w7jXbyLeiNdd8XM2w9U/t7y0Ff/9yi0GE44Za4rF2LN9d11TPA
+mRGunUHBcnWEvgJBQl9nJEiU0Zsnvgc/ubhPgXRR4Xq37Z0j4r7g1SgEEzwxA57d
+emyPxgcYxn/eR44/KJ4EBs+lVDR3veyJm+kXQ99b21/+jh5Xos1AnX5iItreGCc=
+-----END CERTIFICATE-----

chatbot/README.md

@@ -0,0 +1,6 @@
+---
+title: chatbot
+app_file: main.py
+sdk: gradio
+sdk_version: 4.42.0
+---

chatbot/app.py

@@ -0,0 +1,157 @@
+import gradio as gr
+import os
+import requests
+from dotenv import load_dotenv
+
+# Load environment variables from .env file
+load_dotenv()
+
+# Define endpoints and securely load API keys from environment variables
+model_config = {
+    "gpt-4": {
+        "endpoint": "https://roger-m38jr9pd-eastus2.openai.azure.com/openai/deployments/gpt-4/chat/completions?api-version=2024-08-01-preview",
+        "api_key": os.getenv("GPT4_API_KEY")
+    },
+    "gpt-4o": {
+        "endpoint": "https://roger-m38jr9pd-eastus2.openai.azure.com/openai/deployments/gpt-4o/chat/completions?api-version=2024-08-01-preview",
+        "api_key": os.getenv("GPT4O_API_KEY")
+    },
+    "gpt-35-turbo": {
+        "endpoint": "https://rogerkoranteng.openai.azure.com/openai/deployments/gpt-35-turbo/chat/completions?api-version=2024-08-01-preview",
+        "api_key": os.getenv("GPT35_TURBO_API_KEY")
+    },
+    "gpt-4-32k": {
+        "endpoint": "https://roger-m38orjxq-australiaeast.openai.azure.com/openai/deployments/gpt-4-32k/chat/completions?api-version=2024-08-01-preview",
+        "api_key": os.getenv("GPT4_32K_API_KEY")
+    }
+}
+
+
+# Function to generate a response from Azure OpenAI
+def generate_response(model_choice, user_message, chat_history):
+    model_info = model_config.get(model_choice)
+
+    if not model_info:
+        return "Invalid model selection. Please choose a valid model.", chat_history
+
+    chat_history.append({"role": "user", "content": user_message})
+
+    headers = {
+        "Content-Type": "application/json",
+        "api-key": model_info['api_key'],
+    }
+
+    data = {
+        "messages": chat_history,
+        "max_tokens": 150,
+        "temperature": 0.7,
+    }
+
+    try:
+        response = requests.post(
+            model_info["endpoint"],
+            headers=headers,
+            json=data
+        )
+
+        response.raise_for_status()
+        response_data = response.json()
+
+        assistant_message = response_data['choices'][0]['message']['content']
+        chat_history.append({"role": "assistant", "content": assistant_message})
+
+        return assistant_message, chat_history
+
+    except requests.exceptions.RequestException as e:
+        return f"Error: {e}", chat_history
+
+
+# Function to format chat history with improved readability
+def format_chat_history(history):
+    formatted_history = ""
+    for message in history:
+        role = "User:" if message["role"] == "user" else "Sage:"
+        formatted_history += f"{role} {message['content']}\n\n"  # Message content with a line break
+    return formatted_history.strip()  # Remove any trailing whitespace
+
+
+# List of available models
+azure_models = [
+    "gpt-4",
+    "gpt-4o",
+    "gpt-35-turbo",
+    "gpt-4-32k"
+]
+
+
+# Function to handle model change status update
+def change_model(model_choice):
+    return f"Selected model: {model_choice}"
+
+
+# Create the Gradio interface
+with gr.Blocks() as interface:
+    gr.Markdown("## Sage - Your Mental Health Advisor")
+
+    with gr.Tab("Model Selection"):
+        gr.Markdown("### Select Model for Chat")
+        model_dropdown = gr.Dropdown(
+            choices=azure_models,
+            label="Choose a Model",
+            value=azure_models[0],  # Default model
+            interactive=True
+        )
+
+        # Add status update on model change
+        status_textbox = gr.Textbox(label="Model Selection Status", value="Selected model: gpt-4", interactive=False)
+        model_dropdown.change(change_model, inputs=model_dropdown, outputs=status_textbox)
+
+        gr.Markdown("The selected model will be used for chat interaction.")
+
+    with gr.Tab("Chat Interface"):
+        gr.Markdown("### Chat with Sage - Your Mental Health Advisor")
+        chat_history_state = gr.State([])  # Store chat history
+        model_choice_state = gr.State(azure_models[0])  # Default model
+        user_message = gr.Textbox(
+            placeholder="Hello, I am Sage. How can I assist you today?",
+            label="Your Message",
+            lines=2,
+            scale=7
+        )
+
+        model_dropdown.change(
+            lambda x: x, inputs=model_dropdown, outputs=model_choice_state
+        )
+
+        assistant_response = gr.Textbox(label="Assistant's Response")
+        submit_button = gr.Button("Send Message")
+
+        submit_button.click(
+            generate_response,
+            inputs=[model_choice_state, user_message, chat_history_state],
+            outputs=[assistant_response, chat_history_state]
+        )
+
+    # Chat History Tab
+    with gr.Tab("Chat History"):
+        gr.Markdown("### Chat History")
+
+        # Add the fetch history button inside the "Chat History" tab
+        fetch_history_button = gr.Button("Fetch Chat History")
+        chat_history_display = gr.Textbox(label="Chat History", interactive=False, lines=10)
+
+
+        # Function to fetch and display chat history
+        def fetch_chat_history(chat_history):
+            return format_chat_history(chat_history)
+
+
+        # Bind the fetch history button to fetch the chat history
+        fetch_history_button.click(
+            fetch_chat_history,
+            inputs=chat_history_state,
+            outputs=chat_history_display
+        )
+
+    # Launch the Gradio app
+    interface.launch(server_name="0.0.0.0", server_port=8080, share=True)

chatbot/app2.pyn

@@ -0,0 +1,226 @@
+import os
+import gradio as gr
+import pandas as pd
+import numpy as np
+import chromadb
+from chromadb.config import Settings
+from io import StringIO
+from sentence_transformers import SentenceTransformer
+import plotly.express as px
+from sklearn.manifold import TSNE
+
+
+# Constants for Model Configuration
+MODEL_CONFIG = {
+    "gpt-4": {
+        "endpoint": "https://roger-m38jr9pd-eastus2.openai.azure.com/openai/deployments/gpt-4/chat/completions?api-version=2024-08-01-preview",
+        "api_key": os.getenv("GPT4_API_KEY")
+    },
+    "gpt-4o": {
+        "endpoint": "https://roger-m38jr9pd-eastus2.openai.azure.com/openai/deployments/gpt-4o/chat/completions?api-version=2024-08-01-preview",
+        "api_key": os.getenv("GPT4O_API_KEY")
+    },
+    "gpt-35-turbo": {
+        "endpoint": "https://rogerkoranteng.openai.azure.com/openai/deployments/gpt-35-turbo/chat/completions?api-version=2024-08-01-preview",
+        "api_key": os.getenv("GPT35_TURBO_API_KEY")
+    },
+    "gpt-4-32k": {
+        "endpoint": "https://roger-m38orjxq-australiaeast.openai.azure.com/openai/deployments/gpt-4-32k/chat/completions?api-version=2024-08-01-preview",
+        "api_key": os.getenv("GPT4_32K_API_KEY")
+    }
+}
+
+# Initialize Chroma client with DuckDB and Parquet for persistence
+chroma_client = chromadb.Client()
+
+
+# Functions for Data Processing and Embedding
+
+def process_csv_text(temp_file):
+    """Process the uploaded CSV and return the dataframe and column options."""
+    if isinstance(temp_file, str):
+        df = pd.read_csv(StringIO(temp_file))
+    else:
+        df = pd.read_csv(temp_file.name, header='infer', sep=',')
+    return df, gr.Dropdown.update(choices=list(df.columns))
+
+
+def insert_or_update_chroma(col, table, model_name, similarity_metric):
+    """Insert or update embeddings in ChromaDB."""
+    try:
+        collection = chroma_client.create_collection(
+            name="my_collection",
+            embedding_function=SentenceTransformer(model_name),
+            metadata={"hnsw:space": similarity_metric}
+        )
+    except Exception:
+        print("Collection exists, deleting it")
+        chroma_client.delete_collection(name='my_collection')
+        collection = chroma_client.create_collection(
+            name="my_collection",
+            embedding_function=SentenceTransformer(model_name),
+            metadata={"hnsw:space": similarity_metric}
+        )
+
+    if collection:
+        try:
+            collection.add(
+                documents=list(table[col]),
+                metadatas=[{"source": i} for i in range(len(table))],
+                ids=[str(i + 1) for i in range(len(table))]
+            )
+            return "Embedding calculations and insertions successful"
+        except Exception as e:
+            return f"Error in embedding calculations: {e}"
+
+
+def show_fig():
+    """Show t-SNE 2D plot for embeddings."""
+    collection = chroma_client.get_collection(name="my_collection")
+    embeddings = collection.get(include=['embeddings', 'documents'])
+
+    df = pd.DataFrame({
+        'text': embeddings['documents'],
+        'embedding': embeddings['embeddings']
+    })
+
+    embeddings_np = np.array(df['embedding'].tolist())
+    tsne = TSNE(n_components=2, random_state=42)
+    transformed = tsne.fit_transform(embeddings_np)
+
+    df['tsne_x'] = transformed[:, 0]
+    df['tsne_y'] = transformed[:, 1]
+
+    fig = px.scatter(df, x='tsne_x', y='tsne_y', hover_name='text')
+    return fig, transformed
+
+
+def show_test_string_fig(test_string, tsne, model_name, similarity_metric):
+    """Show t-SNE plot with test string to compare embeddings."""
+    collection = chroma_client.get_collection(name="my_collection",
+                                              embedding_function=SentenceTransformer(model_name))
+
+    collection.add(
+        documents=[test_string],
+        metadatas=[{"source": 'test'}],
+        ids=['test_sample']
+    )
+
+    embeddings = collection.get(include=['embeddings', 'documents'])
+    df = pd.DataFrame({
+        'text': embeddings['documents'],
+        'embedding': embeddings['embeddings'],
+        'set': ['orig' if document != test_string else 'test_string' for document in embeddings["documents"]]
+    })
+
+    embeddings_np = np.array(df['embedding'].tolist())
+    transformed = tsne.transform(embeddings_np)
+
+    df['tsne_x'] = transformed[:, 0]
+    df['tsne_y'] = transformed[:, 1]
+
+    fig = px.scatter(df, x='tsne_x', y='tsne_y', hover_name='text', color='set')
+    return fig, tsne
+
+
+def ask_gpt(message, messages_history, embedding_model, system_prompt, temperature, max_tokens, chatgpt_model):
+    """Interacts with the OpenAI API using Azure endpoint."""
+    if len(messages_history) < 1:
+        messages_history = [{"role": "system", "content": system_prompt}]
+
+    model_info = MODEL_CONFIG[chatgpt_model]
+    headers = {"Content-Type": "application/json", "api-key": model_info["api_key"]}
+
+    message = retrieve_similar(message, embedding_model)
+    messages_history += [{"role": "user", "content": message}]
+
+    response = openai.ChatCompletion.create(
+        model=chatgpt_model,
+        messages=messages_history,
+        temperature=temperature,
+        max_tokens=max_tokens
+    )
+
+    return response['choices'][0]['message']['content'], messages_history
+
+
+def retrieve_similar(prompt, embedding_model):
+    """Retrieve similar documents from ChromaDB to enhance context."""
+    # Initialize SentenceTransformer correctly
+    embedding_function = SentenceTransformer(embedding_model)
+
+    collection = chroma_client.get_collection(
+        name="my_collection",
+        embedding_function=embedding_function
+    )
+
+    results = collection.query(query_texts=prompt, n_results=10)
+    additional_context = ''
+    for i, document in enumerate(results['documents'][0]):
+        additional_context += f'{i + 1}. {document}\n'
+
+    return additional_context + f'Question: {prompt}'
+
+
+
+# Gradio Interface Setup
+
+def build_gradio_ui():
+    """Setup the complete Gradio UI."""
+    with gr.Blocks() as demo:
+        # Tab 1: Upload CSV and Display Data
+        with gr.Tab("Upload data"):
+            upload_button = gr.File(label="Upload CSV", file_types=['.csv'], file_count="single")
+            table = gr.Dataframe(type="pandas", interactive=True)
+            cols = gr.Dropdown(choices=[], label='Dataframe columns')
+
+            upload_button.change(fn=process_csv_text, inputs=upload_button, outputs=[table, cols])
+
+        # Tab 2: ChromaDB, Embeddings, and Plotting
+        with gr.Tab("ChromaDB and Embeddings"):
+            cols = gr.Dropdown(choices=[], label='Dataframe columns')
+            embedding_model = gr.Dropdown(value='all-MiniLM-L6-v2',
+                                          choices=['all-MiniLM-L6-v2', 'intfloat/e5-small-v2', 'intfloat/e5-base-v2',
+                                                   'intfloat/e5-large-v2', 'paraphrase-multilingual-MiniLM-L12-v2'],
+                                          label='Embedding Model')
+            similarity_metric = gr.Dropdown(value='cosine', choices=['cosine', 'l2'], label='Similarity Metric')
+            embedding_button = gr.Button(value="Insert or Update Embeddings")
+            text = gr.Textbox(label='Process Status')
+
+            show_embeddings_button = gr.Button(value="Show Embeddings")
+            embeddings_plot = gr.Plot()
+
+            tsne = gr.State(value=None)  # Using gr.State for intermediate results (tsne)
+            test_string = gr.Textbox(label='Test String')
+
+            calculate_2d_repr_button = gr.Button(value="Calculate 2D Representation")
+            embeddings_plot_with_text_string = gr.Plot()
+
+            embedding_button.click(insert_or_update_chroma, inputs=[cols, table, embedding_model, similarity_metric], outputs=[text])
+            show_embeddings_button.click(show_fig, inputs=[], outputs=[embeddings_plot, tsne])
+            calculate_2d_repr_button.click(show_test_string_fig, inputs=[test_string, tsne, embedding_model, similarity_metric], outputs=[embeddings_plot_with_text_string, tsne])
+
+        # Tab 3: Chat with GPT
+        with gr.Tab("Chat"):
+            system_prompt = gr.Textbox(value="You are a helpful assistant.", label="System Message")
+            chatgpt_model = gr.Dropdown(value="gpt-4", choices=list(MODEL_CONFIG.keys()), label="ChatGPT Model")
+            temperature = gr.Slider(minimum=0, maximum=2, step=0.1, value=0.7, label="Temperature")
+            max_tokens = gr.Slider(minimum=50, maximum=2000, step=50, value=300, label="Max Tokens")
+            chatbot = gr.Chatbot(label="ChatGPT Chat")
+            clear_button = gr.Button("Clear Chat History")
+            msg = gr.Textbox()
+            msg_log = gr.Textbox("Message history", label="History")
+
+            # Replacing `.submit()` with `.change()` to trigger callback when user enters a message
+            msg.submit(fn=ask_gpt, inputs=[msg, chatbot, system_prompt, embedding_model, temperature, max_tokens, chatgpt_model], outputs=[msg, chatbot])
+            clear_button.click(fn=lambda: None, inputs=None, outputs=[chatbot])
+
+    return demo
+
+# Launch the Gradio interface
+demo = build_gradio_ui()
+demo.launch(server_name="0.0.0.0", server_port=8080, share=True)
+
+
+
+# Launch the Gradio interf

chatbot/app2.pynnn

@@ -0,0 +1,213 @@
+import os
+import gradio as gr
+import pandas as pd
+import numpy as np
+import chromadb
+from chromadb.config import Settings
+from io import StringIO
+from sentence_transformers import SentenceTransformer
+import openai
+import plotly.express as px
+from sklearn.manifold import TSNE
+
+# Initialize Chroma client with DuckDB and Parquet for persistence
+chroma_client = chromadb.Client(Settings(
+    chroma_db_impl="duckdb+parquet", 
+    persist_directory="./chroma_db"
+))
+
+# Model Configuration for Dynamic Dropdown
+model_config = {
+    "gpt-4": {
+        "endpoint": "https://roger-m38jr9pd-eastus2.openai.azure.com/openai/deployments/gpt-4/chat/completions?api-version=2024-08-01-preview",
+        "api_key": os.getenv("GPT4_API_KEY")
+    },
+    "gpt-4o": {
+        "endpoint": "https://roger-m38jr9pd-eastus2.openai.azure.com/openai/deployments/gpt-4o/chat/completions?api-version=2024-08-01-preview",
+        "api_key": os.getenv("GPT4O_API_KEY")
+    },
+    "gpt-35-turbo": {
+        "endpoint": "https://rogerkoranteng.openai.azure.com/openai/deployments/gpt-35-turbo/chat/completions?api-version=2024-08-01-preview",
+        "api_key": os.getenv("GPT35_TURBO_API_KEY")
+    },
+    "gpt-4-32k": {
+        "endpoint": "https://roger-m38orjxq-australiaeast.openai.azure.com/openai/deployments/gpt-4-32k/chat/completions?api-version=2024-08-01-preview",
+        "api_key": os.getenv("GPT4_32K_API_KEY")
+    }
+}
+
+# Function to process uploaded CSV
+def process_csv_text(temp_file):
+    if isinstance(temp_file, str):
+        df = pd.read_csv(StringIO(temp_file))
+    else:
+        df = pd.read_csv(temp_file.name, header='infer', sep=',')
+    return df, gr.Dropdown.update(choices=list(df.columns))
+
+# Insert or update ChromaDB with embeddings
+def insert_or_update_chroma(col, table, model_name, similarity_metric, client=chroma_client):
+    try:
+        collection = client.create_collection(name="my_collection", 
+                                              embedding_function=SentenceTransformer(model_name),
+                                              metadata={"hnsw:space": similarity_metric})
+    except Exception as e:
+        print("Collection exists, deleting it")
+        client.delete_collection(name='my_collection')
+        collection = client.create_collection(name="my_collection", 
+                                              embedding_function=SentenceTransformer(model_name),
+                                              metadata={"hnsw:space": similarity_metric})
+    if collection:
+        try:
+            collection.add(
+                documents=list(table[col]),
+                metadatas=[{"source": i} for i in range(len(table))],
+                ids=[str(i + 1) for i in range(len(table))]
+            )
+            return "Embedding calculations and insertions successful"
+        except Exception as e:
+            return "Error in embedding calculations"
+
+# Show plot with embeddings using t-SNE
+def show_fig():
+    collection = chroma_client.get_collection(name="my_collection")
+    embeddings = collection.get(include=['embeddings', 'documents'])
+
+    df = pd.DataFrame({
+        'text': embeddings['documents'],
+        'embedding': embeddings['embeddings']
+    })
+
+    embeddings_np = np.array(df['embedding'].tolist())
+    tsne = TSNE(n_components=2, random_state=42)
+    transformed = tsne.fit_transform(embeddings_np)
+
+    df['tsne_x'] = transformed[:, 0]
+    df['tsne_y'] = transformed[:, 1]
+
+    fig = px.scatter(df, x='tsne_x', y='tsne_y', hover_name='text')
+    return fig, transformed
+
+# Show test string figure
+def show_test_string_fig(test_string, tsne, model_name, similarity_metric):
+    collection = chroma_client.get_collection(name="my_collection", 
+                                              embedding_function=SentenceTransformer(model_name))
+    
+    collection.add(
+        documents=[test_string],
+        metadatas=[{"source": 'test'}],
+        ids=['test_sample']
+    )
+
+    embeddings = collection.get(include=['embeddings', 'documents'])
+
+    df = pd.DataFrame({
+        'text': embeddings['documents'],
+        'embedding': embeddings['embeddings'],
+        'set': ['orig' if document != test_string else 'test_string' for document in embeddings["documents"]]
+    })
+
+    embeddings_np = np.array(df['embedding'].tolist())
+    transformed = tsne.transform(embeddings_np)
+
+    df['tsne_x'] = transformed[:, 0]
+    df['tsne_y'] = transformed[:, 1]
+
+    fig = px.scatter(df, x='tsne_x', y='tsne_y', hover_name='text', color='set')
+    return fig, tsne
+
+# Function to interact with OpenAI's Azure API
+def ask_gpt(message, messages_history, embedding_model, system_prompt, temperature, max_tokens, chatgpt_model):
+    if len(messages_history) < 1:
+        messages_history = [{"role": "system", "content": system_prompt}]
+    model_info = model_config[chatgpt_model]
+    headers = {
+        "Content-Type": "application/json",
+        "api-key": model_info["api_key"]
+    }
+
+    message = retrieve_similar(message, embedding_model)
+
+    messages_history += [{"role": "user", "content": message}]
+    response = openai.ChatCompletion.create(
+        model=chatgpt_model,
+        messages=messages_history,
+        temperature=temperature,
+        max_tokens=max_tokens
+    )
+    
+    return response['choices'][0]['message']['content'], messages_history
+
+# Function to retrieve similar questions from ChromaDB
+def retrieve_similar(prompt, embedding_model, client=chroma_client):
+    collection = client.get_collection(name="my_collection", embedding_function=SentenceTransformer(model_name=embedding_model))
+
+    results = collection.query(query_texts=prompt, n_results=10)
+    additional_context = ''
+    for i, document in enumerate(results['documents'][0]):
+        if i == 0:
+            additional_context = 'Information: \n' + str(i+1) + '. ' + document
+        else:
+            additional_context += '\n' + str(i+1) + '. ' + document
+
+    prompt_with_context = additional_context + '\nQuestion: ' + prompt
+    return prompt_with_context
+
+# Gradio App Setup
+with gr.Blocks() as demo:
+    # Tab 1: Upload CSV and Display Data
+    with gr.Tab("Upload data"):
+        upload_button = gr.UploadButton(label="Upload csv", file_types=['.csv'], file_count="single")
+        table = gr.Dataframe(type="pandas", max_rows='20', overflow_row_behaviour='paginate', interactive=True)
+        cols = gr.Dropdown(choices=[], label='Dataframe columns')
+
+        upload_button.upload(fn=process_csv_text, inputs=upload_button, outputs=[table, cols], api_name="upload_csv")
+
+    # Tab 2: ChromaDB, Embeddings, and Plotting
+    with gr.Tab("Select Column and insert embeddings to ChromaDb"):
+        with gr.Row():
+            gr.Markdown("<br>")
+        with gr.Row():
+            cols = gr.Dropdown(choices=['text_column_1_placeholder'], label='Dataframe columns')
+
+        with gr.Row():
+            embedding_model = gr.Dropdown(value='all-MiniLM-L6-v2', choices=['all-MiniLM-L6-v2', 'intfloat/e5-small-v2', 'intfloat/e5-base-v2', 'intfloat/e5-large-v2','paraphrase-multilingual-MiniLM-L12-v2'], label='Embedding model to use')
+            similarity_metric = gr.Dropdown(value='cosine', choices=['cosine', 'l2'], label='Similarity metric to use')
+
+        with gr.Row():
+            embedding_button = gr.Button(value="Insert or update rows from selected column to embeddings db")
+            text = gr.Textbox(label='Process status for Chroma', placeholder='This will be updated once you click "Process status for Chroma"')
+
+        with gr.Row():
+            show_embeddings_button = gr.Button(value="Calculate 2d values from embeddings and show scatter plot")
+            embeddings_plot = gr.Plot()
+
+        with gr.Row():
+            tsne = gr.State(value=None)
+            test_string = gr.Textbox(label='test string to try to embed', value="Insert test string here")
+            
+        with gr.Row():
+            calculate_2d_repr_button = gr.Button(value="See where text string is in 2d")
+            embeddings_plot_with_text_string = gr.Plot()
+
+        embedding_button.click(insert_or_update_chroma, inputs=[cols, table, embedding_model, similarity_metric], outputs=[text])
+        show_embeddings_button.click(show_fig, inputs=[], outputs=[embeddings_plot, tsne])
+        calculate_2d_repr_button.click(show_test_string_fig, inputs=[test_string, tsne, embedding_model, similarity_metric], outputs=[embeddings_plot_with_text_string, tsne])
+
+    # Tab 3: Chat with GPT Models
+    with gr.Tab("Chat"):
+        system_prompt = gr.Textbox(value="You are a helpful assistant.", label="System Message")
+        chatgpt_model = gr.Dropdown(value="gpt-4", choices=list(model_config.keys()), label="ChatGPT Model to Use")
+        temperature = gr.Slider(minimum=0, maximum=2, step=0.1, value=0.7, label="Temperature")
+        max_tokens = gr.Slider(minimum=50, maximum=2000, step=50, value=300, label="Max Tokens")
+        chatbot = gr.Chatbot(label="ChatGPT Chat")
+        clear_button = gr.Button("Clear Chat History")
+
+        msg = gr.Textbox()
+        msg_log = gr.Textbox("Message history will be visible here", label='Message history')
+
+        msg.submit(ask_gpt, [msg, chatbot], [msg, chatbot])
+        chatbot.submit(ask_gpt, [chatbot, system_prompt, embedding_model, temperature, max_tokens, chatgpt_model], [chatbot, system_prompt])
+        clear_button.click(fn=lambda: None, inputs=None, outputs=[chatbot])
+
+# Launch Gradio interface
+demo.launch()

chatbot/chat_history.json

@@ -0,0 +1,62 @@
+[
+    {
+        "role": "assistant",
+        "content": "Hello, I am Sage. How can I assist you today?"
+    },
+    {
+        "role": "user",
+        "content": "Hi Sage, my boyfriend broke up with me. I'm feeling so sad. What should I do?"
+    },
+    {
+        "role": "assistant",
+        "content": "I'm sorry to hear that you're going through a tough time. Breakups can be incredibly painful, but there are ways to help you cope with the emotions and eventually move forward. Here are a few suggestions:\n\n1. **Allow Yourself to Grieve**: It's okay to feel sad, angry, or confused. Allow yourself time to process these emotions. It's an important part of healing.\n\n2. **Reach Out for Support**: Talk to friends and family who care about you. Sharing your feelings can be very therapeutic and remind you that you're not alone.\n\n3. **Stay Active**: Engaging in physical activities can help boost your mood and take your mind off the pain. Consider trying out exercises like walking, yoga, or anything you enjoy.\n\n4. **Pursue Interests**: Reconnect with hobbies and interests that you may have neglected. This can help you find joy and fulfillment in other areas of your life.\n\n5. **Avoid Contact**: It might be helpful to take a break from seeing or communicating with your ex while you heal. Constant contact can often prolong the pain.\n\n6. **Seek Professional Help**: If you find it very difficult to cope, speaking with a therapist or counselor can provide professional support and strategies to manage your feelings.\n\nRemember, it's okay to take things one day at a time. Healing is a process, and it's different for everyone."
+    },
+    {
+        "role": "user",
+        "content": "Hello, Sage. I feel like nobody likes me. What should I do?"
+    },
+    {
+        "role": "assistant",
+        "content": "Feeling like nobody likes you can be really tough, but remember, these feelings don't define your worth or your ability to be liked. Here are some steps you can take to address these feelings:\n\n1. **Reflect on Your Feelings**: Sometimes, these feelings might not reflect reality but rather our own insecurities or past experiences. Try to assess if there's any specific reason you feel this way.\n\n2. **Communicate**: Talk to people you trust about how you feel. They might provide reassurance and help you see that people do care about you.\n\n3. **Build Self-Esteem**: Engage in activities that make you feel good about yourself. This could be hobbies, sports, learning new skills, or volunteering. Accomplishments can boost your self-esteem and change how you perceive yourself.\n\n4. **Social Skills**: Sometimes, improving social skills can boost your confidence in interactions. This might include practicing small talk, being a good listener, or showing interest in others' activities.\n\n5. **Meet New People**: Sometimes the feeling of not being liked can stem from not having found the right group of friends. Try joining clubs, groups, or online communities where you can meet people with similar interests.\n\n6. **Consider Professional Help**: If these feelings are overwhelming and persistent, talking to a therapist or counselor can be very beneficial. They can help you work through these feelings and develop strategies to improve your social interactions.\n\nRemember, many people feel this way at some point in their lives. It's brave to confront these feelings and seek ways to improve your situation. You're not alone in this."
+    },
+    {
+        "role": "user",
+        "content": "Hello, I am feeling sad today, what should I do?"
+    },
+    {
+        "role": "assistant",
+        "content": "I'm sorry to hear that you're feeling sad. It's important to take care of yourself and address your emotions. Here are a few suggestions that might help improve your mood:\n\n1. **Acknowledge Your Feelings**: It's okay to feel sad. Acknowledging your emotions is the first step toward feeling better. Allow yourself some time to just feel what you're feeling without judgment.\n\n2. **Engage in a Favorite Activity**: Doing something you enjoy, such as reading a book, watching a favorite movie, or engaging in a hobby, can provide a comforting distraction and bring some joy.\n\n3. **Connect with Others**: Reach out to friends or family members who can offer support. Sometimes just talking about how you feel can lighten the burden.\n\n4. **Get Some Exercise**: Physical activity can boost endorphins, which are chemicals in the brain that can lighten your mood. Even a short walk can make a difference.\n\n5. **Listen to Music**: Music can have a powerful effect on mood. Choose songs that lift your spirits or calm and soothe you, depending on what you feel you need.\n\n6. **Practice Mindfulness or Meditation**: These practices can help you stay present and reduce negative thoughts. Even a few minutes can have a significant impact.\n\n7. **Consider Professional Help**: If sadness persists and starts to affect your daily life, it might be helpful to talk to a mental health professional who can provide guidance and support.\n\nRemember, it's perfectly okay to have days when you feel down. Taking steps to care for yourself during these times is important."
+    },
+    {
+        "role": "user",
+        "content": "okay thanks how do i contact mental health persons"
+    },
+    {
+        "role": "assistant",
+        "content": "Seeking help from a mental health professional is a positive step towards taking care of your well-being. Here\u2019s how you can get started:\n\n1. **Primary Care Doctor**: You can start by talking to your primary care doctor. They can assess your situation and may provide a referral to a mental health specialist such as a psychologist or psychiatrist.\n\n2. **Insurance Provider**: If you have health insurance, check your provider\u2019s website or call their customer service for a list of covered mental health professionals. They can guide you on how to proceed based on your plan.\n\n3. **Online Directories**: Use reputable online directories or websites that list mental health professionals. Websites like Psychology Today, GoodTherapy, or even Google can help you find therapists in your area.\n\n4. **Local Mental Health Clinics**: Look for community mental health clinics in your area. These often offer services on a sliding scale fee based on income, making them accessible to more people.\n\n5. **Telehealth Services**: Consider online therapy options, which can be more convenient and sometimes more affordable. Platforms like BetterHelp, Talkspace, or Amwell offer various online therapy services.\n\n6. **Employee Assistance Programs (EAP)**: If you are employed, check if your employer offers an Employee Assistance Program, which often includes a few free therapy sessions.\n\n7. **School Counselors**: If you are a student, your school or university likely has counselors available to help students at no extra cost.\n\nRemember, seeking help is a sign of strength, and it\u2019s important to find a professional you feel comfortable with. Don\u2019t hesitate to reach out and take that first step."
+    },
+    {
+        "role": "user",
+        "content": "Hello, I am feeling sad today, what should I do?"
+    },
+    {
+        "role": "assistant",
+        "content": "I'm sorry to hear that you're feeling sad. It's important to take care of yourself and address your emotions. Here are a few suggestions that might help improve your mood:\n\n1. **Acknowledge Your Feelings**: It's okay to feel sad. Acknowledging your emotions is the first step toward feeling better. Allow yourself some time to just feel what you're feeling without judgment.\n\n2. **Engage in a Favorite Activity**: Doing something you enjoy, such as reading a book, watching a favorite movie, or engaging in a hobby, can provide a comforting distraction and bring some joy.\n\n3. **Connect with Others**: Reach out to friends or family members who can offer support. Sometimes just talking about how you feel can lighten the burden.\n\n4. **Get Some Exercise**: Physical activity can boost endorphins, which are chemicals in the brain that can lighten your mood. Even a short walk can make a difference.\n\n5. **Listen to Music**: Music can have a powerful effect on mood. Choose songs that lift your spirits or calm and soothe you, depending on what you feel you need.\n\n6. **Practice Mindfulness or Meditation**: These practices can help you stay present and reduce negative thoughts. Even a few minutes can have a significant impact.\n\n7. **Consider Professional Help**: If sadness persists and starts to affect your daily life, it might be helpful to talk to a mental health professional who can provide guidance and support.\n\nRemember, it's perfectly okay to have days when you feel down. Taking steps to care for yourself during these times is important."
+    },
+    {
+        "role": "user",
+        "content": "I'M SORRY TO HEAR THAT YOUR FEELING SAD ITS IMPORTANT TO TAKE CARE OF YOURSELF AND ADDRESS YOUREMOTIONS ERE ARE A FU SUGGESTIONS THAT MIGHT HELP IMPROVE YOUR MOOD ONE ACKNOWLEDGE YOUR FEELINS ITS OCADE A FEEL SAD ACKNOWLEDGING YOUR EMOTIONS AS THE FIRST STEP TOWARD FEELING BETTER ALLOW YOUR SELF SOME TIME TO JUST FEEL WHAT YOUR FEELING WITHOUT JUDGMAN TWO ENGAGIN A FAVORITE ACTIVITY DOING SOMETHING YOU IN JOY SUCH AS READING A BOOK WATCHING A FVORITE MOVY OR ENGAGING IN A HOBB CAN PROVIDE A COMFORTING DISTRACTION AND BRING SOME JOY THREE CONNECT WITH OTHERS REACH OUT TO FRIENDS OUR FAMLY MEMBERS WHO CAN OFFER SUPPORT SOMETIMES JUST TALTING ABOUT HOW YOU FEEL CAN LIGHTEN THE BURDN FOUR GET SOME EXERCISE FISICAL ACTIVITY CAN BOOSED AN DORPHANS WICH OURE CHEMICULS IN THE BRAIN THAT CAN LIGHTEN YOUR MOOD EVEN A SHORT WALK CAN MAKE A DIFFERENE FIVE LISEN TO MUSIC MUSIC CAN HAVE A POWERFUL AFFECT ON MOOD CHOOSE SONGS THAT LIFT  YOUR  PIRITS ARE COMMAN SOOTHYOU DEPENDING ON WHAT YOU FEEL YOU NEED SIX PRACTICE MINDFULNESS OR MEDITATION VUSE PRACTICES CAN HELP YOU STAKE PRESON AND REDUCE NEGATIVE THOUGHTS EVEN A FEW MINUTES CAN HAVE US O NIFICANT IMPACT SEVEN CAN SIDER PROFESSIONAL HELP IT SADNESS PERSISTS AND STARTS TO EFFECT YOUR DAILY LIFE IT MIGHT BE HELPFUL TO TALK TO A MENTAL HELTH PROFESSIONAL WHO CAN PROVIDE GUIDANCE AND SUPORT REMEMBER ITS PERFECALY OCADE TO HAVE DAYS WHEN YOU FEEL DOWN TACING STEPS TO CARE FOR YOURSELF DURING THESE TIMES AS IMPORDANT"
+    },
+    {
+        "role": "assistant",
+        "content": "Thank you for your concern, but I am an AI language model and do not have feelings. My purpose is to provide assistance and support to those who need it. If you have any other questions or concerns, please feel free to ask."
+    },
+    {
+        "role": "user",
+        "content": "what AI model are you GPT or what"
+    },
+    {
+        "role": "assistant",
+        "content": "Yes, I am a GPT (Generative Pre-trained Transformer) AI language model developed by OpenAI. I am designed to understand natural language and generate human-like responses to various queries and prompts. My purpose is to assist and provide helpful responses to users like you."
+    }
+]

chatbot/fine_tuned_model/config.json

@@ -0,0 +1,2026 @@
+{
+  "_name_or_path": "google/vit-base-patch16-224",
+  "architectures": [
+    "ViTForImageClassification"
+  ],
+  "attention_probs_dropout_prob": 0.0,
+  "encoder_stride": 16,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.0,
+  "hidden_size": 768,
+  "id2label": {
+    "0": "tench, Tinca tinca",
+    "1": "goldfish, Carassius auratus",
+    "2": "great white shark, white shark, man-eater, man-eating shark, Carcharodon carcharias",
+    "3": "tiger shark, Galeocerdo cuvieri",
+    "4": "hammerhead, hammerhead shark",
+    "5": "electric ray, crampfish, numbfish, torpedo",
+    "6": "stingray",
+    "7": "cock",
+    "8": "hen",
+    "9": "ostrich, Struthio camelus",
+    "10": "brambling, Fringilla montifringilla",
+    "11": "goldfinch, Carduelis carduelis",
+    "12": "house finch, linnet, Carpodacus mexicanus",
+    "13": "junco, snowbird",
+    "14": "indigo bunting, indigo finch, indigo bird, Passerina cyanea",
+    "15": "robin, American robin, Turdus migratorius",
+    "16": "bulbul",
+    "17": "jay",
+    "18": "magpie",
+    "19": "chickadee",
+    "20": "water ouzel, dipper",
+    "21": "kite",
+    "22": "bald eagle, American eagle, Haliaeetus leucocephalus",
+    "23": "vulture",
+    "24": "great grey owl, great gray owl, Strix nebulosa",
+    "25": "European fire salamander, Salamandra salamandra",
+    "26": "common newt, Triturus vulgaris",
+    "27": "eft",
+    "28": "spotted salamander, Ambystoma maculatum",
+    "29": "axolotl, mud puppy, Ambystoma mexicanum",
+    "30": "bullfrog, Rana catesbeiana",
+    "31": "tree frog, tree-frog",
+    "32": "tailed frog, bell toad, ribbed toad, tailed toad, Ascaphus trui",
+    "33": "loggerhead, loggerhead turtle, Caretta caretta",
+    "34": "leatherback turtle, leatherback, leathery turtle, Dermochelys coriacea",
+    "35": "mud turtle",
+    "36": "terrapin",
+    "37": "box turtle, box tortoise",
+    "38": "banded gecko",
+    "39": "common iguana, iguana, Iguana iguana",
+    "40": "American chameleon, anole, Anolis carolinensis",
+    "41": "whiptail, whiptail lizard",
+    "42": "agama",
+    "43": "frilled lizard, Chlamydosaurus kingi",
+    "44": "alligator lizard",
+    "45": "Gila monster, Heloderma suspectum",
+    "46": "green lizard, Lacerta viridis",
+    "47": "African chameleon, Chamaeleo chamaeleon",
+    "48": "Komodo dragon, Komodo lizard, dragon lizard, giant lizard, Varanus komodoensis",
+    "49": "African crocodile, Nile crocodile, Crocodylus niloticus",
+    "50": "American alligator, Alligator mississipiensis",
+    "51": "triceratops",
+    "52": "thunder snake, worm snake, Carphophis amoenus",
+    "53": "ringneck snake, ring-necked snake, ring snake",
+    "54": "hognose snake, puff adder, sand viper",
+    "55": "green snake, grass snake",
+    "56": "king snake, kingsnake",
+    "57": "garter snake, grass snake",
+    "58": "water snake",
+    "59": "vine snake",
+    "60": "night snake, Hypsiglena torquata",
+    "61": "boa constrictor, Constrictor constrictor",
+    "62": "rock python, rock snake, Python sebae",
+    "63": "Indian cobra, Naja naja",
+    "64": "green mamba",
+    "65": "sea snake",
+    "66": "horned viper, cerastes, sand viper, horned asp, Cerastes cornutus",
+    "67": "diamondback, diamondback rattlesnake, Crotalus adamanteus",
+    "68": "sidewinder, horned rattlesnake, Crotalus cerastes",
+    "69": "trilobite",
+    "70": "harvestman, daddy longlegs, Phalangium opilio",
+    "71": "scorpion",
+    "72": "black and gold garden spider, Argiope aurantia",
+    "73": "barn spider, Araneus cavaticus",
+    "74": "garden spider, Aranea diademata",
+    "75": "black widow, Latrodectus mactans",
+    "76": "tarantula",
+    "77": "wolf spider, hunting spider",
+    "78": "tick",
+    "79": "centipede",
+    "80": "black grouse",
+    "81": "ptarmigan",
+    "82": "ruffed grouse, partridge, Bonasa umbellus",
+    "83": "prairie chicken, prairie grouse, prairie fowl",
+    "84": "peacock",
+    "85": "quail",
+    "86": "partridge",
+    "87": "African grey, African gray, Psittacus erithacus",
+    "88": "macaw",
+    "89": "sulphur-crested cockatoo, Kakatoe galerita, Cacatua galerita",
+    "90": "lorikeet",
+    "91": "coucal",
+    "92": "bee eater",
+    "93": "hornbill",
+    "94": "hummingbird",
+    "95": "jacamar",
+    "96": "toucan",
+    "97": "drake",
+    "98": "red-breasted merganser, Mergus serrator",
+    "99": "goose",
+    "100": "black swan, Cygnus atratus",
+    "101": "tusker",
+    "102": "echidna, spiny anteater, anteater",
+    "103": "platypus, duckbill, duckbilled platypus, duck-billed platypus, Ornithorhynchus anatinus",
+    "104": "wallaby, brush kangaroo",
+    "105": "koala, koala bear, kangaroo bear, native bear, Phascolarctos cinereus",
+    "106": "wombat",
+    "107": "jellyfish",
+    "108": "sea anemone, anemone",
+    "109": "brain coral",
+    "110": "flatworm, platyhelminth",
+    "111": "nematode, nematode worm, roundworm",
+    "112": "conch",
+    "113": "snail",
+    "114": "slug",
+    "115": "sea slug, nudibranch",
+    "116": "chiton, coat-of-mail shell, sea cradle, polyplacophore",
+    "117": "chambered nautilus, pearly nautilus, nautilus",
+    "118": "Dungeness crab, Cancer magister",
+    "119": "rock crab, Cancer irroratus",
+    "120": "fiddler crab",
+    "121": "king crab, Alaska crab, Alaskan king crab, Alaska king crab, Paralithodes camtschatica",
+    "122": "American lobster, Northern lobster, Maine lobster, Homarus americanus",
+    "123": "spiny lobster, langouste, rock lobster, crawfish, crayfish, sea crawfish",
+    "124": "crayfish, crawfish, crawdad, crawdaddy",
+    "125": "hermit crab",
+    "126": "isopod",
+    "127": "white stork, Ciconia ciconia",
+    "128": "black stork, Ciconia nigra",
+    "129": "spoonbill",
+    "130": "flamingo",
+    "131": "little blue heron, Egretta caerulea",
+    "132": "American egret, great white heron, Egretta albus",
+    "133": "bittern",
+    "134": "crane",
+    "135": "limpkin, Aramus pictus",
+    "136": "European gallinule, Porphyrio porphyrio",
+    "137": "American coot, marsh hen, mud hen, water hen, Fulica americana",
+    "138": "bustard",
+    "139": "ruddy turnstone, Arenaria interpres",
+    "140": "red-backed sandpiper, dunlin, Erolia alpina",
+    "141": "redshank, Tringa totanus",
+    "142": "dowitcher",
+    "143": "oystercatcher, oyster catcher",
+    "144": "pelican",
+    "145": "king penguin, Aptenodytes patagonica",
+    "146": "albatross, mollymawk",
+    "147": "grey whale, gray whale, devilfish, Eschrichtius gibbosus, Eschrichtius robustus",
+    "148": "killer whale, killer, orca, grampus, sea wolf, Orcinus orca",
+    "149": "dugong, Dugong dugon",
+    "150": "sea lion",
+    "151": "Chihuahua",
+    "152": "Japanese spaniel",
+    "153": "Maltese dog, Maltese terrier, Maltese",
+    "154": "Pekinese, Pekingese, Peke",
+    "155": "Shih-Tzu",
+    "156": "Blenheim spaniel",
+    "157": "papillon",
+    "158": "toy terrier",
+    "159": "Rhodesian ridgeback",
+    "160": "Afghan hound, Afghan",
+    "161": "basset, basset hound",
+    "162": "beagle",
+    "163": "bloodhound, sleuthhound",
+    "164": "bluetick",
+    "165": "black-and-tan coonhound",
+    "166": "Walker hound, Walker foxhound",
+    "167": "English foxhound",
+    "168": "redbone",
+    "169": "borzoi, Russian wolfhound",
+    "170": "Irish wolfhound",
+    "171": "Italian greyhound",
+    "172": "whippet",
+    "173": "Ibizan hound, Ibizan Podenco",
+    "174": "Norwegian elkhound, elkhound",
+    "175": "otterhound, otter hound",
+    "176": "Saluki, gazelle hound",
+    "177": "Scottish deerhound, deerhound",
+    "178": "Weimaraner",
+    "179": "Staffordshire bullterrier, Staffordshire bull terrier",
+    "180": "American Staffordshire terrier, Staffordshire terrier, American pit bull terrier, pit bull terrier",
+    "181": "Bedlington terrier",
+    "182": "Border terrier",
+    "183": "Kerry blue terrier",
+    "184": "Irish terrier",
+    "185": "Norfolk terrier",
+    "186": "Norwich terrier",
+    "187": "Yorkshire terrier",
+    "188": "wire-haired fox terrier",
+    "189": "Lakeland terrier",
+    "190": "Sealyham terrier, Sealyham",
+    "191": "Airedale, Airedale terrier",
+    "192": "cairn, cairn terrier",
+    "193": "Australian terrier",
+    "194": "Dandie Dinmont, Dandie Dinmont terrier",
+    "195": "Boston bull, Boston terrier",
+    "196": "miniature schnauzer",
+    "197": "giant schnauzer",
+    "198": "standard schnauzer",
+    "199": "Scotch terrier, Scottish terrier, Scottie",
+    "200": "Tibetan terrier, chrysanthemum dog",
+    "201": "silky terrier, Sydney silky",
+    "202": "soft-coated wheaten terrier",
+    "203": "West Highland white terrier",
+    "204": "Lhasa, Lhasa apso",
+    "205": "flat-coated retriever",
+    "206": "curly-coated retriever",
+    "207": "golden retriever",
+    "208": "Labrador retriever",
+    "209": "Chesapeake Bay retriever",
+    "210": "German short-haired pointer",
+    "211": "vizsla, Hungarian pointer",
+    "212": "English setter",
+    "213": "Irish setter, red setter",
+    "214": "Gordon setter",
+    "215": "Brittany spaniel",
+    "216": "clumber, clumber spaniel",
+    "217": "English springer, English springer spaniel",
+    "218": "Welsh springer spaniel",
+    "219": "cocker spaniel, English cocker spaniel, cocker",
+    "220": "Sussex spaniel",
+    "221": "Irish water spaniel",
+    "222": "kuvasz",
+    "223": "schipperke",
+    "224": "groenendael",
+    "225": "malinois",
+    "226": "briard",
+    "227": "kelpie",
+    "228": "komondor",
+    "229": "Old English sheepdog, bobtail",
+    "230": "Shetland sheepdog, Shetland sheep dog, Shetland",
+    "231": "collie",
+    "232": "Border collie",
+    "233": "Bouvier des Flandres, Bouviers des Flandres",
+    "234": "Rottweiler",
+    "235": "German shepherd, German shepherd dog, German police dog, alsatian",
+    "236": "Doberman, Doberman pinscher",
+    "237": "miniature pinscher",
+    "238": "Greater Swiss Mountain dog",
+    "239": "Bernese mountain dog",
+    "240": "Appenzeller",
+    "241": "EntleBucher",
+    "242": "boxer",
+    "243": "bull mastiff",
+    "244": "Tibetan mastiff",
+    "245": "French bulldog",
+    "246": "Great Dane",
+    "247": "Saint Bernard, St Bernard",
+    "248": "Eskimo dog, husky",
+    "249": "malamute, malemute, Alaskan malamute",
+    "250": "Siberian husky",
+    "251": "dalmatian, coach dog, carriage dog",
+    "252": "affenpinscher, monkey pinscher, monkey dog",
+    "253": "basenji",
+    "254": "pug, pug-dog",
+    "255": "Leonberg",
+    "256": "Newfoundland, Newfoundland dog",
+    "257": "Great Pyrenees",
+    "258": "Samoyed, Samoyede",
+    "259": "Pomeranian",
+    "260": "chow, chow chow",
+    "261": "keeshond",
+    "262": "Brabancon griffon",
+    "263": "Pembroke, Pembroke Welsh corgi",
+    "264": "Cardigan, Cardigan Welsh corgi",
+    "265": "toy poodle",
+    "266": "miniature poodle",
+    "267": "standard poodle",
+    "268": "Mexican hairless",
+    "269": "timber wolf, grey wolf, gray wolf, Canis lupus",
+    "270": "white wolf, Arctic wolf, Canis lupus tundrarum",
+    "271": "red wolf, maned wolf, Canis rufus, Canis niger",
+    "272": "coyote, prairie wolf, brush wolf, Canis latrans",
+    "273": "dingo, warrigal, warragal, Canis dingo",
+    "274": "dhole, Cuon alpinus",
+    "275": "African hunting dog, hyena dog, Cape hunting dog, Lycaon pictus",
+    "276": "hyena, hyaena",
+    "277": "red fox, Vulpes vulpes",
+    "278": "kit fox, Vulpes macrotis",
+    "279": "Arctic fox, white fox, Alopex lagopus",
+    "280": "grey fox, gray fox, Urocyon cinereoargenteus",
+    "281": "tabby, tabby cat",
+    "282": "tiger cat",
+    "283": "Persian cat",
+    "284": "Siamese cat, Siamese",
+    "285": "Egyptian cat",
+    "286": "cougar, puma, catamount, mountain lion, painter, panther, Felis concolor",
+    "287": "lynx, catamount",
+    "288": "leopard, Panthera pardus",
+    "289": "snow leopard, ounce, Panthera uncia",
+    "290": "jaguar, panther, Panthera onca, Felis onca",
+    "291": "lion, king of beasts, Panthera leo",
+    "292": "tiger, Panthera tigris",
+    "293": "cheetah, chetah, Acinonyx jubatus",
+    "294": "brown bear, bruin, Ursus arctos",
+    "295": "American black bear, black bear, Ursus americanus, Euarctos americanus",
+    "296": "ice bear, polar bear, Ursus Maritimus, Thalarctos maritimus",
+    "297": "sloth bear, Melursus ursinus, Ursus ursinus",
+    "298": "mongoose",
+    "299": "meerkat, mierkat",
+    "300": "tiger beetle",
+    "301": "ladybug, ladybeetle, lady beetle, ladybird, ladybird beetle",
+    "302": "ground beetle, carabid beetle",
+    "303": "long-horned beetle, longicorn, longicorn beetle",
+    "304": "leaf beetle, chrysomelid",
+    "305": "dung beetle",
+    "306": "rhinoceros beetle",
+    "307": "weevil",
+    "308": "fly",
+    "309": "bee",
+    "310": "ant, emmet, pismire",
+    "311": "grasshopper, hopper",
+    "312": "cricket",
+    "313": "walking stick, walkingstick, stick insect",
+    "314": "cockroach, roach",
+    "315": "mantis, mantid",
+    "316": "cicada, cicala",
+    "317": "leafhopper",
+    "318": "lacewing, lacewing fly",
+    "319": "dragonfly, darning needle, devil's darning needle, sewing needle, snake feeder, snake doctor, mosquito hawk, skeeter hawk",
+    "320": "damselfly",
+    "321": "admiral",
+    "322": "ringlet, ringlet butterfly",
+    "323": "monarch, monarch butterfly, milkweed butterfly, Danaus plexippus",
+    "324": "cabbage butterfly",
+    "325": "sulphur butterfly, sulfur butterfly",
+    "326": "lycaenid, lycaenid butterfly",
+    "327": "starfish, sea star",
+    "328": "sea urchin",
+    "329": "sea cucumber, holothurian",
+    "330": "wood rabbit, cottontail, cottontail rabbit",
+    "331": "hare",
+    "332": "Angora, Angora rabbit",
+    "333": "hamster",
+    "334": "porcupine, hedgehog",
+    "335": "fox squirrel, eastern fox squirrel, Sciurus niger",
+    "336": "marmot",
+    "337": "beaver",
+    "338": "guinea pig, Cavia cobaya",
+    "339": "sorrel",
+    "340": "zebra",
+    "341": "hog, pig, grunter, squealer, Sus scrofa",
+    "342": "wild boar, boar, Sus scrofa",
+    "343": "warthog",
+    "344": "hippopotamus, hippo, river horse, Hippopotamus amphibius",
+    "345": "ox",
+    "346": "water buffalo, water ox, Asiatic buffalo, Bubalus bubalis",
+    "347": "bison",
+    "348": "ram, tup",
+    "349": "bighorn, bighorn sheep, cimarron, Rocky Mountain bighorn, Rocky Mountain sheep, Ovis canadensis",
+    "350": "ibex, Capra ibex",
+    "351": "hartebeest",
+    "352": "impala, Aepyceros melampus",
+    "353": "gazelle",
+    "354": "Arabian camel, dromedary, Camelus dromedarius",
+    "355": "llama",
+    "356": "weasel",
+    "357": "mink",
+    "358": "polecat, fitch, foulmart, foumart, Mustela putorius",
+    "359": "black-footed ferret, ferret, Mustela nigripes",
+    "360": "otter",
+    "361": "skunk, polecat, wood pussy",
+    "362": "badger",
+    "363": "armadillo",
+    "364": "three-toed sloth, ai, Bradypus tridactylus",
+    "365": "orangutan, orang, orangutang, Pongo pygmaeus",
+    "366": "gorilla, Gorilla gorilla",
+    "367": "chimpanzee, chimp, Pan troglodytes",
+    "368": "gibbon, Hylobates lar",
+    "369": "siamang, Hylobates syndactylus, Symphalangus syndactylus",
+    "370": "guenon, guenon monkey",
+    "371": "patas, hussar monkey, Erythrocebus patas",
+    "372": "baboon",
+    "373": "macaque",
+    "374": "langur",
+    "375": "colobus, colobus monkey",
+    "376": "proboscis monkey, Nasalis larvatus",
+    "377": "marmoset",
+    "378": "capuchin, ringtail, Cebus capucinus",
+    "379": "howler monkey, howler",
+    "380": "titi, titi monkey",
+    "381": "spider monkey, Ateles geoffroyi",
+    "382": "squirrel monkey, Saimiri sciureus",
+    "383": "Madagascar cat, ring-tailed lemur, Lemur catta",
+    "384": "indri, indris, Indri indri, Indri brevicaudatus",
+    "385": "Indian elephant, Elephas maximus",
+    "386": "African elephant, Loxodonta africana",
+    "387": "lesser panda, red panda, panda, bear cat, cat bear, Ailurus fulgens",
+    "388": "giant panda, panda, panda bear, coon bear, Ailuropoda melanoleuca",
+    "389": "barracouta, snoek",
+    "390": "eel",
+    "391": "coho, cohoe, coho salmon, blue jack, silver salmon, Oncorhynchus kisutch",
+    "392": "rock beauty, Holocanthus tricolor",
+    "393": "anemone fish",
+    "394": "sturgeon",
+    "395": "gar, garfish, garpike, billfish, Lepisosteus osseus",
+    "396": "lionfish",
+    "397": "puffer, pufferfish, blowfish, globefish",
+    "398": "abacus",
+    "399": "abaya",
+    "400": "academic gown, academic robe, judge's robe",
+    "401": "accordion, piano accordion, squeeze box",
+    "402": "acoustic guitar",
+    "403": "aircraft carrier, carrier, flattop, attack aircraft carrier",
+    "404": "airliner",
+    "405": "airship, dirigible",
+    "406": "altar",
+    "407": "ambulance",
+    "408": "amphibian, amphibious vehicle",
+    "409": "analog clock",
+    "410": "apiary, bee house",
+    "411": "apron",
+    "412": "ashcan, trash can, garbage can, wastebin, ash bin, ash-bin, ashbin, dustbin, trash barrel, trash bin",
+    "413": "assault rifle, assault gun",
+    "414": "backpack, back pack, knapsack, packsack, rucksack, haversack",
+    "415": "bakery, bakeshop, bakehouse",
+    "416": "balance beam, beam",
+    "417": "balloon",
+    "418": "ballpoint, ballpoint pen, ballpen, Biro",
+    "419": "Band Aid",
+    "420": "banjo",
+    "421": "bannister, banister, balustrade, balusters, handrail",
+    "422": "barbell",
+    "423": "barber chair",
+    "424": "barbershop",
+    "425": "barn",
+    "426": "barometer",
+    "427": "barrel, cask",
+    "428": "barrow, garden cart, lawn cart, wheelbarrow",
+    "429": "baseball",
+    "430": "basketball",
+    "431": "bassinet",
+    "432": "bassoon",
+    "433": "bathing cap, swimming cap",
+    "434": "bath towel",
+    "435": "bathtub, bathing tub, bath, tub",
+    "436": "beach wagon, station wagon, wagon, estate car, beach waggon, station waggon, waggon",
+    "437": "beacon, lighthouse, beacon light, pharos",
+    "438": "beaker",
+    "439": "bearskin, busby, shako",
+    "440": "beer bottle",
+    "441": "beer glass",
+    "442": "bell cote, bell cot",
+    "443": "bib",
+    "444": "bicycle-built-for-two, tandem bicycle, tandem",
+    "445": "bikini, two-piece",
+    "446": "binder, ring-binder",
+    "447": "binoculars, field glasses, opera glasses",
+    "448": "birdhouse",
+    "449": "boathouse",
+    "450": "bobsled, bobsleigh, bob",
+    "451": "bolo tie, bolo, bola tie, bola",
+    "452": "bonnet, poke bonnet",
+    "453": "bookcase",
+    "454": "bookshop, bookstore, bookstall",
+    "455": "bottlecap",
+    "456": "bow",
+    "457": "bow tie, bow-tie, bowtie",
+    "458": "brass, memorial tablet, plaque",
+    "459": "brassiere, bra, bandeau",
+    "460": "breakwater, groin, groyne, mole, bulwark, seawall, jetty",
+    "461": "breastplate, aegis, egis",
+    "462": "broom",
+    "463": "bucket, pail",
+    "464": "buckle",
+    "465": "bulletproof vest",
+    "466": "bullet train, bullet",
+    "467": "butcher shop, meat market",
+    "468": "cab, hack, taxi, taxicab",
+    "469": "caldron, cauldron",
+    "470": "candle, taper, wax light",
+    "471": "cannon",
+    "472": "canoe",
+    "473": "can opener, tin opener",
+    "474": "cardigan",
+    "475": "car mirror",
+    "476": "carousel, carrousel, merry-go-round, roundabout, whirligig",
+    "477": "carpenter's kit, tool kit",
+    "478": "carton",
+    "479": "car wheel",
+    "480": "cash machine, cash dispenser, automated teller machine, automatic teller machine, automated teller, automatic teller, ATM",
+    "481": "cassette",
+    "482": "cassette player",
+    "483": "castle",
+    "484": "catamaran",
+    "485": "CD player",
+    "486": "cello, violoncello",
+    "487": "cellular telephone, cellular phone, cellphone, cell, mobile phone",
+    "488": "chain",
+    "489": "chainlink fence",
+    "490": "chain mail, ring mail, mail, chain armor, chain armour, ring armor, ring armour",
+    "491": "chain saw, chainsaw",
+    "492": "chest",
+    "493": "chiffonier, commode",
+    "494": "chime, bell, gong",
+    "495": "china cabinet, china closet",
+    "496": "Christmas stocking",
+    "497": "church, church building",
+    "498": "cinema, movie theater, movie theatre, movie house, picture palace",
+    "499": "cleaver, meat cleaver, chopper",
+    "500": "cliff dwelling",
+    "501": "cloak",
+    "502": "clog, geta, patten, sabot",
+    "503": "cocktail shaker",
+    "504": "coffee mug",
+    "505": "coffeepot",
+    "506": "coil, spiral, volute, whorl, helix",
+    "507": "combination lock",
+    "508": "computer keyboard, keypad",
+    "509": "confectionery, confectionary, candy store",
+    "510": "container ship, containership, container vessel",
+    "511": "convertible",
+    "512": "corkscrew, bottle screw",
+    "513": "cornet, horn, trumpet, trump",
+    "514": "cowboy boot",
+    "515": "cowboy hat, ten-gallon hat",
+    "516": "cradle",
+    "517": "crane",
+    "518": "crash helmet",
+    "519": "crate",
+    "520": "crib, cot",
+    "521": "Crock Pot",
+    "522": "croquet ball",
+    "523": "crutch",
+    "524": "cuirass",
+    "525": "dam, dike, dyke",
+    "526": "desk",
+    "527": "desktop computer",
+    "528": "dial telephone, dial phone",
+    "529": "diaper, nappy, napkin",
+    "530": "digital clock",
+    "531": "digital watch",
+    "532": "dining table, board",
+    "533": "dishrag, dishcloth",
+    "534": "dishwasher, dish washer, dishwashing machine",
+    "535": "disk brake, disc brake",
+    "536": "dock, dockage, docking facility",
+    "537": "dogsled, dog sled, dog sleigh",
+    "538": "dome",
+    "539": "doormat, welcome mat",
+    "540": "drilling platform, offshore rig",
+    "541": "drum, membranophone, tympan",
+    "542": "drumstick",
+    "543": "dumbbell",
+    "544": "Dutch oven",
+    "545": "electric fan, blower",
+    "546": "electric guitar",
+    "547": "electric locomotive",
+    "548": "entertainment center",
+    "549": "envelope",
+    "550": "espresso maker",
+    "551": "face powder",
+    "552": "feather boa, boa",
+    "553": "file, file cabinet, filing cabinet",
+    "554": "fireboat",
+    "555": "fire engine, fire truck",
+    "556": "fire screen, fireguard",
+    "557": "flagpole, flagstaff",
+    "558": "flute, transverse flute",
+    "559": "folding chair",
+    "560": "football helmet",
+    "561": "forklift",
+    "562": "fountain",
+    "563": "fountain pen",
+    "564": "four-poster",
+    "565": "freight car",
+    "566": "French horn, horn",
+    "567": "frying pan, frypan, skillet",
+    "568": "fur coat",
+    "569": "garbage truck, dustcart",
+    "570": "gasmask, respirator, gas helmet",
+    "571": "gas pump, gasoline pump, petrol pump, island dispenser",
+    "572": "goblet",
+    "573": "go-kart",
+    "574": "golf ball",
+    "575": "golfcart, golf cart",
+    "576": "gondola",
+    "577": "gong, tam-tam",
+    "578": "gown",
+    "579": "grand piano, grand",
+    "580": "greenhouse, nursery, glasshouse",
+    "581": "grille, radiator grille",
+    "582": "grocery store, grocery, food market, market",
+    "583": "guillotine",
+    "584": "hair slide",
+    "585": "hair spray",
+    "586": "half track",
+    "587": "hammer",
+    "588": "hamper",
+    "589": "hand blower, blow dryer, blow drier, hair dryer, hair drier",
+    "590": "hand-held computer, hand-held microcomputer",
+    "591": "handkerchief, hankie, hanky, hankey",
+    "592": "hard disc, hard disk, fixed disk",
+    "593": "harmonica, mouth organ, harp, mouth harp",
+    "594": "harp",
+    "595": "harvester, reaper",
+    "596": "hatchet",
+    "597": "holster",
+    "598": "home theater, home theatre",
+    "599": "honeycomb",
+    "600": "hook, claw",
+    "601": "hoopskirt, crinoline",
+    "602": "horizontal bar, high bar",
+    "603": "horse cart, horse-cart",
+    "604": "hourglass",
+    "605": "iPod",
+    "606": "iron, smoothing iron",
+    "607": "jack-o'-lantern",
+    "608": "jean, blue jean, denim",
+    "609": "jeep, landrover",
+    "610": "jersey, T-shirt, tee shirt",
+    "611": "jigsaw puzzle",
+    "612": "jinrikisha, ricksha, rickshaw",
+    "613": "joystick",
+    "614": "kimono",
+    "615": "knee pad",
+    "616": "knot",
+    "617": "lab coat, laboratory coat",
+    "618": "ladle",
+    "619": "lampshade, lamp shade",
+    "620": "laptop, laptop computer",
+    "621": "lawn mower, mower",
+    "622": "lens cap, lens cover",
+    "623": "letter opener, paper knife, paperknife",
+    "624": "library",
+    "625": "lifeboat",
+    "626": "lighter, light, igniter, ignitor",
+    "627": "limousine, limo",
+    "628": "liner, ocean liner",
+    "629": "lipstick, lip rouge",
+    "630": "Loafer",
+    "631": "lotion",
+    "632": "loudspeaker, speaker, speaker unit, loudspeaker system, speaker system",
+    "633": "loupe, jeweler's loupe",
+    "634": "lumbermill, sawmill",
+    "635": "magnetic compass",
+    "636": "mailbag, postbag",
+    "637": "mailbox, letter box",
+    "638": "maillot",
+    "639": "maillot, tank suit",
+    "640": "manhole cover",
+    "641": "maraca",
+    "642": "marimba, xylophone",
+    "643": "mask",
+    "644": "matchstick",
+    "645": "maypole",
+    "646": "maze, labyrinth",
+    "647": "measuring cup",
+    "648": "medicine chest, medicine cabinet",
+    "649": "megalith, megalithic structure",
+    "650": "microphone, mike",
+    "651": "microwave, microwave oven",
+    "652": "military uniform",
+    "653": "milk can",
+    "654": "minibus",
+    "655": "miniskirt, mini",
+    "656": "minivan",
+    "657": "missile",
+    "658": "mitten",
+    "659": "mixing bowl",
+    "660": "mobile home, manufactured home",
+    "661": "Model T",
+    "662": "modem",
+    "663": "monastery",
+    "664": "monitor",
+    "665": "moped",
+    "666": "mortar",
+    "667": "mortarboard",
+    "668": "mosque",
+    "669": "mosquito net",
+    "670": "motor scooter, scooter",
+    "671": "mountain bike, all-terrain bike, off-roader",
+    "672": "mountain tent",
+    "673": "mouse, computer mouse",
+    "674": "mousetrap",
+    "675": "moving van",
+    "676": "muzzle",
+    "677": "nail",
+    "678": "neck brace",
+    "679": "necklace",
+    "680": "nipple",
+    "681": "notebook, notebook computer",
+    "682": "obelisk",
+    "683": "oboe, hautboy, hautbois",
+    "684": "ocarina, sweet potato",
+    "685": "odometer, hodometer, mileometer, milometer",
+    "686": "oil filter",
+    "687": "organ, pipe organ",
+    "688": "oscilloscope, scope, cathode-ray oscilloscope, CRO",
+    "689": "overskirt",
+    "690": "oxcart",
+    "691": "oxygen mask",
+    "692": "packet",
+    "693": "paddle, boat paddle",
+    "694": "paddlewheel, paddle wheel",
+    "695": "padlock",
+    "696": "paintbrush",
+    "697": "pajama, pyjama, pj's, jammies",
+    "698": "palace",
+    "699": "panpipe, pandean pipe, syrinx",
+    "700": "paper towel",
+    "701": "parachute, chute",
+    "702": "parallel bars, bars",
+    "703": "park bench",
+    "704": "parking meter",
+    "705": "passenger car, coach, carriage",
+    "706": "patio, terrace",
+    "707": "pay-phone, pay-station",
+    "708": "pedestal, plinth, footstall",
+    "709": "pencil box, pencil case",
+    "710": "pencil sharpener",
+    "711": "perfume, essence",
+    "712": "Petri dish",
+    "713": "photocopier",
+    "714": "pick, plectrum, plectron",
+    "715": "pickelhaube",
+    "716": "picket fence, paling",
+    "717": "pickup, pickup truck",
+    "718": "pier",
+    "719": "piggy bank, penny bank",
+    "720": "pill bottle",
+    "721": "pillow",
+    "722": "ping-pong ball",
+    "723": "pinwheel",
+    "724": "pirate, pirate ship",
+    "725": "pitcher, ewer",
+    "726": "plane, carpenter's plane, woodworking plane",
+    "727": "planetarium",
+    "728": "plastic bag",
+    "729": "plate rack",
+    "730": "plow, plough",
+    "731": "plunger, plumber's helper",
+    "732": "Polaroid camera, Polaroid Land camera",
+    "733": "pole",
+    "734": "police van, police wagon, paddy wagon, patrol wagon, wagon, black Maria",
+    "735": "poncho",
+    "736": "pool table, billiard table, snooker table",
+    "737": "pop bottle, soda bottle",
+    "738": "pot, flowerpot",
+    "739": "potter's wheel",
+    "740": "power drill",
+    "741": "prayer rug, prayer mat",
+    "742": "printer",
+    "743": "prison, prison house",
+    "744": "projectile, missile",
+    "745": "projector",
+    "746": "puck, hockey puck",
+    "747": "punching bag, punch bag, punching ball, punchball",
+    "748": "purse",
+    "749": "quill, quill pen",
+    "750": "quilt, comforter, comfort, puff",
+    "751": "racer, race car, racing car",
+    "752": "racket, racquet",
+    "753": "radiator",
+    "754": "radio, wireless",
+    "755": "radio telescope, radio reflector",
+    "756": "rain barrel",
+    "757": "recreational vehicle, RV, R.V.",
+    "758": "reel",
+    "759": "reflex camera",
+    "760": "refrigerator, icebox",
+    "761": "remote control, remote",
+    "762": "restaurant, eating house, eating place, eatery",
+    "763": "revolver, six-gun, six-shooter",
+    "764": "rifle",
+    "765": "rocking chair, rocker",
+    "766": "rotisserie",
+    "767": "rubber eraser, rubber, pencil eraser",
+    "768": "rugby ball",
+    "769": "rule, ruler",
+    "770": "running shoe",
+    "771": "safe",
+    "772": "safety pin",
+    "773": "saltshaker, salt shaker",
+    "774": "sandal",
+    "775": "sarong",
+    "776": "sax, saxophone",
+    "777": "scabbard",
+    "778": "scale, weighing machine",
+    "779": "school bus",
+    "780": "schooner",
+    "781": "scoreboard",
+    "782": "screen, CRT screen",
+    "783": "screw",
+    "784": "screwdriver",
+    "785": "seat belt, seatbelt",
+    "786": "sewing machine",
+    "787": "shield, buckler",
+    "788": "shoe shop, shoe-shop, shoe store",
+    "789": "shoji",
+    "790": "shopping basket",
+    "791": "shopping cart",
+    "792": "shovel",
+    "793": "shower cap",
+    "794": "shower curtain",
+    "795": "ski",
+    "796": "ski mask",
+    "797": "sleeping bag",
+    "798": "slide rule, slipstick",
+    "799": "sliding door",
+    "800": "slot, one-armed bandit",
+    "801": "snorkel",
+    "802": "snowmobile",
+    "803": "snowplow, snowplough",
+    "804": "soap dispenser",
+    "805": "soccer ball",
+    "806": "sock",
+    "807": "solar dish, solar collector, solar furnace",
+    "808": "sombrero",
+    "809": "soup bowl",
+    "810": "space bar",
+    "811": "space heater",
+    "812": "space shuttle",
+    "813": "spatula",
+    "814": "speedboat",
+    "815": "spider web, spider's web",
+    "816": "spindle",
+    "817": "sports car, sport car",
+    "818": "spotlight, spot",
+    "819": "stage",
+    "820": "steam locomotive",
+    "821": "steel arch bridge",
+    "822": "steel drum",
+    "823": "stethoscope",
+    "824": "stole",
+    "825": "stone wall",
+    "826": "stopwatch, stop watch",
+    "827": "stove",
+    "828": "strainer",
+    "829": "streetcar, tram, tramcar, trolley, trolley car",
+    "830": "stretcher",
+    "831": "studio couch, day bed",
+    "832": "stupa, tope",
+    "833": "submarine, pigboat, sub, U-boat",
+    "834": "suit, suit of clothes",
+    "835": "sundial",
+    "836": "sunglass",
+    "837": "sunglasses, dark glasses, shades",
+    "838": "sunscreen, sunblock, sun blocker",
+    "839": "suspension bridge",
+    "840": "swab, swob, mop",
+    "841": "sweatshirt",
+    "842": "swimming trunks, bathing trunks",
+    "843": "swing",
+    "844": "switch, electric switch, electrical switch",
+    "845": "syringe",
+    "846": "table lamp",
+    "847": "tank, army tank, armored combat vehicle, armoured combat vehicle",
+    "848": "tape player",
+    "849": "teapot",
+    "850": "teddy, teddy bear",
+    "851": "television, television system",
+    "852": "tennis ball",
+    "853": "thatch, thatched roof",
+    "854": "theater curtain, theatre curtain",
+    "855": "thimble",
+    "856": "thresher, thrasher, threshing machine",
+    "857": "throne",
+    "858": "tile roof",
+    "859": "toaster",
+    "860": "tobacco shop, tobacconist shop, tobacconist",
+    "861": "toilet seat",
+    "862": "torch",
+    "863": "totem pole",
+    "864": "tow truck, tow car, wrecker",
+    "865": "toyshop",
+    "866": "tractor",
+    "867": "trailer truck, tractor trailer, trucking rig, rig, articulated lorry, semi",
+    "868": "tray",
+    "869": "trench coat",
+    "870": "tricycle, trike, velocipede",
+    "871": "trimaran",
+    "872": "tripod",
+    "873": "triumphal arch",
+    "874": "trolleybus, trolley coach, trackless trolley",
+    "875": "trombone",
+    "876": "tub, vat",
+    "877": "turnstile",
+    "878": "typewriter keyboard",
+    "879": "umbrella",
+    "880": "unicycle, monocycle",
+    "881": "upright, upright piano",
+    "882": "vacuum, vacuum cleaner",
+    "883": "vase",
+    "884": "vault",
+    "885": "velvet",
+    "886": "vending machine",
+    "887": "vestment",
+    "888": "viaduct",
+    "889": "violin, fiddle",
+    "890": "volleyball",
+    "891": "waffle iron",
+    "892": "wall clock",
+    "893": "wallet, billfold, notecase, pocketbook",
+    "894": "wardrobe, closet, press",
+    "895": "warplane, military plane",
+    "896": "washbasin, handbasin, washbowl, lavabo, wash-hand basin",
+    "897": "washer, automatic washer, washing machine",
+    "898": "water bottle",
+    "899": "water jug",
+    "900": "water tower",
+    "901": "whiskey jug",
+    "902": "whistle",
+    "903": "wig",
+    "904": "window screen",
+    "905": "window shade",
+    "906": "Windsor tie",
+    "907": "wine bottle",
+    "908": "wing",
+    "909": "wok",
+    "910": "wooden spoon",
+    "911": "wool, woolen, woollen",
+    "912": "worm fence, snake fence, snake-rail fence, Virginia fence",
+    "913": "wreck",
+    "914": "yawl",
+    "915": "yurt",
+    "916": "web site, website, internet site, site",
+    "917": "comic book",
+    "918": "crossword puzzle, crossword",
+    "919": "street sign",
+    "920": "traffic light, traffic signal, stoplight",
+    "921": "book jacket, dust cover, dust jacket, dust wrapper",
+    "922": "menu",
+    "923": "plate",
+    "924": "guacamole",
+    "925": "consomme",
+    "926": "hot pot, hotpot",
+    "927": "trifle",
+    "928": "ice cream, icecream",
+    "929": "ice lolly, lolly, lollipop, popsicle",
+    "930": "French loaf",
+    "931": "bagel, beigel",
+    "932": "pretzel",
+    "933": "cheeseburger",
+    "934": "hotdog, hot dog, red hot",
+    "935": "mashed potato",
+    "936": "head cabbage",
+    "937": "broccoli",
+    "938": "cauliflower",
+    "939": "zucchini, courgette",
+    "940": "spaghetti squash",
+    "941": "acorn squash",
+    "942": "butternut squash",
+    "943": "cucumber, cuke",
+    "944": "artichoke, globe artichoke",
+    "945": "bell pepper",
+    "946": "cardoon",
+    "947": "mushroom",
+    "948": "Granny Smith",
+    "949": "strawberry",
+    "950": "orange",
+    "951": "lemon",
+    "952": "fig",
+    "953": "pineapple, ananas",
+    "954": "banana",
+    "955": "jackfruit, jak, jack",
+    "956": "custard apple",
+    "957": "pomegranate",
+    "958": "hay",
+    "959": "carbonara",
+    "960": "chocolate sauce, chocolate syrup",
+    "961": "dough",
+    "962": "meat loaf, meatloaf",
+    "963": "pizza, pizza pie",
+    "964": "potpie",
+    "965": "burrito",
+    "966": "red wine",
+    "967": "espresso",
+    "968": "cup",
+    "969": "eggnog",
+    "970": "alp",
+    "971": "bubble",
+    "972": "cliff, drop, drop-off",
+    "973": "coral reef",
+    "974": "geyser",
+    "975": "lakeside, lakeshore",
+    "976": "promontory, headland, head, foreland",
+    "977": "sandbar, sand bar",
+    "978": "seashore, coast, seacoast, sea-coast",
+    "979": "valley, vale",
+    "980": "volcano",
+    "981": "ballplayer, baseball player",
+    "982": "groom, bridegroom",
+    "983": "scuba diver",
+    "984": "rapeseed",
+    "985": "daisy",
+    "986": "yellow lady's slipper, yellow lady-slipper, Cypripedium calceolus, Cypripedium parviflorum",
+    "987": "corn",
+    "988": "acorn",
+    "989": "hip, rose hip, rosehip",
+    "990": "buckeye, horse chestnut, conker",
+    "991": "coral fungus",
+    "992": "agaric",
+    "993": "gyromitra",
+    "994": "stinkhorn, carrion fungus",
+    "995": "earthstar",
+    "996": "hen-of-the-woods, hen of the woods, Polyporus frondosus, Grifola frondosa",
+    "997": "bolete",
+    "998": "ear, spike, capitulum",
+    "999": "toilet tissue, toilet paper, bathroom tissue"
+  },
+  "image_size": 224,
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "label2id": {
+    "Afghan hound, Afghan": 160,
+    "African chameleon, Chamaeleo chamaeleon": 47,
+    "African crocodile, Nile crocodile, Crocodylus niloticus": 49,
+    "African elephant, Loxodonta africana": 386,
+    "African grey, African gray, Psittacus erithacus": 87,
+    "African hunting dog, hyena dog, Cape hunting dog, Lycaon pictus": 275,
+    "Airedale, Airedale terrier": 191,
+    "American Staffordshire terrier, Staffordshire terrier, American pit bull terrier, pit bull terrier": 180,
+    "American alligator, Alligator mississipiensis": 50,
+    "American black bear, black bear, Ursus americanus, Euarctos americanus": 295,
+    "American chameleon, anole, Anolis carolinensis": 40,
+    "American coot, marsh hen, mud hen, water hen, Fulica americana": 137,
+    "American egret, great white heron, Egretta albus": 132,
+    "American lobster, Northern lobster, Maine lobster, Homarus americanus": 122,
+    "Angora, Angora rabbit": 332,
+    "Appenzeller": 240,
+    "Arabian camel, dromedary, Camelus dromedarius": 354,
+    "Arctic fox, white fox, Alopex lagopus": 279,
+    "Australian terrier": 193,
+    "Band Aid": 419,
+    "Bedlington terrier": 181,
+    "Bernese mountain dog": 239,
+    "Blenheim spaniel": 156,
+    "Border collie": 232,
+    "Border terrier": 182,
+    "Boston bull, Boston terrier": 195,
+    "Bouvier des Flandres, Bouviers des Flandres": 233,
+    "Brabancon griffon": 262,
+    "Brittany spaniel": 215,
+    "CD player": 485,
+    "Cardigan, Cardigan Welsh corgi": 264,
+    "Chesapeake Bay retriever": 209,
+    "Chihuahua": 151,
+    "Christmas stocking": 496,
+    "Crock Pot": 521,
+    "Dandie Dinmont, Dandie Dinmont terrier": 194,
+    "Doberman, Doberman pinscher": 236,
+    "Dungeness crab, Cancer magister": 118,
+    "Dutch oven": 544,
+    "Egyptian cat": 285,
+    "English foxhound": 167,
+    "English setter": 212,
+    "English springer, English springer spaniel": 217,
+    "EntleBucher": 241,
+    "Eskimo dog, husky": 248,
+    "European fire salamander, Salamandra salamandra": 25,
+    "European gallinule, Porphyrio porphyrio": 136,
+    "French bulldog": 245,
+    "French horn, horn": 566,
+    "French loaf": 930,
+    "German shepherd, German shepherd dog, German police dog, alsatian": 235,
+    "German short-haired pointer": 210,
+    "Gila monster, Heloderma suspectum": 45,
+    "Gordon setter": 214,
+    "Granny Smith": 948,
+    "Great Dane": 246,
+    "Great Pyrenees": 257,
+    "Greater Swiss Mountain dog": 238,
+    "Ibizan hound, Ibizan Podenco": 173,
+    "Indian cobra, Naja naja": 63,
+    "Indian elephant, Elephas maximus": 385,
+    "Irish setter, red setter": 213,
+    "Irish terrier": 184,
+    "Irish water spaniel": 221,
+    "Irish wolfhound": 170,
+    "Italian greyhound": 171,
+    "Japanese spaniel": 152,
+    "Kerry blue terrier": 183,
+    "Komodo dragon, Komodo lizard, dragon lizard, giant lizard, Varanus komodoensis": 48,
+    "Labrador retriever": 208,
+    "Lakeland terrier": 189,
+    "Leonberg": 255,
+    "Lhasa, Lhasa apso": 204,
+    "Loafer": 630,
+    "Madagascar cat, ring-tailed lemur, Lemur catta": 383,
+    "Maltese dog, Maltese terrier, Maltese": 153,
+    "Mexican hairless": 268,
+    "Model T": 661,
+    "Newfoundland, Newfoundland dog": 256,
+    "Norfolk terrier": 185,
+    "Norwegian elkhound, elkhound": 174,
+    "Norwich terrier": 186,
+    "Old English sheepdog, bobtail": 229,
+    "Pekinese, Pekingese, Peke": 154,
+    "Pembroke, Pembroke Welsh corgi": 263,
+    "Persian cat": 283,
+    "Petri dish": 712,
+    "Polaroid camera, Polaroid Land camera": 732,
+    "Pomeranian": 259,
+    "Rhodesian ridgeback": 159,
+    "Rottweiler": 234,
+    "Saint Bernard, St Bernard": 247,
+    "Saluki, gazelle hound": 176,
+    "Samoyed, Samoyede": 258,
+    "Scotch terrier, Scottish terrier, Scottie": 199,
+    "Scottish deerhound, deerhound": 177,
+    "Sealyham terrier, Sealyham": 190,
+    "Shetland sheepdog, Shetland sheep dog, Shetland": 230,
+    "Shih-Tzu": 155,
+    "Siamese cat, Siamese": 284,
+    "Siberian husky": 250,
+    "Staffordshire bullterrier, Staffordshire bull terrier": 179,
+    "Sussex spaniel": 220,
+    "Tibetan mastiff": 244,
+    "Tibetan terrier, chrysanthemum dog": 200,
+    "Walker hound, Walker foxhound": 166,
+    "Weimaraner": 178,
+    "Welsh springer spaniel": 218,
+    "West Highland white terrier": 203,
+    "Windsor tie": 906,
+    "Yorkshire terrier": 187,
+    "abacus": 398,
+    "abaya": 399,
+    "academic gown, academic robe, judge's robe": 400,
+    "accordion, piano accordion, squeeze box": 401,
+    "acorn": 988,
+    "acorn squash": 941,
+    "acoustic guitar": 402,
+    "admiral": 321,
+    "affenpinscher, monkey pinscher, monkey dog": 252,
+    "agama": 42,
+    "agaric": 992,
+    "aircraft carrier, carrier, flattop, attack aircraft carrier": 403,
+    "airliner": 404,
+    "airship, dirigible": 405,
+    "albatross, mollymawk": 146,
+    "alligator lizard": 44,
+    "alp": 970,
+    "altar": 406,
+    "ambulance": 407,
+    "amphibian, amphibious vehicle": 408,
+    "analog clock": 409,
+    "anemone fish": 393,
+    "ant, emmet, pismire": 310,
+    "apiary, bee house": 410,
+    "apron": 411,
+    "armadillo": 363,
+    "artichoke, globe artichoke": 944,
+    "ashcan, trash can, garbage can, wastebin, ash bin, ash-bin, ashbin, dustbin, trash barrel, trash bin": 412,
+    "assault rifle, assault gun": 413,
+    "axolotl, mud puppy, Ambystoma mexicanum": 29,
+    "baboon": 372,
+    "backpack, back pack, knapsack, packsack, rucksack, haversack": 414,
+    "badger": 362,
+    "bagel, beigel": 931,
+    "bakery, bakeshop, bakehouse": 415,
+    "balance beam, beam": 416,
+    "bald eagle, American eagle, Haliaeetus leucocephalus": 22,
+    "balloon": 417,
+    "ballplayer, baseball player": 981,
+    "ballpoint, ballpoint pen, ballpen, Biro": 418,
+    "banana": 954,
+    "banded gecko": 38,
+    "banjo": 420,
+    "bannister, banister, balustrade, balusters, handrail": 421,
+    "barbell": 422,
+    "barber chair": 423,
+    "barbershop": 424,
+    "barn": 425,
+    "barn spider, Araneus cavaticus": 73,
+    "barometer": 426,
+    "barracouta, snoek": 389,
+    "barrel, cask": 427,
+    "barrow, garden cart, lawn cart, wheelbarrow": 428,
+    "baseball": 429,
+    "basenji": 253,
+    "basketball": 430,
+    "basset, basset hound": 161,
+    "bassinet": 431,
+    "bassoon": 432,
+    "bath towel": 434,
+    "bathing cap, swimming cap": 433,
+    "bathtub, bathing tub, bath, tub": 435,
+    "beach wagon, station wagon, wagon, estate car, beach waggon, station waggon, waggon": 436,
+    "beacon, lighthouse, beacon light, pharos": 437,
+    "beagle": 162,
+    "beaker": 438,
+    "bearskin, busby, shako": 439,
+    "beaver": 337,
+    "bee": 309,
+    "bee eater": 92,
+    "beer bottle": 440,
+    "beer glass": 441,
+    "bell cote, bell cot": 442,
+    "bell pepper": 945,
+    "bib": 443,
+    "bicycle-built-for-two, tandem bicycle, tandem": 444,
+    "bighorn, bighorn sheep, cimarron, Rocky Mountain bighorn, Rocky Mountain sheep, Ovis canadensis": 349,
+    "bikini, two-piece": 445,
+    "binder, ring-binder": 446,
+    "binoculars, field glasses, opera glasses": 447,
+    "birdhouse": 448,
+    "bison": 347,
+    "bittern": 133,
+    "black and gold garden spider, Argiope aurantia": 72,
+    "black grouse": 80,
+    "black stork, Ciconia nigra": 128,
+    "black swan, Cygnus atratus": 100,
+    "black widow, Latrodectus mactans": 75,
+    "black-and-tan coonhound": 165,
+    "black-footed ferret, ferret, Mustela nigripes": 359,
+    "bloodhound, sleuthhound": 163,
+    "bluetick": 164,
+    "boa constrictor, Constrictor constrictor": 61,
+    "boathouse": 449,
+    "bobsled, bobsleigh, bob": 450,
+    "bolete": 997,
+    "bolo tie, bolo, bola tie, bola": 451,
+    "bonnet, poke bonnet": 452,
+    "book jacket, dust cover, dust jacket, dust wrapper": 921,
+    "bookcase": 453,
+    "bookshop, bookstore, bookstall": 454,
+    "borzoi, Russian wolfhound": 169,
+    "bottlecap": 455,
+    "bow": 456,
+    "bow tie, bow-tie, bowtie": 457,
+    "box turtle, box tortoise": 37,
+    "boxer": 242,
+    "brain coral": 109,
+    "brambling, Fringilla montifringilla": 10,
+    "brass, memorial tablet, plaque": 458,
+    "brassiere, bra, bandeau": 459,
+    "breakwater, groin, groyne, mole, bulwark, seawall, jetty": 460,
+    "breastplate, aegis, egis": 461,
+    "briard": 226,
+    "broccoli": 937,
+    "broom": 462,
+    "brown bear, bruin, Ursus arctos": 294,
+    "bubble": 971,
+    "bucket, pail": 463,
+    "buckeye, horse chestnut, conker": 990,
+    "buckle": 464,
+    "bulbul": 16,
+    "bull mastiff": 243,
+    "bullet train, bullet": 466,
+    "bulletproof vest": 465,
+    "bullfrog, Rana catesbeiana": 30,
+    "burrito": 965,
+    "bustard": 138,
+    "butcher shop, meat market": 467,
+    "butternut squash": 942,
+    "cab, hack, taxi, taxicab": 468,
+    "cabbage butterfly": 324,
+    "cairn, cairn terrier": 192,
+    "caldron, cauldron": 469,
+    "can opener, tin opener": 473,
+    "candle, taper, wax light": 470,
+    "cannon": 471,
+    "canoe": 472,
+    "capuchin, ringtail, Cebus capucinus": 378,
+    "car mirror": 475,
+    "car wheel": 479,
+    "carbonara": 959,
+    "cardigan": 474,
+    "cardoon": 946,
+    "carousel, carrousel, merry-go-round, roundabout, whirligig": 476,
+    "carpenter's kit, tool kit": 477,
+    "carton": 478,
+    "cash machine, cash dispenser, automated teller machine, automatic teller machine, automated teller, automatic teller, ATM": 480,
+    "cassette": 481,
+    "cassette player": 482,
+    "castle": 483,
+    "catamaran": 484,
+    "cauliflower": 938,
+    "cello, violoncello": 486,
+    "cellular telephone, cellular phone, cellphone, cell, mobile phone": 487,
+    "centipede": 79,
+    "chain": 488,
+    "chain mail, ring mail, mail, chain armor, chain armour, ring armor, ring armour": 490,
+    "chain saw, chainsaw": 491,
+    "chainlink fence": 489,
+    "chambered nautilus, pearly nautilus, nautilus": 117,
+    "cheeseburger": 933,
+    "cheetah, chetah, Acinonyx jubatus": 293,
+    "chest": 492,
+    "chickadee": 19,
+    "chiffonier, commode": 493,
+    "chime, bell, gong": 494,
+    "chimpanzee, chimp, Pan troglodytes": 367,
+    "china cabinet, china closet": 495,
+    "chiton, coat-of-mail shell, sea cradle, polyplacophore": 116,
+    "chocolate sauce, chocolate syrup": 960,
+    "chow, chow chow": 260,
+    "church, church building": 497,
+    "cicada, cicala": 316,
+    "cinema, movie theater, movie theatre, movie house, picture palace": 498,
+    "cleaver, meat cleaver, chopper": 499,
+    "cliff dwelling": 500,
+    "cliff, drop, drop-off": 972,
+    "cloak": 501,
+    "clog, geta, patten, sabot": 502,
+    "clumber, clumber spaniel": 216,
+    "cock": 7,
+    "cocker spaniel, English cocker spaniel, cocker": 219,
+    "cockroach, roach": 314,
+    "cocktail shaker": 503,
+    "coffee mug": 504,
+    "coffeepot": 505,
+    "coho, cohoe, coho salmon, blue jack, silver salmon, Oncorhynchus kisutch": 391,
+    "coil, spiral, volute, whorl, helix": 506,
+    "collie": 231,
+    "colobus, colobus monkey": 375,
+    "combination lock": 507,
+    "comic book": 917,
+    "common iguana, iguana, Iguana iguana": 39,
+    "common newt, Triturus vulgaris": 26,
+    "computer keyboard, keypad": 508,
+    "conch": 112,
+    "confectionery, confectionary, candy store": 509,
+    "consomme": 925,
+    "container ship, containership, container vessel": 510,
+    "convertible": 511,
+    "coral fungus": 991,
+    "coral reef": 973,
+    "corkscrew, bottle screw": 512,
+    "corn": 987,
+    "cornet, horn, trumpet, trump": 513,
+    "coucal": 91,
+    "cougar, puma, catamount, mountain lion, painter, panther, Felis concolor": 286,
+    "cowboy boot": 514,
+    "cowboy hat, ten-gallon hat": 515,
+    "coyote, prairie wolf, brush wolf, Canis latrans": 272,
+    "cradle": 516,
+    "crane": 517,
+    "crash helmet": 518,
+    "crate": 519,
+    "crayfish, crawfish, crawdad, crawdaddy": 124,
+    "crib, cot": 520,
+    "cricket": 312,
+    "croquet ball": 522,
+    "crossword puzzle, crossword": 918,
+    "crutch": 523,
+    "cucumber, cuke": 943,
+    "cuirass": 524,
+    "cup": 968,
+    "curly-coated retriever": 206,
+    "custard apple": 956,
+    "daisy": 985,
+    "dalmatian, coach dog, carriage dog": 251,
+    "dam, dike, dyke": 525,
+    "damselfly": 320,
+    "desk": 526,
+    "desktop computer": 527,
+    "dhole, Cuon alpinus": 274,
+    "dial telephone, dial phone": 528,
+    "diamondback, diamondback rattlesnake, Crotalus adamanteus": 67,
+    "diaper, nappy, napkin": 529,
+    "digital clock": 530,
+    "digital watch": 531,
+    "dingo, warrigal, warragal, Canis dingo": 273,
+    "dining table, board": 532,
+    "dishrag, dishcloth": 533,
+    "dishwasher, dish washer, dishwashing machine": 534,
+    "disk brake, disc brake": 535,
+    "dock, dockage, docking facility": 536,
+    "dogsled, dog sled, dog sleigh": 537,
+    "dome": 538,
+    "doormat, welcome mat": 539,
+    "dough": 961,
+    "dowitcher": 142,
+    "dragonfly, darning needle, devil's darning needle, sewing needle, snake feeder, snake doctor, mosquito hawk, skeeter hawk": 319,
+    "drake": 97,
+    "drilling platform, offshore rig": 540,
+    "drum, membranophone, tympan": 541,
+    "drumstick": 542,
+    "dugong, Dugong dugon": 149,
+    "dumbbell": 543,
+    "dung beetle": 305,
+    "ear, spike, capitulum": 998,
+    "earthstar": 995,
+    "echidna, spiny anteater, anteater": 102,
+    "eel": 390,
+    "eft": 27,
+    "eggnog": 969,
+    "electric fan, blower": 545,
+    "electric guitar": 546,
+    "electric locomotive": 547,
+    "electric ray, crampfish, numbfish, torpedo": 5,
+    "entertainment center": 548,
+    "envelope": 549,
+    "espresso": 967,
+    "espresso maker": 550,
+    "face powder": 551,
+    "feather boa, boa": 552,
+    "fiddler crab": 120,
+    "fig": 952,
+    "file, file cabinet, filing cabinet": 553,
+    "fire engine, fire truck": 555,
+    "fire screen, fireguard": 556,
+    "fireboat": 554,
+    "flagpole, flagstaff": 557,
+    "flamingo": 130,
+    "flat-coated retriever": 205,
+    "flatworm, platyhelminth": 110,
+    "flute, transverse flute": 558,
+    "fly": 308,
+    "folding chair": 559,
+    "football helmet": 560,
+    "forklift": 561,
+    "fountain": 562,
+    "fountain pen": 563,
+    "four-poster": 564,
+    "fox squirrel, eastern fox squirrel, Sciurus niger": 335,
+    "freight car": 565,
+    "frilled lizard, Chlamydosaurus kingi": 43,
+    "frying pan, frypan, skillet": 567,
+    "fur coat": 568,
+    "gar, garfish, garpike, billfish, Lepisosteus osseus": 395,
+    "garbage truck, dustcart": 569,
+    "garden spider, Aranea diademata": 74,
+    "garter snake, grass snake": 57,
+    "gas pump, gasoline pump, petrol pump, island dispenser": 571,
+    "gasmask, respirator, gas helmet": 570,
+    "gazelle": 353,
+    "geyser": 974,
+    "giant panda, panda, panda bear, coon bear, Ailuropoda melanoleuca": 388,
+    "giant schnauzer": 197,
+    "gibbon, Hylobates lar": 368,
+    "go-kart": 573,
+    "goblet": 572,
+    "golden retriever": 207,
+    "goldfinch, Carduelis carduelis": 11,
+    "goldfish, Carassius auratus": 1,
+    "golf ball": 574,
+    "golfcart, golf cart": 575,
+    "gondola": 576,
+    "gong, tam-tam": 577,
+    "goose": 99,
+    "gorilla, Gorilla gorilla": 366,
+    "gown": 578,
+    "grand piano, grand": 579,
+    "grasshopper, hopper": 311,
+    "great grey owl, great gray owl, Strix nebulosa": 24,
+    "great white shark, white shark, man-eater, man-eating shark, Carcharodon carcharias": 2,
+    "green lizard, Lacerta viridis": 46,
+    "green mamba": 64,
+    "green snake, grass snake": 55,
+    "greenhouse, nursery, glasshouse": 580,
+    "grey fox, gray fox, Urocyon cinereoargenteus": 280,
+    "grey whale, gray whale, devilfish, Eschrichtius gibbosus, Eschrichtius robustus": 147,
+    "grille, radiator grille": 581,
+    "grocery store, grocery, food market, market": 582,
+    "groenendael": 224,
+    "groom, bridegroom": 982,
+    "ground beetle, carabid beetle": 302,
+    "guacamole": 924,
+    "guenon, guenon monkey": 370,
+    "guillotine": 583,
+    "guinea pig, Cavia cobaya": 338,
+    "gyromitra": 993,
+    "hair slide": 584,
+    "hair spray": 585,
+    "half track": 586,
+    "hammer": 587,
+    "hammerhead, hammerhead shark": 4,
+    "hamper": 588,
+    "hamster": 333,
+    "hand blower, blow dryer, blow drier, hair dryer, hair drier": 589,
+    "hand-held computer, hand-held microcomputer": 590,
+    "handkerchief, hankie, hanky, hankey": 591,
+    "hard disc, hard disk, fixed disk": 592,
+    "hare": 331,
+    "harmonica, mouth organ, harp, mouth harp": 593,
+    "harp": 594,
+    "hartebeest": 351,
+    "harvester, reaper": 595,
+    "harvestman, daddy longlegs, Phalangium opilio": 70,
+    "hatchet": 596,
+    "hay": 958,
+    "head cabbage": 936,
+    "hen": 8,
+    "hen-of-the-woods, hen of the woods, Polyporus frondosus, Grifola frondosa": 996,
+    "hermit crab": 125,
+    "hip, rose hip, rosehip": 989,
+    "hippopotamus, hippo, river horse, Hippopotamus amphibius": 344,
+    "hog, pig, grunter, squealer, Sus scrofa": 341,
+    "hognose snake, puff adder, sand viper": 54,
+    "holster": 597,
+    "home theater, home theatre": 598,
+    "honeycomb": 599,
+    "hook, claw": 600,
+    "hoopskirt, crinoline": 601,
+    "horizontal bar, high bar": 602,
+    "hornbill": 93,
+    "horned viper, cerastes, sand viper, horned asp, Cerastes cornutus": 66,
+    "horse cart, horse-cart": 603,
+    "hot pot, hotpot": 926,
+    "hotdog, hot dog, red hot": 934,
+    "hourglass": 604,
+    "house finch, linnet, Carpodacus mexicanus": 12,
+    "howler monkey, howler": 379,
+    "hummingbird": 94,
+    "hyena, hyaena": 276,
+    "iPod": 605,
+    "ibex, Capra ibex": 350,
+    "ice bear, polar bear, Ursus Maritimus, Thalarctos maritimus": 296,
+    "ice cream, icecream": 928,
+    "ice lolly, lolly, lollipop, popsicle": 929,
+    "impala, Aepyceros melampus": 352,
+    "indigo bunting, indigo finch, indigo bird, Passerina cyanea": 14,
+    "indri, indris, Indri indri, Indri brevicaudatus": 384,
+    "iron, smoothing iron": 606,
+    "isopod": 126,
+    "jacamar": 95,
+    "jack-o'-lantern": 607,
+    "jackfruit, jak, jack": 955,
+    "jaguar, panther, Panthera onca, Felis onca": 290,
+    "jay": 17,
+    "jean, blue jean, denim": 608,
+    "jeep, landrover": 609,
+    "jellyfish": 107,
+    "jersey, T-shirt, tee shirt": 610,
+    "jigsaw puzzle": 611,
+    "jinrikisha, ricksha, rickshaw": 612,
+    "joystick": 613,
+    "junco, snowbird": 13,
+    "keeshond": 261,
+    "kelpie": 227,
+    "killer whale, killer, orca, grampus, sea wolf, Orcinus orca": 148,
+    "kimono": 614,
+    "king crab, Alaska crab, Alaskan king crab, Alaska king crab, Paralithodes camtschatica": 121,
+    "king penguin, Aptenodytes patagonica": 145,
+    "king snake, kingsnake": 56,
+    "kit fox, Vulpes macrotis": 278,
+    "kite": 21,
+    "knee pad": 615,
+    "knot": 616,
+    "koala, koala bear, kangaroo bear, native bear, Phascolarctos cinereus": 105,
+    "komondor": 228,
+    "kuvasz": 222,
+    "lab coat, laboratory coat": 617,
+    "lacewing, lacewing fly": 318,
+    "ladle": 618,
+    "ladybug, ladybeetle, lady beetle, ladybird, ladybird beetle": 301,
+    "lakeside, lakeshore": 975,
+    "lampshade, lamp shade": 619,
+    "langur": 374,
+    "laptop, laptop computer": 620,
+    "lawn mower, mower": 621,
+    "leaf beetle, chrysomelid": 304,
+    "leafhopper": 317,
+    "leatherback turtle, leatherback, leathery turtle, Dermochelys coriacea": 34,
+    "lemon": 951,
+    "lens cap, lens cover": 622,
+    "leopard, Panthera pardus": 288,
+    "lesser panda, red panda, panda, bear cat, cat bear, Ailurus fulgens": 387,
+    "letter opener, paper knife, paperknife": 623,
+    "library": 624,
+    "lifeboat": 625,
+    "lighter, light, igniter, ignitor": 626,
+    "limousine, limo": 627,
+    "limpkin, Aramus pictus": 135,
+    "liner, ocean liner": 628,
+    "lion, king of beasts, Panthera leo": 291,
+    "lionfish": 396,
+    "lipstick, lip rouge": 629,
+    "little blue heron, Egretta caerulea": 131,
+    "llama": 355,
+    "loggerhead, loggerhead turtle, Caretta caretta": 33,
+    "long-horned beetle, longicorn, longicorn beetle": 303,
+    "lorikeet": 90,
+    "lotion": 631,
+    "loudspeaker, speaker, speaker unit, loudspeaker system, speaker system": 632,
+    "loupe, jeweler's loupe": 633,
+    "lumbermill, sawmill": 634,
+    "lycaenid, lycaenid butterfly": 326,
+    "lynx, catamount": 287,
+    "macaque": 373,
+    "macaw": 88,
+    "magnetic compass": 635,
+    "magpie": 18,
+    "mailbag, postbag": 636,
+    "mailbox, letter box": 637,
+    "maillot": 638,
+    "maillot, tank suit": 639,
+    "malamute, malemute, Alaskan malamute": 249,
+    "malinois": 225,
+    "manhole cover": 640,
+    "mantis, mantid": 315,
+    "maraca": 641,
+    "marimba, xylophone": 642,
+    "marmoset": 377,
+    "marmot": 336,
+    "mashed potato": 935,
+    "mask": 643,
+    "matchstick": 644,
+    "maypole": 645,
+    "maze, labyrinth": 646,
+    "measuring cup": 647,
+    "meat loaf, meatloaf": 962,
+    "medicine chest, medicine cabinet": 648,
+    "meerkat, mierkat": 299,
+    "megalith, megalithic structure": 649,
+    "menu": 922,
+    "microphone, mike": 650,
+    "microwave, microwave oven": 651,
+    "military uniform": 652,
+    "milk can": 653,
+    "miniature pinscher": 237,
+    "miniature poodle": 266,
+    "miniature schnauzer": 196,
+    "minibus": 654,
+    "miniskirt, mini": 655,
+    "minivan": 656,
+    "mink": 357,
+    "missile": 657,
+    "mitten": 658,
+    "mixing bowl": 659,
+    "mobile home, manufactured home": 660,
+    "modem": 662,
+    "monarch, monarch butterfly, milkweed butterfly, Danaus plexippus": 323,
+    "monastery": 663,
+    "mongoose": 298,
+    "monitor": 664,
+    "moped": 665,
+    "mortar": 666,
+    "mortarboard": 667,
+    "mosque": 668,
+    "mosquito net": 669,
+    "motor scooter, scooter": 670,
+    "mountain bike, all-terrain bike, off-roader": 671,
+    "mountain tent": 672,
+    "mouse, computer mouse": 673,
+    "mousetrap": 674,
+    "moving van": 675,
+    "mud turtle": 35,
+    "mushroom": 947,
+    "muzzle": 676,
+    "nail": 677,
+    "neck brace": 678,
+    "necklace": 679,
+    "nematode, nematode worm, roundworm": 111,
+    "night snake, Hypsiglena torquata": 60,
+    "nipple": 680,
+    "notebook, notebook computer": 681,
+    "obelisk": 682,
+    "oboe, hautboy, hautbois": 683,
+    "ocarina, sweet potato": 684,
+    "odometer, hodometer, mileometer, milometer": 685,
+    "oil filter": 686,
+    "orange": 950,
+    "orangutan, orang, orangutang, Pongo pygmaeus": 365,
+    "organ, pipe organ": 687,
+    "oscilloscope, scope, cathode-ray oscilloscope, CRO": 688,
+    "ostrich, Struthio camelus": 9,
+    "otter": 360,
+    "otterhound, otter hound": 175,
+    "overskirt": 689,
+    "ox": 345,
+    "oxcart": 690,
+    "oxygen mask": 691,
+    "oystercatcher, oyster catcher": 143,
+    "packet": 692,
+    "paddle, boat paddle": 693,
+    "paddlewheel, paddle wheel": 694,
+    "padlock": 695,
+    "paintbrush": 696,
+    "pajama, pyjama, pj's, jammies": 697,
+    "palace": 698,
+    "panpipe, pandean pipe, syrinx": 699,
+    "paper towel": 700,
+    "papillon": 157,
+    "parachute, chute": 701,
+    "parallel bars, bars": 702,
+    "park bench": 703,
+    "parking meter": 704,
+    "partridge": 86,
+    "passenger car, coach, carriage": 705,
+    "patas, hussar monkey, Erythrocebus patas": 371,
+    "patio, terrace": 706,
+    "pay-phone, pay-station": 707,
+    "peacock": 84,
+    "pedestal, plinth, footstall": 708,
+    "pelican": 144,
+    "pencil box, pencil case": 709,
+    "pencil sharpener": 710,
+    "perfume, essence": 711,
+    "photocopier": 713,
+    "pick, plectrum, plectron": 714,
+    "pickelhaube": 715,
+    "picket fence, paling": 716,
+    "pickup, pickup truck": 717,
+    "pier": 718,
+    "piggy bank, penny bank": 719,
+    "pill bottle": 720,
+    "pillow": 721,
+    "pineapple, ananas": 953,
+    "ping-pong ball": 722,
+    "pinwheel": 723,
+    "pirate, pirate ship": 724,
+    "pitcher, ewer": 725,
+    "pizza, pizza pie": 963,
+    "plane, carpenter's plane, woodworking plane": 726,
+    "planetarium": 727,
+    "plastic bag": 728,
+    "plate": 923,
+    "plate rack": 729,
+    "platypus, duckbill, duckbilled platypus, duck-billed platypus, Ornithorhynchus anatinus": 103,
+    "plow, plough": 730,
+    "plunger, plumber's helper": 731,
+    "pole": 733,
+    "polecat, fitch, foulmart, foumart, Mustela putorius": 358,
+    "police van, police wagon, paddy wagon, patrol wagon, wagon, black Maria": 734,
+    "pomegranate": 957,
+    "poncho": 735,
+    "pool table, billiard table, snooker table": 736,
+    "pop bottle, soda bottle": 737,
+    "porcupine, hedgehog": 334,
+    "pot, flowerpot": 738,
+    "potpie": 964,
+    "potter's wheel": 739,
+    "power drill": 740,
+    "prairie chicken, prairie grouse, prairie fowl": 83,
+    "prayer rug, prayer mat": 741,
+    "pretzel": 932,
+    "printer": 742,
+    "prison, prison house": 743,
+    "proboscis monkey, Nasalis larvatus": 376,
+    "projectile, missile": 744,
+    "projector": 745,
+    "promontory, headland, head, foreland": 976,
+    "ptarmigan": 81,
+    "puck, hockey puck": 746,
+    "puffer, pufferfish, blowfish, globefish": 397,
+    "pug, pug-dog": 254,
+    "punching bag, punch bag, punching ball, punchball": 747,
+    "purse": 748,
+    "quail": 85,
+    "quill, quill pen": 749,
+    "quilt, comforter, comfort, puff": 750,
+    "racer, race car, racing car": 751,
+    "racket, racquet": 752,
+    "radiator": 753,
+    "radio telescope, radio reflector": 755,
+    "radio, wireless": 754,
+    "rain barrel": 756,
+    "ram, tup": 348,
+    "rapeseed": 984,
+    "recreational vehicle, RV, R.V.": 757,
+    "red fox, Vulpes vulpes": 277,
+    "red wine": 966,
+    "red wolf, maned wolf, Canis rufus, Canis niger": 271,
+    "red-backed sandpiper, dunlin, Erolia alpina": 140,
+    "red-breasted merganser, Mergus serrator": 98,
+    "redbone": 168,
+    "redshank, Tringa totanus": 141,
+    "reel": 758,
+    "reflex camera": 759,
+    "refrigerator, icebox": 760,
+    "remote control, remote": 761,
+    "restaurant, eating house, eating place, eatery": 762,
+    "revolver, six-gun, six-shooter": 763,
+    "rhinoceros beetle": 306,
+    "rifle": 764,
+    "ringlet, ringlet butterfly": 322,
+    "ringneck snake, ring-necked snake, ring snake": 53,
+    "robin, American robin, Turdus migratorius": 15,
+    "rock beauty, Holocanthus tricolor": 392,
+    "rock crab, Cancer irroratus": 119,
+    "rock python, rock snake, Python sebae": 62,
+    "rocking chair, rocker": 765,
+    "rotisserie": 766,
+    "rubber eraser, rubber, pencil eraser": 767,
+    "ruddy turnstone, Arenaria interpres": 139,
+    "ruffed grouse, partridge, Bonasa umbellus": 82,
+    "rugby ball": 768,
+    "rule, ruler": 769,
+    "running shoe": 770,
+    "safe": 771,
+    "safety pin": 772,
+    "saltshaker, salt shaker": 773,
+    "sandal": 774,
+    "sandbar, sand bar": 977,
+    "sarong": 775,
+    "sax, saxophone": 776,
+    "scabbard": 777,
+    "scale, weighing machine": 778,
+    "schipperke": 223,
+    "school bus": 779,
+    "schooner": 780,
+    "scoreboard": 781,
+    "scorpion": 71,
+    "screen, CRT screen": 782,
+    "screw": 783,
+    "screwdriver": 784,
+    "scuba diver": 983,
+    "sea anemone, anemone": 108,
+    "sea cucumber, holothurian": 329,
+    "sea lion": 150,
+    "sea slug, nudibranch": 115,
+    "sea snake": 65,
+    "sea urchin": 328,
+    "seashore, coast, seacoast, sea-coast": 978,
+    "seat belt, seatbelt": 785,
+    "sewing machine": 786,
+    "shield, buckler": 787,
+    "shoe shop, shoe-shop, shoe store": 788,
+    "shoji": 789,
+    "shopping basket": 790,
+    "shopping cart": 791,
+    "shovel": 792,
+    "shower cap": 793,
+    "shower curtain": 794,
+    "siamang, Hylobates syndactylus, Symphalangus syndactylus": 369,
+    "sidewinder, horned rattlesnake, Crotalus cerastes": 68,
+    "silky terrier, Sydney silky": 201,
+    "ski": 795,
+    "ski mask": 796,
+    "skunk, polecat, wood pussy": 361,
+    "sleeping bag": 797,
+    "slide rule, slipstick": 798,
+    "sliding door": 799,
+    "slot, one-armed bandit": 800,
+    "sloth bear, Melursus ursinus, Ursus ursinus": 297,
+    "slug": 114,
+    "snail": 113,
+    "snorkel": 801,
+    "snow leopard, ounce, Panthera uncia": 289,
+    "snowmobile": 802,
+    "snowplow, snowplough": 803,
+    "soap dispenser": 804,
+    "soccer ball": 805,
+    "sock": 806,
+    "soft-coated wheaten terrier": 202,
+    "solar dish, solar collector, solar furnace": 807,
+    "sombrero": 808,
+    "sorrel": 339,
+    "soup bowl": 809,
+    "space bar": 810,
+    "space heater": 811,
+    "space shuttle": 812,
+    "spaghetti squash": 940,
+    "spatula": 813,
+    "speedboat": 814,
+    "spider monkey, Ateles geoffroyi": 381,
+    "spider web, spider's web": 815,
+    "spindle": 816,
+    "spiny lobster, langouste, rock lobster, crawfish, crayfish, sea crawfish": 123,
+    "spoonbill": 129,
+    "sports car, sport car": 817,
+    "spotlight, spot": 818,
+    "spotted salamander, Ambystoma maculatum": 28,
+    "squirrel monkey, Saimiri sciureus": 382,
+    "stage": 819,
+    "standard poodle": 267,
+    "standard schnauzer": 198,
+    "starfish, sea star": 327,
+    "steam locomotive": 820,
+    "steel arch bridge": 821,
+    "steel drum": 822,
+    "stethoscope": 823,
+    "stingray": 6,
+    "stinkhorn, carrion fungus": 994,
+    "stole": 824,
+    "stone wall": 825,
+    "stopwatch, stop watch": 826,
+    "stove": 827,
+    "strainer": 828,
+    "strawberry": 949,
+    "street sign": 919,
+    "streetcar, tram, tramcar, trolley, trolley car": 829,
+    "stretcher": 830,
+    "studio couch, day bed": 831,
+    "stupa, tope": 832,
+    "sturgeon": 394,
+    "submarine, pigboat, sub, U-boat": 833,
+    "suit, suit of clothes": 834,
+    "sulphur butterfly, sulfur butterfly": 325,
+    "sulphur-crested cockatoo, Kakatoe galerita, Cacatua galerita": 89,
+    "sundial": 835,
+    "sunglass": 836,
+    "sunglasses, dark glasses, shades": 837,
+    "sunscreen, sunblock, sun blocker": 838,
+    "suspension bridge": 839,
+    "swab, swob, mop": 840,
+    "sweatshirt": 841,
+    "swimming trunks, bathing trunks": 842,
+    "swing": 843,
+    "switch, electric switch, electrical switch": 844,
+    "syringe": 845,
+    "tabby, tabby cat": 281,
+    "table lamp": 846,
+    "tailed frog, bell toad, ribbed toad, tailed toad, Ascaphus trui": 32,
+    "tank, army tank, armored combat vehicle, armoured combat vehicle": 847,
+    "tape player": 848,
+    "tarantula": 76,
+    "teapot": 849,
+    "teddy, teddy bear": 850,
+    "television, television system": 851,
+    "tench, Tinca tinca": 0,
+    "tennis ball": 852,
+    "terrapin": 36,
+    "thatch, thatched roof": 853,
+    "theater curtain, theatre curtain": 854,
+    "thimble": 855,
+    "three-toed sloth, ai, Bradypus tridactylus": 364,
+    "thresher, thrasher, threshing machine": 856,
+    "throne": 857,
+    "thunder snake, worm snake, Carphophis amoenus": 52,
+    "tick": 78,
+    "tiger beetle": 300,
+    "tiger cat": 282,
+    "tiger shark, Galeocerdo cuvieri": 3,
+    "tiger, Panthera tigris": 292,
+    "tile roof": 858,
+    "timber wolf, grey wolf, gray wolf, Canis lupus": 269,
+    "titi, titi monkey": 380,
+    "toaster": 859,
+    "tobacco shop, tobacconist shop, tobacconist": 860,
+    "toilet seat": 861,
+    "toilet tissue, toilet paper, bathroom tissue": 999,
+    "torch": 862,
+    "totem pole": 863,
+    "toucan": 96,
+    "tow truck, tow car, wrecker": 864,
+    "toy poodle": 265,
+    "toy terrier": 158,
+    "toyshop": 865,
+    "tractor": 866,
+    "traffic light, traffic signal, stoplight": 920,
+    "trailer truck, tractor trailer, trucking rig, rig, articulated lorry, semi": 867,
+    "tray": 868,
+    "tree frog, tree-frog": 31,
+    "trench coat": 869,
+    "triceratops": 51,
+    "tricycle, trike, velocipede": 870,
+    "trifle": 927,
+    "trilobite": 69,
+    "trimaran": 871,
+    "tripod": 872,
+    "triumphal arch": 873,
+    "trolleybus, trolley coach, trackless trolley": 874,
+    "trombone": 875,
+    "tub, vat": 876,
+    "turnstile": 877,
+    "tusker": 101,
+    "typewriter keyboard": 878,
+    "umbrella": 879,
+    "unicycle, monocycle": 880,
+    "upright, upright piano": 881,
+    "vacuum, vacuum cleaner": 882,
+    "valley, vale": 979,
+    "vase": 883,
+    "vault": 884,
+    "velvet": 885,
+    "vending machine": 886,
+    "vestment": 887,
+    "viaduct": 888,
+    "vine snake": 59,
+    "violin, fiddle": 889,
+    "vizsla, Hungarian pointer": 211,
+    "volcano": 980,
+    "volleyball": 890,
+    "vulture": 23,
+    "waffle iron": 891,
+    "walking stick, walkingstick, stick insect": 313,
+    "wall clock": 892,
+    "wallaby, brush kangaroo": 104,
+    "wallet, billfold, notecase, pocketbook": 893,
+    "wardrobe, closet, press": 894,
+    "warplane, military plane": 895,
+    "warthog": 343,
+    "washbasin, handbasin, washbowl, lavabo, wash-hand basin": 896,
+    "washer, automatic washer, washing machine": 897,
+    "water bottle": 898,
+    "water buffalo, water ox, Asiatic buffalo, Bubalus bubalis": 346,
+    "water jug": 899,
+    "water ouzel, dipper": 20,
+    "water snake": 58,
+    "water tower": 900,
+    "weasel": 356,
+    "web site, website, internet site, site": 916,
+    "weevil": 307,
+    "whippet": 172,
+    "whiptail, whiptail lizard": 41,
+    "whiskey jug": 901,
+    "whistle": 902,
+    "white stork, Ciconia ciconia": 127,
+    "white wolf, Arctic wolf, Canis lupus tundrarum": 270,
+    "wig": 903,
+    "wild boar, boar, Sus scrofa": 342,
+    "window screen": 904,
+    "window shade": 905,
+    "wine bottle": 907,
+    "wing": 908,
+    "wire-haired fox terrier": 188,
+    "wok": 909,
+    "wolf spider, hunting spider": 77,
+    "wombat": 106,
+    "wood rabbit, cottontail, cottontail rabbit": 330,
+    "wooden spoon": 910,
+    "wool, woolen, woollen": 911,
+    "worm fence, snake fence, snake-rail fence, Virginia fence": 912,
+    "wreck": 913,
+    "yawl": 914,
+    "yellow lady's slipper, yellow lady-slipper, Cypripedium calceolus, Cypripedium parviflorum": 986,
+    "yurt": 915,
+    "zebra": 340,
+    "zucchini, courgette": 939
+  },
+  "layer_norm_eps": 1e-12,
+  "model_type": "vit",
+  "num_attention_heads": 12,
+  "num_channels": 3,
+  "num_hidden_layers": 12,
+  "patch_size": 16,
+  "qkv_bias": true,
+  "torch_dtype": "float32",
+  "transformers_version": "4.44.2"
+}

chatbot/fine_tuned_model/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8b6d2fa2eaa34206ba3f12bf21f91a932ed30645c668e95db4512cb9189ca029
+size 343220892

chatbot/fined-tuned-model.lora.h5

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6fc9f1de53fe3d4eee5c536a0d566dafaf1d11d0167c526506bc9d89c7c3ebe3
+size 5560280

chatbot/inventory_log.json

@@ -0,0 +1,91 @@
+{
+    "coffee mug": {
+        "category": "coffee mug",
+        "count": 2,
+        "last_detected": "2024-11-07T12:17:46.882928",
+        "history": [
+            {
+                "timestamp": "2024-11-07T12:15:25.208394",
+                "count": 0
+            },
+            {
+                "timestamp": "2024-11-07T12:17:46.882928",
+                "count": 1
+            }
+        ]
+    },
+    "ladle": {
+        "category": "ladle",
+        "count": 1,
+        "last_detected": "2024-11-07T12:15:25.315552",
+        "history": [
+            {
+                "timestamp": "2024-11-07T12:15:25.315552",
+                "count": 0
+            }
+        ]
+    },
+    "wool, woolen, woollen": {
+        "category": "wool, woolen, woollen",
+        "count": 2,
+        "last_detected": "2024-11-07T12:17:46.998784",
+        "history": [
+            {
+                "timestamp": "2024-11-07T12:15:25.412992",
+                "count": 0
+            },
+            {
+                "timestamp": "2024-11-07T12:17:46.998784",
+                "count": 1
+            }
+        ]
+    },
+    "Windsor tie": {
+        "category": "Windsor tie",
+        "count": 1,
+        "last_detected": "2024-11-07T12:16:27.597227",
+        "history": [
+            {
+                "timestamp": "2024-11-07T12:16:27.597227",
+                "count": 0
+            }
+        ]
+    },
+    "ballplayer, baseball player": {
+        "category": "ballplayer, baseball player",
+        "count": 1,
+        "last_detected": "2024-11-07T12:16:27.646567",
+        "history": [
+            {
+                "timestamp": "2024-11-07T12:16:27.646567",
+                "count": 0
+            }
+        ]
+    },
+    "analog clock": {
+        "category": "analog clock",
+        "count": 1,
+        "last_detected": "2024-11-07T12:17:47.055052",
+        "history": [
+            {
+                "timestamp": "2024-11-07T12:17:47.055052",
+                "count": 0
+            }
+        ]
+    },
+    "dining table, board": {
+        "category": "dining table, board",
+        "count": 2,
+        "last_detected": "2024-11-07T12:17:47.188026",
+        "history": [
+            {
+                "timestamp": "2024-11-07T12:17:47.120347",
+                "count": 0
+            },
+            {
+                "timestamp": "2024-11-07T12:17:47.188026",
+                "count": 1
+            }
+        ]
+    }
+}

chatbot/main.py.save

@@ -0,0 +1,52 @@
+import gradio as grimport os
+import keras
+import keras_nlp
+
+import os
+
+os.environ["KERAS_BACKEND"] = "jax"  
+# Avoid memory fragmentation on JAX backend.
+os.environ["XLA_PYTHON_CLIENT_MEM_FRACTION"]="1.00"
+
+import os
+
+# Set Kaggle API credentials
+os.environ["KAGGLE_USERNAME"] = "rogerkorantenng"
+os.environ["KAGGLE_KEY"] = "9a33b6e88bcb6058b1281d777fa6808d"
+
+# Load environment variables
+load_dotenv()
+
+# Replace this with the path or method to load your local model
+gemma_lm = keras_nlp.models.GemmaCausalLM.from_preset("gemma_2b_en")
+
+def generate_response(message, history):
+    # Format the conversation history for the local model
+    formatted_history = []
+    for user, assistant in history:
+        formatted_history.append(f"Instruction:\n{user}\n\nResponse:\n{assistant}")
+
+    # Add the latest user message to the history
+    formatted_history.append(f"Instruction:\n{message}\n\nResponse:\n")
+
+    # Join formatted history into a single string for input
+    input_text = "\n".join(formatted_history)
+
+    # Generate response from the local model
+    # Make sure to adjust this part according to your model's API
+    response = gemma_lm.generate(input_text, max_length=256)
+    
+    # Extract the response text
+    # Adjust the response extraction based on the actual structure of your model's output
+    return response[0]  # Change this line if necessary
+
+# Create the Gradio interface
+gr.ChatInterface(
+    generate_response,
+    chatbot=gr.Chatbot(height=300),
+    textbox=gr.Textbox(placeholder="You can ask me anything", container=False, scale=7),
+    title="Local Model Chat Bot",
+    retry_btn=None,
+    undo_btn="Delete Previous",
+    clear_btn="Clear"
+).launch(share=True)

chatbot/main2.py

@@ -0,0 +1,48 @@
+import gradio as gr
+import os
+import keras_nlp
+from transformers import AutoModelForCausalLM
+
+# Set Kaggle API credentials using values from environment variables
+os.environ["KAGGLE_USERNAME"] = os.environ.get("KAGGLE_USERNAME")
+os.environ["KAGGLE_KEY"] = os.environ.get("KAGGLE_KEY")
+
+
+# Load LoRA weights if you have them
+LoRA_weights_path = "fined-tuned-model.lora.h5"
+gemma_lm = keras_nlp.models.GemmaCausalLM.from_preset("gemma_2b_en")
+gemma_lm.backbone.enable_lora(rank=4)  # Enable LoRA with rank 4
+gemma_lm.preprocessor.sequence_length = 512  # Limit sequence length
+gemma_lm.backbone.load_lora_weights(LoRA_weights_path)  # Load LoRA weights
+
+# Define the response generation function
+def generate_response(message):
+    # Create a prompt template
+    template = "Instruction:\n{instruction}\n\nResponse:\n{response}"
+
+    # Create the prompt with the current message
+    prompt = template.format(instruction=message, response="")
+    print("Prompt:\n", prompt)
+
+    # Generate response from the model
+    response = gemma_lm.generate(prompt, max_length=256)
+    # Only keep the generated response
+    response = response.split("Response:")[-1].strip()
+
+    print("Generated Response:\n", response)
+
+    # Extract and return the generated response text
+    return response  # Adjust this if your model's output structure differs
+
+# Create the Gradio chat interface
+interface = gr.Interface(
+    fn=generate_response,  # Function that generates responses
+    inputs=gr.Textbox(placeholder="Hello, I am Sage, your mental health advisor", lines=2, scale=7),
+    outputs=gr.Textbox(),
+    title="Sage, your Mental Health Advisor",
+#     description="Chat with Sage, your mental health advisor.",
+#     live=True
+)
+proxy_prefix = os.environ.get("PROXY_PREFIX")
+# Launch the Gradio app
+interface.launch(server_name="0.0.0.0", server_port=8080, root_path=proxy_prefix, share=True)

chatbot/model_config.json

@@ -0,0 +1,27 @@
+{
+    "gpt-4": {
+        "endpoint": "https://roger-m38jr9pd-eastus2.openai.azure.com/openai/deployments/gpt-4/chat/completions?api-version=2024-08-01-preview",
+        "api_key": "XTtd4wdC0mlEdbQqp0KWOe9aWrlg4z4HurxXlgBf5ASUvrB9iPkwJQQJ99AKACHYHv6XJ3w3AAABACOGaeSg",
+        "model_path": null
+    },
+    "gpt-4o": {
+        "endpoint": "https://roger-m38jr9pd-eastus2.openai.azure.com/openai/deployments/gpt-4o/chat/completions?api-version=2024-08-01-preview",
+        "api_key": "XTtd4wdC0mlEdbQqp0KWOe9aWrlg4z4HurxXlgBf5ASUvrB9iPkwJQQJ99AKACHYHv6XJ3w3AAABACOGaeSg",
+        "model_path": null
+    },
+    "gpt-35-turbo": {
+        "endpoint": "https://rogerkoranteng.openai.azure.com/openai/deployments/gpt-35-turbo/chat/completions?api-version=2024-08-01-preview",
+        "api_key": "cfe75d17d47c40318db6230d2677d84f",
+        "model_path": null
+    },
+    "gpt-4-32k": {
+        "endpoint": "https://roger-m38orjxq-australiaeast.openai.azure.com/openai/deployments/gpt-4-32k/chat/completions?api-version=2024-08-01-preview",
+        "api_key": "AebO97c0w0lnrppFNpbuPVueKth7HfGOdMbKeaX5R2Uql6wvJA5qJQQJ99AKACL93NaXJ3w3AAABACOGh8MD",
+        "model_path": null
+    },
+    "microsoft/Phi-3.5-mini-instruct": {
+        "endpoint": null,
+        "model_path": "./models/microsoft/Phi-3.5-mini-instruct",
+        "api_key": null
+    }
+}

chatbot/models/500/config.json

@@ -0,0 +1,2026 @@
+{
+  "_name_or_path": "google/vit-base-patch16-224",
+  "architectures": [
+    "ViTForImageClassification"
+  ],
+  "attention_probs_dropout_prob": 0.0,
+  "encoder_stride": 16,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.0,
+  "hidden_size": 768,
+  "id2label": {
+    "0": "tench, Tinca tinca",
+    "1": "goldfish, Carassius auratus",
+    "2": "great white shark, white shark, man-eater, man-eating shark, Carcharodon carcharias",
+    "3": "tiger shark, Galeocerdo cuvieri",
+    "4": "hammerhead, hammerhead shark",
+    "5": "electric ray, crampfish, numbfish, torpedo",
+    "6": "stingray",
+    "7": "cock",
+    "8": "hen",
+    "9": "ostrich, Struthio camelus",
+    "10": "brambling, Fringilla montifringilla",
+    "11": "goldfinch, Carduelis carduelis",
+    "12": "house finch, linnet, Carpodacus mexicanus",
+    "13": "junco, snowbird",
+    "14": "indigo bunting, indigo finch, indigo bird, Passerina cyanea",
+    "15": "robin, American robin, Turdus migratorius",
+    "16": "bulbul",
+    "17": "jay",
+    "18": "magpie",
+    "19": "chickadee",
+    "20": "water ouzel, dipper",
+    "21": "kite",
+    "22": "bald eagle, American eagle, Haliaeetus leucocephalus",
+    "23": "vulture",
+    "24": "great grey owl, great gray owl, Strix nebulosa",
+    "25": "European fire salamander, Salamandra salamandra",
+    "26": "common newt, Triturus vulgaris",
+    "27": "eft",
+    "28": "spotted salamander, Ambystoma maculatum",
+    "29": "axolotl, mud puppy, Ambystoma mexicanum",
+    "30": "bullfrog, Rana catesbeiana",
+    "31": "tree frog, tree-frog",
+    "32": "tailed frog, bell toad, ribbed toad, tailed toad, Ascaphus trui",
+    "33": "loggerhead, loggerhead turtle, Caretta caretta",
+    "34": "leatherback turtle, leatherback, leathery turtle, Dermochelys coriacea",
+    "35": "mud turtle",
+    "36": "terrapin",
+    "37": "box turtle, box tortoise",
+    "38": "banded gecko",
+    "39": "common iguana, iguana, Iguana iguana",
+    "40": "American chameleon, anole, Anolis carolinensis",
+    "41": "whiptail, whiptail lizard",
+    "42": "agama",
+    "43": "frilled lizard, Chlamydosaurus kingi",
+    "44": "alligator lizard",
+    "45": "Gila monster, Heloderma suspectum",
+    "46": "green lizard, Lacerta viridis",
+    "47": "African chameleon, Chamaeleo chamaeleon",
+    "48": "Komodo dragon, Komodo lizard, dragon lizard, giant lizard, Varanus komodoensis",
+    "49": "African crocodile, Nile crocodile, Crocodylus niloticus",
+    "50": "American alligator, Alligator mississipiensis",
+    "51": "triceratops",
+    "52": "thunder snake, worm snake, Carphophis amoenus",
+    "53": "ringneck snake, ring-necked snake, ring snake",
+    "54": "hognose snake, puff adder, sand viper",
+    "55": "green snake, grass snake",
+    "56": "king snake, kingsnake",
+    "57": "garter snake, grass snake",
+    "58": "water snake",
+    "59": "vine snake",
+    "60": "night snake, Hypsiglena torquata",
+    "61": "boa constrictor, Constrictor constrictor",
+    "62": "rock python, rock snake, Python sebae",
+    "63": "Indian cobra, Naja naja",
+    "64": "green mamba",
+    "65": "sea snake",
+    "66": "horned viper, cerastes, sand viper, horned asp, Cerastes cornutus",
+    "67": "diamondback, diamondback rattlesnake, Crotalus adamanteus",
+    "68": "sidewinder, horned rattlesnake, Crotalus cerastes",
+    "69": "trilobite",
+    "70": "harvestman, daddy longlegs, Phalangium opilio",
+    "71": "scorpion",
+    "72": "black and gold garden spider, Argiope aurantia",
+    "73": "barn spider, Araneus cavaticus",
+    "74": "garden spider, Aranea diademata",
+    "75": "black widow, Latrodectus mactans",
+    "76": "tarantula",
+    "77": "wolf spider, hunting spider",
+    "78": "tick",
+    "79": "centipede",
+    "80": "black grouse",
+    "81": "ptarmigan",
+    "82": "ruffed grouse, partridge, Bonasa umbellus",
+    "83": "prairie chicken, prairie grouse, prairie fowl",
+    "84": "peacock",
+    "85": "quail",
+    "86": "partridge",
+    "87": "African grey, African gray, Psittacus erithacus",
+    "88": "macaw",
+    "89": "sulphur-crested cockatoo, Kakatoe galerita, Cacatua galerita",
+    "90": "lorikeet",
+    "91": "coucal",
+    "92": "bee eater",
+    "93": "hornbill",
+    "94": "hummingbird",
+    "95": "jacamar",
+    "96": "toucan",
+    "97": "drake",
+    "98": "red-breasted merganser, Mergus serrator",
+    "99": "goose",
+    "100": "black swan, Cygnus atratus",
+    "101": "tusker",
+    "102": "echidna, spiny anteater, anteater",
+    "103": "platypus, duckbill, duckbilled platypus, duck-billed platypus, Ornithorhynchus anatinus",
+    "104": "wallaby, brush kangaroo",
+    "105": "koala, koala bear, kangaroo bear, native bear, Phascolarctos cinereus",
+    "106": "wombat",
+    "107": "jellyfish",
+    "108": "sea anemone, anemone",
+    "109": "brain coral",
+    "110": "flatworm, platyhelminth",
+    "111": "nematode, nematode worm, roundworm",
+    "112": "conch",
+    "113": "snail",
+    "114": "slug",
+    "115": "sea slug, nudibranch",
+    "116": "chiton, coat-of-mail shell, sea cradle, polyplacophore",
+    "117": "chambered nautilus, pearly nautilus, nautilus",
+    "118": "Dungeness crab, Cancer magister",
+    "119": "rock crab, Cancer irroratus",
+    "120": "fiddler crab",
+    "121": "king crab, Alaska crab, Alaskan king crab, Alaska king crab, Paralithodes camtschatica",
+    "122": "American lobster, Northern lobster, Maine lobster, Homarus americanus",
+    "123": "spiny lobster, langouste, rock lobster, crawfish, crayfish, sea crawfish",
+    "124": "crayfish, crawfish, crawdad, crawdaddy",
+    "125": "hermit crab",
+    "126": "isopod",
+    "127": "white stork, Ciconia ciconia",
+    "128": "black stork, Ciconia nigra",
+    "129": "spoonbill",
+    "130": "flamingo",
+    "131": "little blue heron, Egretta caerulea",
+    "132": "American egret, great white heron, Egretta albus",
+    "133": "bittern",
+    "134": "crane",
+    "135": "limpkin, Aramus pictus",
+    "136": "European gallinule, Porphyrio porphyrio",
+    "137": "American coot, marsh hen, mud hen, water hen, Fulica americana",
+    "138": "bustard",
+    "139": "ruddy turnstone, Arenaria interpres",
+    "140": "red-backed sandpiper, dunlin, Erolia alpina",
+    "141": "redshank, Tringa totanus",
+    "142": "dowitcher",
+    "143": "oystercatcher, oyster catcher",
+    "144": "pelican",
+    "145": "king penguin, Aptenodytes patagonica",
+    "146": "albatross, mollymawk",
+    "147": "grey whale, gray whale, devilfish, Eschrichtius gibbosus, Eschrichtius robustus",
+    "148": "killer whale, killer, orca, grampus, sea wolf, Orcinus orca",
+    "149": "dugong, Dugong dugon",
+    "150": "sea lion",
+    "151": "Chihuahua",
+    "152": "Japanese spaniel",
+    "153": "Maltese dog, Maltese terrier, Maltese",
+    "154": "Pekinese, Pekingese, Peke",
+    "155": "Shih-Tzu",
+    "156": "Blenheim spaniel",
+    "157": "papillon",
+    "158": "toy terrier",
+    "159": "Rhodesian ridgeback",
+    "160": "Afghan hound, Afghan",
+    "161": "basset, basset hound",
+    "162": "beagle",
+    "163": "bloodhound, sleuthhound",
+    "164": "bluetick",
+    "165": "black-and-tan coonhound",
+    "166": "Walker hound, Walker foxhound",
+    "167": "English foxhound",
+    "168": "redbone",
+    "169": "borzoi, Russian wolfhound",
+    "170": "Irish wolfhound",
+    "171": "Italian greyhound",
+    "172": "whippet",
+    "173": "Ibizan hound, Ibizan Podenco",
+    "174": "Norwegian elkhound, elkhound",
+    "175": "otterhound, otter hound",
+    "176": "Saluki, gazelle hound",
+    "177": "Scottish deerhound, deerhound",
+    "178": "Weimaraner",
+    "179": "Staffordshire bullterrier, Staffordshire bull terrier",
+    "180": "American Staffordshire terrier, Staffordshire terrier, American pit bull terrier, pit bull terrier",
+    "181": "Bedlington terrier",
+    "182": "Border terrier",
+    "183": "Kerry blue terrier",
+    "184": "Irish terrier",
+    "185": "Norfolk terrier",
+    "186": "Norwich terrier",
+    "187": "Yorkshire terrier",
+    "188": "wire-haired fox terrier",
+    "189": "Lakeland terrier",
+    "190": "Sealyham terrier, Sealyham",
+    "191": "Airedale, Airedale terrier",
+    "192": "cairn, cairn terrier",
+    "193": "Australian terrier",
+    "194": "Dandie Dinmont, Dandie Dinmont terrier",
+    "195": "Boston bull, Boston terrier",
+    "196": "miniature schnauzer",
+    "197": "giant schnauzer",
+    "198": "standard schnauzer",
+    "199": "Scotch terrier, Scottish terrier, Scottie",
+    "200": "Tibetan terrier, chrysanthemum dog",
+    "201": "silky terrier, Sydney silky",
+    "202": "soft-coated wheaten terrier",
+    "203": "West Highland white terrier",
+    "204": "Lhasa, Lhasa apso",
+    "205": "flat-coated retriever",
+    "206": "curly-coated retriever",
+    "207": "golden retriever",
+    "208": "Labrador retriever",
+    "209": "Chesapeake Bay retriever",
+    "210": "German short-haired pointer",
+    "211": "vizsla, Hungarian pointer",
+    "212": "English setter",
+    "213": "Irish setter, red setter",
+    "214": "Gordon setter",
+    "215": "Brittany spaniel",
+    "216": "clumber, clumber spaniel",
+    "217": "English springer, English springer spaniel",
+    "218": "Welsh springer spaniel",
+    "219": "cocker spaniel, English cocker spaniel, cocker",
+    "220": "Sussex spaniel",
+    "221": "Irish water spaniel",
+    "222": "kuvasz",
+    "223": "schipperke",
+    "224": "groenendael",
+    "225": "malinois",
+    "226": "briard",
+    "227": "kelpie",
+    "228": "komondor",
+    "229": "Old English sheepdog, bobtail",
+    "230": "Shetland sheepdog, Shetland sheep dog, Shetland",
+    "231": "collie",
+    "232": "Border collie",
+    "233": "Bouvier des Flandres, Bouviers des Flandres",
+    "234": "Rottweiler",
+    "235": "German shepherd, German shepherd dog, German police dog, alsatian",
+    "236": "Doberman, Doberman pinscher",
+    "237": "miniature pinscher",
+    "238": "Greater Swiss Mountain dog",
+    "239": "Bernese mountain dog",
+    "240": "Appenzeller",
+    "241": "EntleBucher",
+    "242": "boxer",
+    "243": "bull mastiff",
+    "244": "Tibetan mastiff",
+    "245": "French bulldog",
+    "246": "Great Dane",
+    "247": "Saint Bernard, St Bernard",
+    "248": "Eskimo dog, husky",
+    "249": "malamute, malemute, Alaskan malamute",
+    "250": "Siberian husky",
+    "251": "dalmatian, coach dog, carriage dog",
+    "252": "affenpinscher, monkey pinscher, monkey dog",
+    "253": "basenji",
+    "254": "pug, pug-dog",
+    "255": "Leonberg",
+    "256": "Newfoundland, Newfoundland dog",
+    "257": "Great Pyrenees",
+    "258": "Samoyed, Samoyede",
+    "259": "Pomeranian",
+    "260": "chow, chow chow",
+    "261": "keeshond",
+    "262": "Brabancon griffon",
+    "263": "Pembroke, Pembroke Welsh corgi",
+    "264": "Cardigan, Cardigan Welsh corgi",
+    "265": "toy poodle",
+    "266": "miniature poodle",
+    "267": "standard poodle",
+    "268": "Mexican hairless",
+    "269": "timber wolf, grey wolf, gray wolf, Canis lupus",
+    "270": "white wolf, Arctic wolf, Canis lupus tundrarum",
+    "271": "red wolf, maned wolf, Canis rufus, Canis niger",
+    "272": "coyote, prairie wolf, brush wolf, Canis latrans",
+    "273": "dingo, warrigal, warragal, Canis dingo",
+    "274": "dhole, Cuon alpinus",
+    "275": "African hunting dog, hyena dog, Cape hunting dog, Lycaon pictus",
+    "276": "hyena, hyaena",
+    "277": "red fox, Vulpes vulpes",
+    "278": "kit fox, Vulpes macrotis",
+    "279": "Arctic fox, white fox, Alopex lagopus",
+    "280": "grey fox, gray fox, Urocyon cinereoargenteus",
+    "281": "tabby, tabby cat",
+    "282": "tiger cat",
+    "283": "Persian cat",
+    "284": "Siamese cat, Siamese",
+    "285": "Egyptian cat",
+    "286": "cougar, puma, catamount, mountain lion, painter, panther, Felis concolor",
+    "287": "lynx, catamount",
+    "288": "leopard, Panthera pardus",
+    "289": "snow leopard, ounce, Panthera uncia",
+    "290": "jaguar, panther, Panthera onca, Felis onca",
+    "291": "lion, king of beasts, Panthera leo",
+    "292": "tiger, Panthera tigris",
+    "293": "cheetah, chetah, Acinonyx jubatus",
+    "294": "brown bear, bruin, Ursus arctos",
+    "295": "American black bear, black bear, Ursus americanus, Euarctos americanus",
+    "296": "ice bear, polar bear, Ursus Maritimus, Thalarctos maritimus",
+    "297": "sloth bear, Melursus ursinus, Ursus ursinus",
+    "298": "mongoose",
+    "299": "meerkat, mierkat",
+    "300": "tiger beetle",
+    "301": "ladybug, ladybeetle, lady beetle, ladybird, ladybird beetle",
+    "302": "ground beetle, carabid beetle",
+    "303": "long-horned beetle, longicorn, longicorn beetle",
+    "304": "leaf beetle, chrysomelid",
+    "305": "dung beetle",
+    "306": "rhinoceros beetle",
+    "307": "weevil",
+    "308": "fly",
+    "309": "bee",
+    "310": "ant, emmet, pismire",
+    "311": "grasshopper, hopper",
+    "312": "cricket",
+    "313": "walking stick, walkingstick, stick insect",
+    "314": "cockroach, roach",
+    "315": "mantis, mantid",
+    "316": "cicada, cicala",
+    "317": "leafhopper",
+    "318": "lacewing, lacewing fly",
+    "319": "dragonfly, darning needle, devil's darning needle, sewing needle, snake feeder, snake doctor, mosquito hawk, skeeter hawk",
+    "320": "damselfly",
+    "321": "admiral",
+    "322": "ringlet, ringlet butterfly",
+    "323": "monarch, monarch butterfly, milkweed butterfly, Danaus plexippus",
+    "324": "cabbage butterfly",
+    "325": "sulphur butterfly, sulfur butterfly",
+    "326": "lycaenid, lycaenid butterfly",
+    "327": "starfish, sea star",
+    "328": "sea urchin",
+    "329": "sea cucumber, holothurian",
+    "330": "wood rabbit, cottontail, cottontail rabbit",
+    "331": "hare",
+    "332": "Angora, Angora rabbit",
+    "333": "hamster",
+    "334": "porcupine, hedgehog",
+    "335": "fox squirrel, eastern fox squirrel, Sciurus niger",
+    "336": "marmot",
+    "337": "beaver",
+    "338": "guinea pig, Cavia cobaya",
+    "339": "sorrel",
+    "340": "zebra",
+    "341": "hog, pig, grunter, squealer, Sus scrofa",
+    "342": "wild boar, boar, Sus scrofa",
+    "343": "warthog",
+    "344": "hippopotamus, hippo, river horse, Hippopotamus amphibius",
+    "345": "ox",
+    "346": "water buffalo, water ox, Asiatic buffalo, Bubalus bubalis",
+    "347": "bison",
+    "348": "ram, tup",
+    "349": "bighorn, bighorn sheep, cimarron, Rocky Mountain bighorn, Rocky Mountain sheep, Ovis canadensis",
+    "350": "ibex, Capra ibex",
+    "351": "hartebeest",
+    "352": "impala, Aepyceros melampus",
+    "353": "gazelle",
+    "354": "Arabian camel, dromedary, Camelus dromedarius",
+    "355": "llama",
+    "356": "weasel",
+    "357": "mink",
+    "358": "polecat, fitch, foulmart, foumart, Mustela putorius",
+    "359": "black-footed ferret, ferret, Mustela nigripes",
+    "360": "otter",
+    "361": "skunk, polecat, wood pussy",
+    "362": "badger",
+    "363": "armadillo",
+    "364": "three-toed sloth, ai, Bradypus tridactylus",
+    "365": "orangutan, orang, orangutang, Pongo pygmaeus",
+    "366": "gorilla, Gorilla gorilla",
+    "367": "chimpanzee, chimp, Pan troglodytes",
+    "368": "gibbon, Hylobates lar",
+    "369": "siamang, Hylobates syndactylus, Symphalangus syndactylus",
+    "370": "guenon, guenon monkey",
+    "371": "patas, hussar monkey, Erythrocebus patas",
+    "372": "baboon",
+    "373": "macaque",
+    "374": "langur",
+    "375": "colobus, colobus monkey",
+    "376": "proboscis monkey, Nasalis larvatus",
+    "377": "marmoset",
+    "378": "capuchin, ringtail, Cebus capucinus",
+    "379": "howler monkey, howler",
+    "380": "titi, titi monkey",
+    "381": "spider monkey, Ateles geoffroyi",
+    "382": "squirrel monkey, Saimiri sciureus",
+    "383": "Madagascar cat, ring-tailed lemur, Lemur catta",
+    "384": "indri, indris, Indri indri, Indri brevicaudatus",
+    "385": "Indian elephant, Elephas maximus",
+    "386": "African elephant, Loxodonta africana",
+    "387": "lesser panda, red panda, panda, bear cat, cat bear, Ailurus fulgens",
+    "388": "giant panda, panda, panda bear, coon bear, Ailuropoda melanoleuca",
+    "389": "barracouta, snoek",
+    "390": "eel",
+    "391": "coho, cohoe, coho salmon, blue jack, silver salmon, Oncorhynchus kisutch",
+    "392": "rock beauty, Holocanthus tricolor",
+    "393": "anemone fish",
+    "394": "sturgeon",
+    "395": "gar, garfish, garpike, billfish, Lepisosteus osseus",
+    "396": "lionfish",
+    "397": "puffer, pufferfish, blowfish, globefish",
+    "398": "abacus",
+    "399": "abaya",
+    "400": "academic gown, academic robe, judge's robe",
+    "401": "accordion, piano accordion, squeeze box",
+    "402": "acoustic guitar",
+    "403": "aircraft carrier, carrier, flattop, attack aircraft carrier",
+    "404": "airliner",
+    "405": "airship, dirigible",
+    "406": "altar",
+    "407": "ambulance",
+    "408": "amphibian, amphibious vehicle",
+    "409": "analog clock",
+    "410": "apiary, bee house",
+    "411": "apron",
+    "412": "ashcan, trash can, garbage can, wastebin, ash bin, ash-bin, ashbin, dustbin, trash barrel, trash bin",
+    "413": "assault rifle, assault gun",
+    "414": "backpack, back pack, knapsack, packsack, rucksack, haversack",
+    "415": "bakery, bakeshop, bakehouse",
+    "416": "balance beam, beam",
+    "417": "balloon",
+    "418": "ballpoint, ballpoint pen, ballpen, Biro",
+    "419": "Band Aid",
+    "420": "banjo",
+    "421": "bannister, banister, balustrade, balusters, handrail",
+    "422": "barbell",
+    "423": "barber chair",
+    "424": "barbershop",
+    "425": "barn",
+    "426": "barometer",
+    "427": "barrel, cask",
+    "428": "barrow, garden cart, lawn cart, wheelbarrow",
+    "429": "baseball",
+    "430": "basketball",
+    "431": "bassinet",
+    "432": "bassoon",
+    "433": "bathing cap, swimming cap",
+    "434": "bath towel",
+    "435": "bathtub, bathing tub, bath, tub",
+    "436": "beach wagon, station wagon, wagon, estate car, beach waggon, station waggon, waggon",
+    "437": "beacon, lighthouse, beacon light, pharos",
+    "438": "beaker",
+    "439": "bearskin, busby, shako",
+    "440": "beer bottle",
+    "441": "beer glass",
+    "442": "bell cote, bell cot",
+    "443": "bib",
+    "444": "bicycle-built-for-two, tandem bicycle, tandem",
+    "445": "bikini, two-piece",
+    "446": "binder, ring-binder",
+    "447": "binoculars, field glasses, opera glasses",
+    "448": "birdhouse",
+    "449": "boathouse",
+    "450": "bobsled, bobsleigh, bob",
+    "451": "bolo tie, bolo, bola tie, bola",
+    "452": "bonnet, poke bonnet",
+    "453": "bookcase",
+    "454": "bookshop, bookstore, bookstall",
+    "455": "bottlecap",
+    "456": "bow",
+    "457": "bow tie, bow-tie, bowtie",
+    "458": "brass, memorial tablet, plaque",
+    "459": "brassiere, bra, bandeau",
+    "460": "breakwater, groin, groyne, mole, bulwark, seawall, jetty",
+    "461": "breastplate, aegis, egis",
+    "462": "broom",
+    "463": "bucket, pail",
+    "464": "buckle",
+    "465": "bulletproof vest",
+    "466": "bullet train, bullet",
+    "467": "butcher shop, meat market",
+    "468": "cab, hack, taxi, taxicab",
+    "469": "caldron, cauldron",
+    "470": "candle, taper, wax light",
+    "471": "cannon",
+    "472": "canoe",
+    "473": "can opener, tin opener",
+    "474": "cardigan",
+    "475": "car mirror",
+    "476": "carousel, carrousel, merry-go-round, roundabout, whirligig",
+    "477": "carpenter's kit, tool kit",
+    "478": "carton",
+    "479": "car wheel",
+    "480": "cash machine, cash dispenser, automated teller machine, automatic teller machine, automated teller, automatic teller, ATM",
+    "481": "cassette",
+    "482": "cassette player",
+    "483": "castle",
+    "484": "catamaran",
+    "485": "CD player",
+    "486": "cello, violoncello",
+    "487": "cellular telephone, cellular phone, cellphone, cell, mobile phone",
+    "488": "chain",
+    "489": "chainlink fence",
+    "490": "chain mail, ring mail, mail, chain armor, chain armour, ring armor, ring armour",
+    "491": "chain saw, chainsaw",
+    "492": "chest",
+    "493": "chiffonier, commode",
+    "494": "chime, bell, gong",
+    "495": "china cabinet, china closet",
+    "496": "Christmas stocking",
+    "497": "church, church building",
+    "498": "cinema, movie theater, movie theatre, movie house, picture palace",
+    "499": "cleaver, meat cleaver, chopper",
+    "500": "cliff dwelling",
+    "501": "cloak",
+    "502": "clog, geta, patten, sabot",
+    "503": "cocktail shaker",
+    "504": "coffee mug",
+    "505": "coffeepot",
+    "506": "coil, spiral, volute, whorl, helix",
+    "507": "combination lock",
+    "508": "computer keyboard, keypad",
+    "509": "confectionery, confectionary, candy store",
+    "510": "container ship, containership, container vessel",
+    "511": "convertible",
+    "512": "corkscrew, bottle screw",
+    "513": "cornet, horn, trumpet, trump",
+    "514": "cowboy boot",
+    "515": "cowboy hat, ten-gallon hat",
+    "516": "cradle",
+    "517": "crane",
+    "518": "crash helmet",
+    "519": "crate",
+    "520": "crib, cot",
+    "521": "Crock Pot",
+    "522": "croquet ball",
+    "523": "crutch",
+    "524": "cuirass",
+    "525": "dam, dike, dyke",
+    "526": "desk",
+    "527": "desktop computer",
+    "528": "dial telephone, dial phone",
+    "529": "diaper, nappy, napkin",
+    "530": "digital clock",
+    "531": "digital watch",
+    "532": "dining table, board",
+    "533": "dishrag, dishcloth",
+    "534": "dishwasher, dish washer, dishwashing machine",
+    "535": "disk brake, disc brake",
+    "536": "dock, dockage, docking facility",
+    "537": "dogsled, dog sled, dog sleigh",
+    "538": "dome",
+    "539": "doormat, welcome mat",
+    "540": "drilling platform, offshore rig",
+    "541": "drum, membranophone, tympan",
+    "542": "drumstick",
+    "543": "dumbbell",
+    "544": "Dutch oven",
+    "545": "electric fan, blower",
+    "546": "electric guitar",
+    "547": "electric locomotive",
+    "548": "entertainment center",
+    "549": "envelope",
+    "550": "espresso maker",
+    "551": "face powder",
+    "552": "feather boa, boa",
+    "553": "file, file cabinet, filing cabinet",
+    "554": "fireboat",
+    "555": "fire engine, fire truck",
+    "556": "fire screen, fireguard",
+    "557": "flagpole, flagstaff",
+    "558": "flute, transverse flute",
+    "559": "folding chair",
+    "560": "football helmet",
+    "561": "forklift",
+    "562": "fountain",
+    "563": "fountain pen",
+    "564": "four-poster",
+    "565": "freight car",
+    "566": "French horn, horn",
+    "567": "frying pan, frypan, skillet",
+    "568": "fur coat",
+    "569": "garbage truck, dustcart",
+    "570": "gasmask, respirator, gas helmet",
+    "571": "gas pump, gasoline pump, petrol pump, island dispenser",
+    "572": "goblet",
+    "573": "go-kart",
+    "574": "golf ball",
+    "575": "golfcart, golf cart",
+    "576": "gondola",
+    "577": "gong, tam-tam",
+    "578": "gown",
+    "579": "grand piano, grand",
+    "580": "greenhouse, nursery, glasshouse",
+    "581": "grille, radiator grille",
+    "582": "grocery store, grocery, food market, market",
+    "583": "guillotine",
+    "584": "hair slide",
+    "585": "hair spray",
+    "586": "half track",
+    "587": "hammer",
+    "588": "hamper",
+    "589": "hand blower, blow dryer, blow drier, hair dryer, hair drier",
+    "590": "hand-held computer, hand-held microcomputer",
+    "591": "handkerchief, hankie, hanky, hankey",
+    "592": "hard disc, hard disk, fixed disk",
+    "593": "harmonica, mouth organ, harp, mouth harp",
+    "594": "harp",
+    "595": "harvester, reaper",
+    "596": "hatchet",
+    "597": "holster",
+    "598": "home theater, home theatre",
+    "599": "honeycomb",
+    "600": "hook, claw",
+    "601": "hoopskirt, crinoline",
+    "602": "horizontal bar, high bar",
+    "603": "horse cart, horse-cart",
+    "604": "hourglass",
+    "605": "iPod",
+    "606": "iron, smoothing iron",
+    "607": "jack-o'-lantern",
+    "608": "jean, blue jean, denim",
+    "609": "jeep, landrover",
+    "610": "jersey, T-shirt, tee shirt",
+    "611": "jigsaw puzzle",
+    "612": "jinrikisha, ricksha, rickshaw",
+    "613": "joystick",
+    "614": "kimono",
+    "615": "knee pad",
+    "616": "knot",
+    "617": "lab coat, laboratory coat",
+    "618": "ladle",
+    "619": "lampshade, lamp shade",
+    "620": "laptop, laptop computer",
+    "621": "lawn mower, mower",
+    "622": "lens cap, lens cover",
+    "623": "letter opener, paper knife, paperknife",
+    "624": "library",
+    "625": "lifeboat",
+    "626": "lighter, light, igniter, ignitor",
+    "627": "limousine, limo",
+    "628": "liner, ocean liner",
+    "629": "lipstick, lip rouge",
+    "630": "Loafer",
+    "631": "lotion",
+    "632": "loudspeaker, speaker, speaker unit, loudspeaker system, speaker system",
+    "633": "loupe, jeweler's loupe",
+    "634": "lumbermill, sawmill",
+    "635": "magnetic compass",
+    "636": "mailbag, postbag",
+    "637": "mailbox, letter box",
+    "638": "maillot",
+    "639": "maillot, tank suit",
+    "640": "manhole cover",
+    "641": "maraca",
+    "642": "marimba, xylophone",
+    "643": "mask",
+    "644": "matchstick",
+    "645": "maypole",
+    "646": "maze, labyrinth",
+    "647": "measuring cup",
+    "648": "medicine chest, medicine cabinet",
+    "649": "megalith, megalithic structure",
+    "650": "microphone, mike",
+    "651": "microwave, microwave oven",
+    "652": "military uniform",
+    "653": "milk can",
+    "654": "minibus",
+    "655": "miniskirt, mini",
+    "656": "minivan",
+    "657": "missile",
+    "658": "mitten",
+    "659": "mixing bowl",
+    "660": "mobile home, manufactured home",
+    "661": "Model T",
+    "662": "modem",
+    "663": "monastery",
+    "664": "monitor",
+    "665": "moped",
+    "666": "mortar",
+    "667": "mortarboard",
+    "668": "mosque",
+    "669": "mosquito net",
+    "670": "motor scooter, scooter",
+    "671": "mountain bike, all-terrain bike, off-roader",
+    "672": "mountain tent",
+    "673": "mouse, computer mouse",
+    "674": "mousetrap",
+    "675": "moving van",
+    "676": "muzzle",
+    "677": "nail",
+    "678": "neck brace",
+    "679": "necklace",
+    "680": "nipple",
+    "681": "notebook, notebook computer",
+    "682": "obelisk",
+    "683": "oboe, hautboy, hautbois",
+    "684": "ocarina, sweet potato",
+    "685": "odometer, hodometer, mileometer, milometer",
+    "686": "oil filter",
+    "687": "organ, pipe organ",
+    "688": "oscilloscope, scope, cathode-ray oscilloscope, CRO",
+    "689": "overskirt",
+    "690": "oxcart",
+    "691": "oxygen mask",
+    "692": "packet",
+    "693": "paddle, boat paddle",
+    "694": "paddlewheel, paddle wheel",
+    "695": "padlock",
+    "696": "paintbrush",
+    "697": "pajama, pyjama, pj's, jammies",
+    "698": "palace",
+    "699": "panpipe, pandean pipe, syrinx",
+    "700": "paper towel",
+    "701": "parachute, chute",
+    "702": "parallel bars, bars",
+    "703": "park bench",
+    "704": "parking meter",
+    "705": "passenger car, coach, carriage",
+    "706": "patio, terrace",
+    "707": "pay-phone, pay-station",
+    "708": "pedestal, plinth, footstall",
+    "709": "pencil box, pencil case",
+    "710": "pencil sharpener",
+    "711": "perfume, essence",
+    "712": "Petri dish",
+    "713": "photocopier",
+    "714": "pick, plectrum, plectron",
+    "715": "pickelhaube",
+    "716": "picket fence, paling",
+    "717": "pickup, pickup truck",
+    "718": "pier",
+    "719": "piggy bank, penny bank",
+    "720": "pill bottle",
+    "721": "pillow",
+    "722": "ping-pong ball",
+    "723": "pinwheel",
+    "724": "pirate, pirate ship",
+    "725": "pitcher, ewer",
+    "726": "plane, carpenter's plane, woodworking plane",
+    "727": "planetarium",
+    "728": "plastic bag",
+    "729": "plate rack",
+    "730": "plow, plough",
+    "731": "plunger, plumber's helper",
+    "732": "Polaroid camera, Polaroid Land camera",
+    "733": "pole",
+    "734": "police van, police wagon, paddy wagon, patrol wagon, wagon, black Maria",
+    "735": "poncho",
+    "736": "pool table, billiard table, snooker table",
+    "737": "pop bottle, soda bottle",
+    "738": "pot, flowerpot",
+    "739": "potter's wheel",
+    "740": "power drill",
+    "741": "prayer rug, prayer mat",
+    "742": "printer",
+    "743": "prison, prison house",
+    "744": "projectile, missile",
+    "745": "projector",
+    "746": "puck, hockey puck",
+    "747": "punching bag, punch bag, punching ball, punchball",
+    "748": "purse",
+    "749": "quill, quill pen",
+    "750": "quilt, comforter, comfort, puff",
+    "751": "racer, race car, racing car",
+    "752": "racket, racquet",
+    "753": "radiator",
+    "754": "radio, wireless",
+    "755": "radio telescope, radio reflector",
+    "756": "rain barrel",
+    "757": "recreational vehicle, RV, R.V.",
+    "758": "reel",
+    "759": "reflex camera",
+    "760": "refrigerator, icebox",
+    "761": "remote control, remote",
+    "762": "restaurant, eating house, eating place, eatery",
+    "763": "revolver, six-gun, six-shooter",
+    "764": "rifle",
+    "765": "rocking chair, rocker",
+    "766": "rotisserie",
+    "767": "rubber eraser, rubber, pencil eraser",
+    "768": "rugby ball",
+    "769": "rule, ruler",
+    "770": "running shoe",
+    "771": "safe",
+    "772": "safety pin",
+    "773": "saltshaker, salt shaker",
+    "774": "sandal",
+    "775": "sarong",
+    "776": "sax, saxophone",
+    "777": "scabbard",
+    "778": "scale, weighing machine",
+    "779": "school bus",
+    "780": "schooner",
+    "781": "scoreboard",
+    "782": "screen, CRT screen",
+    "783": "screw",
+    "784": "screwdriver",
+    "785": "seat belt, seatbelt",
+    "786": "sewing machine",
+    "787": "shield, buckler",
+    "788": "shoe shop, shoe-shop, shoe store",
+    "789": "shoji",
+    "790": "shopping basket",
+    "791": "shopping cart",
+    "792": "shovel",
+    "793": "shower cap",
+    "794": "shower curtain",
+    "795": "ski",
+    "796": "ski mask",
+    "797": "sleeping bag",
+    "798": "slide rule, slipstick",
+    "799": "sliding door",
+    "800": "slot, one-armed bandit",
+    "801": "snorkel",
+    "802": "snowmobile",
+    "803": "snowplow, snowplough",
+    "804": "soap dispenser",
+    "805": "soccer ball",
+    "806": "sock",
+    "807": "solar dish, solar collector, solar furnace",
+    "808": "sombrero",
+    "809": "soup bowl",
+    "810": "space bar",
+    "811": "space heater",
+    "812": "space shuttle",
+    "813": "spatula",
+    "814": "speedboat",
+    "815": "spider web, spider's web",
+    "816": "spindle",
+    "817": "sports car, sport car",
+    "818": "spotlight, spot",
+    "819": "stage",
+    "820": "steam locomotive",
+    "821": "steel arch bridge",
+    "822": "steel drum",
+    "823": "stethoscope",
+    "824": "stole",
+    "825": "stone wall",
+    "826": "stopwatch, stop watch",
+    "827": "stove",
+    "828": "strainer",
+    "829": "streetcar, tram, tramcar, trolley, trolley car",
+    "830": "stretcher",
+    "831": "studio couch, day bed",
+    "832": "stupa, tope",
+    "833": "submarine, pigboat, sub, U-boat",
+    "834": "suit, suit of clothes",
+    "835": "sundial",
+    "836": "sunglass",
+    "837": "sunglasses, dark glasses, shades",
+    "838": "sunscreen, sunblock, sun blocker",
+    "839": "suspension bridge",
+    "840": "swab, swob, mop",
+    "841": "sweatshirt",
+    "842": "swimming trunks, bathing trunks",
+    "843": "swing",
+    "844": "switch, electric switch, electrical switch",
+    "845": "syringe",
+    "846": "table lamp",
+    "847": "tank, army tank, armored combat vehicle, armoured combat vehicle",
+    "848": "tape player",
+    "849": "teapot",
+    "850": "teddy, teddy bear",
+    "851": "television, television system",
+    "852": "tennis ball",
+    "853": "thatch, thatched roof",
+    "854": "theater curtain, theatre curtain",
+    "855": "thimble",
+    "856": "thresher, thrasher, threshing machine",
+    "857": "throne",
+    "858": "tile roof",
+    "859": "toaster",
+    "860": "tobacco shop, tobacconist shop, tobacconist",
+    "861": "toilet seat",
+    "862": "torch",
+    "863": "totem pole",
+    "864": "tow truck, tow car, wrecker",
+    "865": "toyshop",
+    "866": "tractor",
+    "867": "trailer truck, tractor trailer, trucking rig, rig, articulated lorry, semi",
+    "868": "tray",
+    "869": "trench coat",
+    "870": "tricycle, trike, velocipede",
+    "871": "trimaran",
+    "872": "tripod",
+    "873": "triumphal arch",
+    "874": "trolleybus, trolley coach, trackless trolley",
+    "875": "trombone",
+    "876": "tub, vat",
+    "877": "turnstile",
+    "878": "typewriter keyboard",
+    "879": "umbrella",
+    "880": "unicycle, monocycle",
+    "881": "upright, upright piano",
+    "882": "vacuum, vacuum cleaner",
+    "883": "vase",
+    "884": "vault",
+    "885": "velvet",
+    "886": "vending machine",
+    "887": "vestment",
+    "888": "viaduct",
+    "889": "violin, fiddle",
+    "890": "volleyball",
+    "891": "waffle iron",
+    "892": "wall clock",
+    "893": "wallet, billfold, notecase, pocketbook",
+    "894": "wardrobe, closet, press",
+    "895": "warplane, military plane",
+    "896": "washbasin, handbasin, washbowl, lavabo, wash-hand basin",
+    "897": "washer, automatic washer, washing machine",
+    "898": "water bottle",
+    "899": "water jug",
+    "900": "water tower",
+    "901": "whiskey jug",
+    "902": "whistle",
+    "903": "wig",
+    "904": "window screen",
+    "905": "window shade",
+    "906": "Windsor tie",
+    "907": "wine bottle",
+    "908": "wing",
+    "909": "wok",
+    "910": "wooden spoon",
+    "911": "wool, woolen, woollen",
+    "912": "worm fence, snake fence, snake-rail fence, Virginia fence",
+    "913": "wreck",
+    "914": "yawl",
+    "915": "yurt",
+    "916": "web site, website, internet site, site",
+    "917": "comic book",
+    "918": "crossword puzzle, crossword",
+    "919": "street sign",
+    "920": "traffic light, traffic signal, stoplight",
+    "921": "book jacket, dust cover, dust jacket, dust wrapper",
+    "922": "menu",
+    "923": "plate",
+    "924": "guacamole",
+    "925": "consomme",
+    "926": "hot pot, hotpot",
+    "927": "trifle",
+    "928": "ice cream, icecream",
+    "929": "ice lolly, lolly, lollipop, popsicle",
+    "930": "French loaf",
+    "931": "bagel, beigel",
+    "932": "pretzel",
+    "933": "cheeseburger",
+    "934": "hotdog, hot dog, red hot",
+    "935": "mashed potato",
+    "936": "head cabbage",
+    "937": "broccoli",
+    "938": "cauliflower",
+    "939": "zucchini, courgette",
+    "940": "spaghetti squash",
+    "941": "acorn squash",
+    "942": "butternut squash",
+    "943": "cucumber, cuke",
+    "944": "artichoke, globe artichoke",
+    "945": "bell pepper",
+    "946": "cardoon",
+    "947": "mushroom",
+    "948": "Granny Smith",
+    "949": "strawberry",
+    "950": "orange",
+    "951": "lemon",
+    "952": "fig",
+    "953": "pineapple, ananas",
+    "954": "banana",
+    "955": "jackfruit, jak, jack",
+    "956": "custard apple",
+    "957": "pomegranate",
+    "958": "hay",
+    "959": "carbonara",
+    "960": "chocolate sauce, chocolate syrup",
+    "961": "dough",
+    "962": "meat loaf, meatloaf",
+    "963": "pizza, pizza pie",
+    "964": "potpie",
+    "965": "burrito",
+    "966": "red wine",
+    "967": "espresso",
+    "968": "cup",
+    "969": "eggnog",
+    "970": "alp",
+    "971": "bubble",
+    "972": "cliff, drop, drop-off",
+    "973": "coral reef",
+    "974": "geyser",
+    "975": "lakeside, lakeshore",
+    "976": "promontory, headland, head, foreland",
+    "977": "sandbar, sand bar",
+    "978": "seashore, coast, seacoast, sea-coast",
+    "979": "valley, vale",
+    "980": "volcano",
+    "981": "ballplayer, baseball player",
+    "982": "groom, bridegroom",
+    "983": "scuba diver",
+    "984": "rapeseed",
+    "985": "daisy",
+    "986": "yellow lady's slipper, yellow lady-slipper, Cypripedium calceolus, Cypripedium parviflorum",
+    "987": "corn",
+    "988": "acorn",
+    "989": "hip, rose hip, rosehip",
+    "990": "buckeye, horse chestnut, conker",
+    "991": "coral fungus",
+    "992": "agaric",
+    "993": "gyromitra",
+    "994": "stinkhorn, carrion fungus",
+    "995": "earthstar",
+    "996": "hen-of-the-woods, hen of the woods, Polyporus frondosus, Grifola frondosa",
+    "997": "bolete",
+    "998": "ear, spike, capitulum",
+    "999": "toilet tissue, toilet paper, bathroom tissue"
+  },
+  "image_size": 224,
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "label2id": {
+    "Afghan hound, Afghan": 160,
+    "African chameleon, Chamaeleo chamaeleon": 47,
+    "African crocodile, Nile crocodile, Crocodylus niloticus": 49,
+    "African elephant, Loxodonta africana": 386,
+    "African grey, African gray, Psittacus erithacus": 87,
+    "African hunting dog, hyena dog, Cape hunting dog, Lycaon pictus": 275,
+    "Airedale, Airedale terrier": 191,
+    "American Staffordshire terrier, Staffordshire terrier, American pit bull terrier, pit bull terrier": 180,
+    "American alligator, Alligator mississipiensis": 50,
+    "American black bear, black bear, Ursus americanus, Euarctos americanus": 295,
+    "American chameleon, anole, Anolis carolinensis": 40,
+    "American coot, marsh hen, mud hen, water hen, Fulica americana": 137,
+    "American egret, great white heron, Egretta albus": 132,
+    "American lobster, Northern lobster, Maine lobster, Homarus americanus": 122,
+    "Angora, Angora rabbit": 332,
+    "Appenzeller": 240,
+    "Arabian camel, dromedary, Camelus dromedarius": 354,
+    "Arctic fox, white fox, Alopex lagopus": 279,
+    "Australian terrier": 193,
+    "Band Aid": 419,
+    "Bedlington terrier": 181,
+    "Bernese mountain dog": 239,
+    "Blenheim spaniel": 156,
+    "Border collie": 232,
+    "Border terrier": 182,
+    "Boston bull, Boston terrier": 195,
+    "Bouvier des Flandres, Bouviers des Flandres": 233,
+    "Brabancon griffon": 262,
+    "Brittany spaniel": 215,
+    "CD player": 485,
+    "Cardigan, Cardigan Welsh corgi": 264,
+    "Chesapeake Bay retriever": 209,
+    "Chihuahua": 151,
+    "Christmas stocking": 496,
+    "Crock Pot": 521,
+    "Dandie Dinmont, Dandie Dinmont terrier": 194,
+    "Doberman, Doberman pinscher": 236,
+    "Dungeness crab, Cancer magister": 118,
+    "Dutch oven": 544,
+    "Egyptian cat": 285,
+    "English foxhound": 167,
+    "English setter": 212,
+    "English springer, English springer spaniel": 217,
+    "EntleBucher": 241,
+    "Eskimo dog, husky": 248,
+    "European fire salamander, Salamandra salamandra": 25,
+    "European gallinule, Porphyrio porphyrio": 136,
+    "French bulldog": 245,
+    "French horn, horn": 566,
+    "French loaf": 930,
+    "German shepherd, German shepherd dog, German police dog, alsatian": 235,
+    "German short-haired pointer": 210,
+    "Gila monster, Heloderma suspectum": 45,
+    "Gordon setter": 214,
+    "Granny Smith": 948,
+    "Great Dane": 246,
+    "Great Pyrenees": 257,
+    "Greater Swiss Mountain dog": 238,
+    "Ibizan hound, Ibizan Podenco": 173,
+    "Indian cobra, Naja naja": 63,
+    "Indian elephant, Elephas maximus": 385,
+    "Irish setter, red setter": 213,
+    "Irish terrier": 184,
+    "Irish water spaniel": 221,
+    "Irish wolfhound": 170,
+    "Italian greyhound": 171,
+    "Japanese spaniel": 152,
+    "Kerry blue terrier": 183,
+    "Komodo dragon, Komodo lizard, dragon lizard, giant lizard, Varanus komodoensis": 48,
+    "Labrador retriever": 208,
+    "Lakeland terrier": 189,
+    "Leonberg": 255,
+    "Lhasa, Lhasa apso": 204,
+    "Loafer": 630,
+    "Madagascar cat, ring-tailed lemur, Lemur catta": 383,
+    "Maltese dog, Maltese terrier, Maltese": 153,
+    "Mexican hairless": 268,
+    "Model T": 661,
+    "Newfoundland, Newfoundland dog": 256,
+    "Norfolk terrier": 185,
+    "Norwegian elkhound, elkhound": 174,
+    "Norwich terrier": 186,
+    "Old English sheepdog, bobtail": 229,
+    "Pekinese, Pekingese, Peke": 154,
+    "Pembroke, Pembroke Welsh corgi": 263,
+    "Persian cat": 283,
+    "Petri dish": 712,
+    "Polaroid camera, Polaroid Land camera": 732,
+    "Pomeranian": 259,
+    "Rhodesian ridgeback": 159,
+    "Rottweiler": 234,
+    "Saint Bernard, St Bernard": 247,
+    "Saluki, gazelle hound": 176,
+    "Samoyed, Samoyede": 258,
+    "Scotch terrier, Scottish terrier, Scottie": 199,
+    "Scottish deerhound, deerhound": 177,
+    "Sealyham terrier, Sealyham": 190,
+    "Shetland sheepdog, Shetland sheep dog, Shetland": 230,
+    "Shih-Tzu": 155,
+    "Siamese cat, Siamese": 284,
+    "Siberian husky": 250,
+    "Staffordshire bullterrier, Staffordshire bull terrier": 179,
+    "Sussex spaniel": 220,
+    "Tibetan mastiff": 244,
+    "Tibetan terrier, chrysanthemum dog": 200,
+    "Walker hound, Walker foxhound": 166,
+    "Weimaraner": 178,
+    "Welsh springer spaniel": 218,
+    "West Highland white terrier": 203,
+    "Windsor tie": 906,
+    "Yorkshire terrier": 187,
+    "abacus": 398,
+    "abaya": 399,
+    "academic gown, academic robe, judge's robe": 400,
+    "accordion, piano accordion, squeeze box": 401,
+    "acorn": 988,
+    "acorn squash": 941,
+    "acoustic guitar": 402,
+    "admiral": 321,
+    "affenpinscher, monkey pinscher, monkey dog": 252,
+    "agama": 42,
+    "agaric": 992,
+    "aircraft carrier, carrier, flattop, attack aircraft carrier": 403,
+    "airliner": 404,
+    "airship, dirigible": 405,
+    "albatross, mollymawk": 146,
+    "alligator lizard": 44,
+    "alp": 970,
+    "altar": 406,
+    "ambulance": 407,
+    "amphibian, amphibious vehicle": 408,
+    "analog clock": 409,
+    "anemone fish": 393,
+    "ant, emmet, pismire": 310,
+    "apiary, bee house": 410,
+    "apron": 411,
+    "armadillo": 363,
+    "artichoke, globe artichoke": 944,
+    "ashcan, trash can, garbage can, wastebin, ash bin, ash-bin, ashbin, dustbin, trash barrel, trash bin": 412,
+    "assault rifle, assault gun": 413,
+    "axolotl, mud puppy, Ambystoma mexicanum": 29,
+    "baboon": 372,
+    "backpack, back pack, knapsack, packsack, rucksack, haversack": 414,
+    "badger": 362,
+    "bagel, beigel": 931,
+    "bakery, bakeshop, bakehouse": 415,
+    "balance beam, beam": 416,
+    "bald eagle, American eagle, Haliaeetus leucocephalus": 22,
+    "balloon": 417,
+    "ballplayer, baseball player": 981,
+    "ballpoint, ballpoint pen, ballpen, Biro": 418,
+    "banana": 954,
+    "banded gecko": 38,
+    "banjo": 420,
+    "bannister, banister, balustrade, balusters, handrail": 421,
+    "barbell": 422,
+    "barber chair": 423,
+    "barbershop": 424,
+    "barn": 425,
+    "barn spider, Araneus cavaticus": 73,
+    "barometer": 426,
+    "barracouta, snoek": 389,
+    "barrel, cask": 427,
+    "barrow, garden cart, lawn cart, wheelbarrow": 428,
+    "baseball": 429,
+    "basenji": 253,
+    "basketball": 430,
+    "basset, basset hound": 161,
+    "bassinet": 431,
+    "bassoon": 432,
+    "bath towel": 434,
+    "bathing cap, swimming cap": 433,
+    "bathtub, bathing tub, bath, tub": 435,
+    "beach wagon, station wagon, wagon, estate car, beach waggon, station waggon, waggon": 436,
+    "beacon, lighthouse, beacon light, pharos": 437,
+    "beagle": 162,
+    "beaker": 438,
+    "bearskin, busby, shako": 439,
+    "beaver": 337,
+    "bee": 309,
+    "bee eater": 92,
+    "beer bottle": 440,
+    "beer glass": 441,
+    "bell cote, bell cot": 442,
+    "bell pepper": 945,
+    "bib": 443,
+    "bicycle-built-for-two, tandem bicycle, tandem": 444,
+    "bighorn, bighorn sheep, cimarron, Rocky Mountain bighorn, Rocky Mountain sheep, Ovis canadensis": 349,
+    "bikini, two-piece": 445,
+    "binder, ring-binder": 446,
+    "binoculars, field glasses, opera glasses": 447,
+    "birdhouse": 448,
+    "bison": 347,
+    "bittern": 133,
+    "black and gold garden spider, Argiope aurantia": 72,
+    "black grouse": 80,
+    "black stork, Ciconia nigra": 128,
+    "black swan, Cygnus atratus": 100,
+    "black widow, Latrodectus mactans": 75,
+    "black-and-tan coonhound": 165,
+    "black-footed ferret, ferret, Mustela nigripes": 359,
+    "bloodhound, sleuthhound": 163,
+    "bluetick": 164,
+    "boa constrictor, Constrictor constrictor": 61,
+    "boathouse": 449,
+    "bobsled, bobsleigh, bob": 450,
+    "bolete": 997,
+    "bolo tie, bolo, bola tie, bola": 451,
+    "bonnet, poke bonnet": 452,
+    "book jacket, dust cover, dust jacket, dust wrapper": 921,
+    "bookcase": 453,
+    "bookshop, bookstore, bookstall": 454,
+    "borzoi, Russian wolfhound": 169,
+    "bottlecap": 455,
+    "bow": 456,
+    "bow tie, bow-tie, bowtie": 457,
+    "box turtle, box tortoise": 37,
+    "boxer": 242,
+    "brain coral": 109,
+    "brambling, Fringilla montifringilla": 10,
+    "brass, memorial tablet, plaque": 458,
+    "brassiere, bra, bandeau": 459,
+    "breakwater, groin, groyne, mole, bulwark, seawall, jetty": 460,
+    "breastplate, aegis, egis": 461,
+    "briard": 226,
+    "broccoli": 937,
+    "broom": 462,
+    "brown bear, bruin, Ursus arctos": 294,
+    "bubble": 971,
+    "bucket, pail": 463,
+    "buckeye, horse chestnut, conker": 990,
+    "buckle": 464,
+    "bulbul": 16,
+    "bull mastiff": 243,
+    "bullet train, bullet": 466,
+    "bulletproof vest": 465,
+    "bullfrog, Rana catesbeiana": 30,
+    "burrito": 965,
+    "bustard": 138,
+    "butcher shop, meat market": 467,
+    "butternut squash": 942,
+    "cab, hack, taxi, taxicab": 468,
+    "cabbage butterfly": 324,
+    "cairn, cairn terrier": 192,
+    "caldron, cauldron": 469,
+    "can opener, tin opener": 473,
+    "candle, taper, wax light": 470,
+    "cannon": 471,
+    "canoe": 472,
+    "capuchin, ringtail, Cebus capucinus": 378,
+    "car mirror": 475,
+    "car wheel": 479,
+    "carbonara": 959,
+    "cardigan": 474,
+    "cardoon": 946,
+    "carousel, carrousel, merry-go-round, roundabout, whirligig": 476,
+    "carpenter's kit, tool kit": 477,
+    "carton": 478,
+    "cash machine, cash dispenser, automated teller machine, automatic teller machine, automated teller, automatic teller, ATM": 480,
+    "cassette": 481,
+    "cassette player": 482,
+    "castle": 483,
+    "catamaran": 484,
+    "cauliflower": 938,
+    "cello, violoncello": 486,
+    "cellular telephone, cellular phone, cellphone, cell, mobile phone": 487,
+    "centipede": 79,
+    "chain": 488,
+    "chain mail, ring mail, mail, chain armor, chain armour, ring armor, ring armour": 490,
+    "chain saw, chainsaw": 491,
+    "chainlink fence": 489,
+    "chambered nautilus, pearly nautilus, nautilus": 117,
+    "cheeseburger": 933,
+    "cheetah, chetah, Acinonyx jubatus": 293,
+    "chest": 492,
+    "chickadee": 19,
+    "chiffonier, commode": 493,
+    "chime, bell, gong": 494,
+    "chimpanzee, chimp, Pan troglodytes": 367,
+    "china cabinet, china closet": 495,
+    "chiton, coat-of-mail shell, sea cradle, polyplacophore": 116,
+    "chocolate sauce, chocolate syrup": 960,
+    "chow, chow chow": 260,
+    "church, church building": 497,
+    "cicada, cicala": 316,
+    "cinema, movie theater, movie theatre, movie house, picture palace": 498,
+    "cleaver, meat cleaver, chopper": 499,
+    "cliff dwelling": 500,
+    "cliff, drop, drop-off": 972,
+    "cloak": 501,
+    "clog, geta, patten, sabot": 502,
+    "clumber, clumber spaniel": 216,
+    "cock": 7,
+    "cocker spaniel, English cocker spaniel, cocker": 219,
+    "cockroach, roach": 314,
+    "cocktail shaker": 503,
+    "coffee mug": 504,
+    "coffeepot": 505,
+    "coho, cohoe, coho salmon, blue jack, silver salmon, Oncorhynchus kisutch": 391,
+    "coil, spiral, volute, whorl, helix": 506,
+    "collie": 231,
+    "colobus, colobus monkey": 375,
+    "combination lock": 507,
+    "comic book": 917,
+    "common iguana, iguana, Iguana iguana": 39,
+    "common newt, Triturus vulgaris": 26,
+    "computer keyboard, keypad": 508,
+    "conch": 112,
+    "confectionery, confectionary, candy store": 509,
+    "consomme": 925,
+    "container ship, containership, container vessel": 510,
+    "convertible": 511,
+    "coral fungus": 991,
+    "coral reef": 973,
+    "corkscrew, bottle screw": 512,
+    "corn": 987,
+    "cornet, horn, trumpet, trump": 513,
+    "coucal": 91,
+    "cougar, puma, catamount, mountain lion, painter, panther, Felis concolor": 286,
+    "cowboy boot": 514,
+    "cowboy hat, ten-gallon hat": 515,
+    "coyote, prairie wolf, brush wolf, Canis latrans": 272,
+    "cradle": 516,
+    "crane": 517,
+    "crash helmet": 518,
+    "crate": 519,
+    "crayfish, crawfish, crawdad, crawdaddy": 124,
+    "crib, cot": 520,
+    "cricket": 312,
+    "croquet ball": 522,
+    "crossword puzzle, crossword": 918,
+    "crutch": 523,
+    "cucumber, cuke": 943,
+    "cuirass": 524,
+    "cup": 968,
+    "curly-coated retriever": 206,
+    "custard apple": 956,
+    "daisy": 985,
+    "dalmatian, coach dog, carriage dog": 251,
+    "dam, dike, dyke": 525,
+    "damselfly": 320,
+    "desk": 526,
+    "desktop computer": 527,
+    "dhole, Cuon alpinus": 274,
+    "dial telephone, dial phone": 528,
+    "diamondback, diamondback rattlesnake, Crotalus adamanteus": 67,
+    "diaper, nappy, napkin": 529,
+    "digital clock": 530,
+    "digital watch": 531,
+    "dingo, warrigal, warragal, Canis dingo": 273,
+    "dining table, board": 532,
+    "dishrag, dishcloth": 533,
+    "dishwasher, dish washer, dishwashing machine": 534,
+    "disk brake, disc brake": 535,
+    "dock, dockage, docking facility": 536,
+    "dogsled, dog sled, dog sleigh": 537,
+    "dome": 538,
+    "doormat, welcome mat": 539,
+    "dough": 961,
+    "dowitcher": 142,
+    "dragonfly, darning needle, devil's darning needle, sewing needle, snake feeder, snake doctor, mosquito hawk, skeeter hawk": 319,
+    "drake": 97,
+    "drilling platform, offshore rig": 540,
+    "drum, membranophone, tympan": 541,
+    "drumstick": 542,
+    "dugong, Dugong dugon": 149,
+    "dumbbell": 543,
+    "dung beetle": 305,
+    "ear, spike, capitulum": 998,
+    "earthstar": 995,
+    "echidna, spiny anteater, anteater": 102,
+    "eel": 390,
+    "eft": 27,
+    "eggnog": 969,
+    "electric fan, blower": 545,
+    "electric guitar": 546,
+    "electric locomotive": 547,
+    "electric ray, crampfish, numbfish, torpedo": 5,
+    "entertainment center": 548,
+    "envelope": 549,
+    "espresso": 967,
+    "espresso maker": 550,
+    "face powder": 551,
+    "feather boa, boa": 552,
+    "fiddler crab": 120,
+    "fig": 952,
+    "file, file cabinet, filing cabinet": 553,
+    "fire engine, fire truck": 555,
+    "fire screen, fireguard": 556,
+    "fireboat": 554,
+    "flagpole, flagstaff": 557,
+    "flamingo": 130,
+    "flat-coated retriever": 205,
+    "flatworm, platyhelminth": 110,
+    "flute, transverse flute": 558,
+    "fly": 308,
+    "folding chair": 559,
+    "football helmet": 560,
+    "forklift": 561,
+    "fountain": 562,
+    "fountain pen": 563,
+    "four-poster": 564,
+    "fox squirrel, eastern fox squirrel, Sciurus niger": 335,
+    "freight car": 565,
+    "frilled lizard, Chlamydosaurus kingi": 43,
+    "frying pan, frypan, skillet": 567,
+    "fur coat": 568,
+    "gar, garfish, garpike, billfish, Lepisosteus osseus": 395,
+    "garbage truck, dustcart": 569,
+    "garden spider, Aranea diademata": 74,
+    "garter snake, grass snake": 57,
+    "gas pump, gasoline pump, petrol pump, island dispenser": 571,
+    "gasmask, respirator, gas helmet": 570,
+    "gazelle": 353,
+    "geyser": 974,
+    "giant panda, panda, panda bear, coon bear, Ailuropoda melanoleuca": 388,
+    "giant schnauzer": 197,
+    "gibbon, Hylobates lar": 368,
+    "go-kart": 573,
+    "goblet": 572,
+    "golden retriever": 207,
+    "goldfinch, Carduelis carduelis": 11,
+    "goldfish, Carassius auratus": 1,
+    "golf ball": 574,
+    "golfcart, golf cart": 575,
+    "gondola": 576,
+    "gong, tam-tam": 577,
+    "goose": 99,
+    "gorilla, Gorilla gorilla": 366,
+    "gown": 578,
+    "grand piano, grand": 579,
+    "grasshopper, hopper": 311,
+    "great grey owl, great gray owl, Strix nebulosa": 24,
+    "great white shark, white shark, man-eater, man-eating shark, Carcharodon carcharias": 2,
+    "green lizard, Lacerta viridis": 46,
+    "green mamba": 64,
+    "green snake, grass snake": 55,
+    "greenhouse, nursery, glasshouse": 580,
+    "grey fox, gray fox, Urocyon cinereoargenteus": 280,
+    "grey whale, gray whale, devilfish, Eschrichtius gibbosus, Eschrichtius robustus": 147,
+    "grille, radiator grille": 581,
+    "grocery store, grocery, food market, market": 582,
+    "groenendael": 224,
+    "groom, bridegroom": 982,
+    "ground beetle, carabid beetle": 302,
+    "guacamole": 924,
+    "guenon, guenon monkey": 370,
+    "guillotine": 583,
+    "guinea pig, Cavia cobaya": 338,
+    "gyromitra": 993,
+    "hair slide": 584,
+    "hair spray": 585,
+    "half track": 586,
+    "hammer": 587,
+    "hammerhead, hammerhead shark": 4,
+    "hamper": 588,
+    "hamster": 333,
+    "hand blower, blow dryer, blow drier, hair dryer, hair drier": 589,
+    "hand-held computer, hand-held microcomputer": 590,
+    "handkerchief, hankie, hanky, hankey": 591,
+    "hard disc, hard disk, fixed disk": 592,
+    "hare": 331,
+    "harmonica, mouth organ, harp, mouth harp": 593,
+    "harp": 594,
+    "hartebeest": 351,
+    "harvester, reaper": 595,
+    "harvestman, daddy longlegs, Phalangium opilio": 70,
+    "hatchet": 596,
+    "hay": 958,
+    "head cabbage": 936,
+    "hen": 8,
+    "hen-of-the-woods, hen of the woods, Polyporus frondosus, Grifola frondosa": 996,
+    "hermit crab": 125,
+    "hip, rose hip, rosehip": 989,
+    "hippopotamus, hippo, river horse, Hippopotamus amphibius": 344,
+    "hog, pig, grunter, squealer, Sus scrofa": 341,
+    "hognose snake, puff adder, sand viper": 54,
+    "holster": 597,
+    "home theater, home theatre": 598,
+    "honeycomb": 599,
+    "hook, claw": 600,
+    "hoopskirt, crinoline": 601,
+    "horizontal bar, high bar": 602,
+    "hornbill": 93,
+    "horned viper, cerastes, sand viper, horned asp, Cerastes cornutus": 66,
+    "horse cart, horse-cart": 603,
+    "hot pot, hotpot": 926,
+    "hotdog, hot dog, red hot": 934,
+    "hourglass": 604,
+    "house finch, linnet, Carpodacus mexicanus": 12,
+    "howler monkey, howler": 379,
+    "hummingbird": 94,
+    "hyena, hyaena": 276,
+    "iPod": 605,
+    "ibex, Capra ibex": 350,
+    "ice bear, polar bear, Ursus Maritimus, Thalarctos maritimus": 296,
+    "ice cream, icecream": 928,
+    "ice lolly, lolly, lollipop, popsicle": 929,
+    "impala, Aepyceros melampus": 352,
+    "indigo bunting, indigo finch, indigo bird, Passerina cyanea": 14,
+    "indri, indris, Indri indri, Indri brevicaudatus": 384,
+    "iron, smoothing iron": 606,
+    "isopod": 126,
+    "jacamar": 95,
+    "jack-o'-lantern": 607,
+    "jackfruit, jak, jack": 955,
+    "jaguar, panther, Panthera onca, Felis onca": 290,
+    "jay": 17,
+    "jean, blue jean, denim": 608,
+    "jeep, landrover": 609,
+    "jellyfish": 107,
+    "jersey, T-shirt, tee shirt": 610,
+    "jigsaw puzzle": 611,
+    "jinrikisha, ricksha, rickshaw": 612,
+    "joystick": 613,
+    "junco, snowbird": 13,
+    "keeshond": 261,
+    "kelpie": 227,
+    "killer whale, killer, orca, grampus, sea wolf, Orcinus orca": 148,
+    "kimono": 614,
+    "king crab, Alaska crab, Alaskan king crab, Alaska king crab, Paralithodes camtschatica": 121,
+    "king penguin, Aptenodytes patagonica": 145,
+    "king snake, kingsnake": 56,
+    "kit fox, Vulpes macrotis": 278,
+    "kite": 21,
+    "knee pad": 615,
+    "knot": 616,
+    "koala, koala bear, kangaroo bear, native bear, Phascolarctos cinereus": 105,
+    "komondor": 228,
+    "kuvasz": 222,
+    "lab coat, laboratory coat": 617,
+    "lacewing, lacewing fly": 318,
+    "ladle": 618,
+    "ladybug, ladybeetle, lady beetle, ladybird, ladybird beetle": 301,
+    "lakeside, lakeshore": 975,
+    "lampshade, lamp shade": 619,
+    "langur": 374,
+    "laptop, laptop computer": 620,
+    "lawn mower, mower": 621,
+    "leaf beetle, chrysomelid": 304,
+    "leafhopper": 317,
+    "leatherback turtle, leatherback, leathery turtle, Dermochelys coriacea": 34,
+    "lemon": 951,
+    "lens cap, lens cover": 622,
+    "leopard, Panthera pardus": 288,
+    "lesser panda, red panda, panda, bear cat, cat bear, Ailurus fulgens": 387,
+    "letter opener, paper knife, paperknife": 623,
+    "library": 624,
+    "lifeboat": 625,
+    "lighter, light, igniter, ignitor": 626,
+    "limousine, limo": 627,
+    "limpkin, Aramus pictus": 135,
+    "liner, ocean liner": 628,
+    "lion, king of beasts, Panthera leo": 291,
+    "lionfish": 396,
+    "lipstick, lip rouge": 629,
+    "little blue heron, Egretta caerulea": 131,
+    "llama": 355,
+    "loggerhead, loggerhead turtle, Caretta caretta": 33,
+    "long-horned beetle, longicorn, longicorn beetle": 303,
+    "lorikeet": 90,
+    "lotion": 631,
+    "loudspeaker, speaker, speaker unit, loudspeaker system, speaker system": 632,
+    "loupe, jeweler's loupe": 633,
+    "lumbermill, sawmill": 634,
+    "lycaenid, lycaenid butterfly": 326,
+    "lynx, catamount": 287,
+    "macaque": 373,
+    "macaw": 88,
+    "magnetic compass": 635,
+    "magpie": 18,
+    "mailbag, postbag": 636,
+    "mailbox, letter box": 637,
+    "maillot": 638,
+    "maillot, tank suit": 639,
+    "malamute, malemute, Alaskan malamute": 249,
+    "malinois": 225,
+    "manhole cover": 640,
+    "mantis, mantid": 315,
+    "maraca": 641,
+    "marimba, xylophone": 642,
+    "marmoset": 377,
+    "marmot": 336,
+    "mashed potato": 935,
+    "mask": 643,
+    "matchstick": 644,
+    "maypole": 645,
+    "maze, labyrinth": 646,
+    "measuring cup": 647,
+    "meat loaf, meatloaf": 962,
+    "medicine chest, medicine cabinet": 648,
+    "meerkat, mierkat": 299,
+    "megalith, megalithic structure": 649,
+    "menu": 922,
+    "microphone, mike": 650,
+    "microwave, microwave oven": 651,
+    "military uniform": 652,
+    "milk can": 653,
+    "miniature pinscher": 237,
+    "miniature poodle": 266,
+    "miniature schnauzer": 196,
+    "minibus": 654,
+    "miniskirt, mini": 655,
+    "minivan": 656,
+    "mink": 357,
+    "missile": 657,
+    "mitten": 658,
+    "mixing bowl": 659,
+    "mobile home, manufactured home": 660,
+    "modem": 662,
+    "monarch, monarch butterfly, milkweed butterfly, Danaus plexippus": 323,
+    "monastery": 663,
+    "mongoose": 298,
+    "monitor": 664,
+    "moped": 665,
+    "mortar": 666,
+    "mortarboard": 667,
+    "mosque": 668,
+    "mosquito net": 669,
+    "motor scooter, scooter": 670,
+    "mountain bike, all-terrain bike, off-roader": 671,
+    "mountain tent": 672,
+    "mouse, computer mouse": 673,
+    "mousetrap": 674,
+    "moving van": 675,
+    "mud turtle": 35,
+    "mushroom": 947,
+    "muzzle": 676,
+    "nail": 677,
+    "neck brace": 678,
+    "necklace": 679,
+    "nematode, nematode worm, roundworm": 111,
+    "night snake, Hypsiglena torquata": 60,
+    "nipple": 680,
+    "notebook, notebook computer": 681,
+    "obelisk": 682,
+    "oboe, hautboy, hautbois": 683,
+    "ocarina, sweet potato": 684,
+    "odometer, hodometer, mileometer, milometer": 685,
+    "oil filter": 686,
+    "orange": 950,
+    "orangutan, orang, orangutang, Pongo pygmaeus": 365,
+    "organ, pipe organ": 687,
+    "oscilloscope, scope, cathode-ray oscilloscope, CRO": 688,
+    "ostrich, Struthio camelus": 9,
+    "otter": 360,
+    "otterhound, otter hound": 175,
+    "overskirt": 689,
+    "ox": 345,
+    "oxcart": 690,
+    "oxygen mask": 691,
+    "oystercatcher, oyster catcher": 143,
+    "packet": 692,
+    "paddle, boat paddle": 693,
+    "paddlewheel, paddle wheel": 694,
+    "padlock": 695,
+    "paintbrush": 696,
+    "pajama, pyjama, pj's, jammies": 697,
+    "palace": 698,
+    "panpipe, pandean pipe, syrinx": 699,
+    "paper towel": 700,
+    "papillon": 157,
+    "parachute, chute": 701,
+    "parallel bars, bars": 702,
+    "park bench": 703,
+    "parking meter": 704,
+    "partridge": 86,
+    "passenger car, coach, carriage": 705,
+    "patas, hussar monkey, Erythrocebus patas": 371,
+    "patio, terrace": 706,
+    "pay-phone, pay-station": 707,
+    "peacock": 84,
+    "pedestal, plinth, footstall": 708,
+    "pelican": 144,
+    "pencil box, pencil case": 709,
+    "pencil sharpener": 710,
+    "perfume, essence": 711,
+    "photocopier": 713,
+    "pick, plectrum, plectron": 714,
+    "pickelhaube": 715,
+    "picket fence, paling": 716,
+    "pickup, pickup truck": 717,
+    "pier": 718,
+    "piggy bank, penny bank": 719,
+    "pill bottle": 720,
+    "pillow": 721,
+    "pineapple, ananas": 953,
+    "ping-pong ball": 722,
+    "pinwheel": 723,
+    "pirate, pirate ship": 724,
+    "pitcher, ewer": 725,
+    "pizza, pizza pie": 963,
+    "plane, carpenter's plane, woodworking plane": 726,
+    "planetarium": 727,
+    "plastic bag": 728,
+    "plate": 923,
+    "plate rack": 729,
+    "platypus, duckbill, duckbilled platypus, duck-billed platypus, Ornithorhynchus anatinus": 103,
+    "plow, plough": 730,
+    "plunger, plumber's helper": 731,
+    "pole": 733,
+    "polecat, fitch, foulmart, foumart, Mustela putorius": 358,
+    "police van, police wagon, paddy wagon, patrol wagon, wagon, black Maria": 734,
+    "pomegranate": 957,
+    "poncho": 735,
+    "pool table, billiard table, snooker table": 736,
+    "pop bottle, soda bottle": 737,
+    "porcupine, hedgehog": 334,
+    "pot, flowerpot": 738,
+    "potpie": 964,
+    "potter's wheel": 739,
+    "power drill": 740,
+    "prairie chicken, prairie grouse, prairie fowl": 83,
+    "prayer rug, prayer mat": 741,
+    "pretzel": 932,
+    "printer": 742,
+    "prison, prison house": 743,
+    "proboscis monkey, Nasalis larvatus": 376,
+    "projectile, missile": 744,
+    "projector": 745,
+    "promontory, headland, head, foreland": 976,
+    "ptarmigan": 81,
+    "puck, hockey puck": 746,
+    "puffer, pufferfish, blowfish, globefish": 397,
+    "pug, pug-dog": 254,
+    "punching bag, punch bag, punching ball, punchball": 747,
+    "purse": 748,
+    "quail": 85,
+    "quill, quill pen": 749,
+    "quilt, comforter, comfort, puff": 750,
+    "racer, race car, racing car": 751,
+    "racket, racquet": 752,
+    "radiator": 753,
+    "radio telescope, radio reflector": 755,
+    "radio, wireless": 754,
+    "rain barrel": 756,
+    "ram, tup": 348,
+    "rapeseed": 984,
+    "recreational vehicle, RV, R.V.": 757,
+    "red fox, Vulpes vulpes": 277,
+    "red wine": 966,
+    "red wolf, maned wolf, Canis rufus, Canis niger": 271,
+    "red-backed sandpiper, dunlin, Erolia alpina": 140,
+    "red-breasted merganser, Mergus serrator": 98,
+    "redbone": 168,
+    "redshank, Tringa totanus": 141,
+    "reel": 758,
+    "reflex camera": 759,
+    "refrigerator, icebox": 760,
+    "remote control, remote": 761,
+    "restaurant, eating house, eating place, eatery": 762,
+    "revolver, six-gun, six-shooter": 763,
+    "rhinoceros beetle": 306,
+    "rifle": 764,
+    "ringlet, ringlet butterfly": 322,
+    "ringneck snake, ring-necked snake, ring snake": 53,
+    "robin, American robin, Turdus migratorius": 15,
+    "rock beauty, Holocanthus tricolor": 392,
+    "rock crab, Cancer irroratus": 119,
+    "rock python, rock snake, Python sebae": 62,
+    "rocking chair, rocker": 765,
+    "rotisserie": 766,
+    "rubber eraser, rubber, pencil eraser": 767,
+    "ruddy turnstone, Arenaria interpres": 139,
+    "ruffed grouse, partridge, Bonasa umbellus": 82,
+    "rugby ball": 768,
+    "rule, ruler": 769,
+    "running shoe": 770,
+    "safe": 771,
+    "safety pin": 772,
+    "saltshaker, salt shaker": 773,
+    "sandal": 774,
+    "sandbar, sand bar": 977,
+    "sarong": 775,
+    "sax, saxophone": 776,
+    "scabbard": 777,
+    "scale, weighing machine": 778,
+    "schipperke": 223,
+    "school bus": 779,
+    "schooner": 780,
+    "scoreboard": 781,
+    "scorpion": 71,
+    "screen, CRT screen": 782,
+    "screw": 783,
+    "screwdriver": 784,
+    "scuba diver": 983,
+    "sea anemone, anemone": 108,
+    "sea cucumber, holothurian": 329,
+    "sea lion": 150,
+    "sea slug, nudibranch": 115,
+    "sea snake": 65,
+    "sea urchin": 328,
+    "seashore, coast, seacoast, sea-coast": 978,
+    "seat belt, seatbelt": 785,
+    "sewing machine": 786,
+    "shield, buckler": 787,
+    "shoe shop, shoe-shop, shoe store": 788,
+    "shoji": 789,
+    "shopping basket": 790,
+    "shopping cart": 791,
+    "shovel": 792,
+    "shower cap": 793,
+    "shower curtain": 794,
+    "siamang, Hylobates syndactylus, Symphalangus syndactylus": 369,
+    "sidewinder, horned rattlesnake, Crotalus cerastes": 68,
+    "silky terrier, Sydney silky": 201,
+    "ski": 795,
+    "ski mask": 796,
+    "skunk, polecat, wood pussy": 361,
+    "sleeping bag": 797,
+    "slide rule, slipstick": 798,
+    "sliding door": 799,
+    "slot, one-armed bandit": 800,
+    "sloth bear, Melursus ursinus, Ursus ursinus": 297,
+    "slug": 114,
+    "snail": 113,
+    "snorkel": 801,
+    "snow leopard, ounce, Panthera uncia": 289,
+    "snowmobile": 802,
+    "snowplow, snowplough": 803,
+    "soap dispenser": 804,
+    "soccer ball": 805,
+    "sock": 806,
+    "soft-coated wheaten terrier": 202,
+    "solar dish, solar collector, solar furnace": 807,
+    "sombrero": 808,
+    "sorrel": 339,
+    "soup bowl": 809,
+    "space bar": 810,
+    "space heater": 811,
+    "space shuttle": 812,
+    "spaghetti squash": 940,
+    "spatula": 813,
+    "speedboat": 814,
+    "spider monkey, Ateles geoffroyi": 381,
+    "spider web, spider's web": 815,
+    "spindle": 816,
+    "spiny lobster, langouste, rock lobster, crawfish, crayfish, sea crawfish": 123,
+    "spoonbill": 129,
+    "sports car, sport car": 817,
+    "spotlight, spot": 818,
+    "spotted salamander, Ambystoma maculatum": 28,
+    "squirrel monkey, Saimiri sciureus": 382,
+    "stage": 819,
+    "standard poodle": 267,
+    "standard schnauzer": 198,
+    "starfish, sea star": 327,
+    "steam locomotive": 820,
+    "steel arch bridge": 821,
+    "steel drum": 822,
+    "stethoscope": 823,
+    "stingray": 6,
+    "stinkhorn, carrion fungus": 994,
+    "stole": 824,
+    "stone wall": 825,
+    "stopwatch, stop watch": 826,
+    "stove": 827,
+    "strainer": 828,
+    "strawberry": 949,
+    "street sign": 919,
+    "streetcar, tram, tramcar, trolley, trolley car": 829,
+    "stretcher": 830,
+    "studio couch, day bed": 831,
+    "stupa, tope": 832,
+    "sturgeon": 394,
+    "submarine, pigboat, sub, U-boat": 833,
+    "suit, suit of clothes": 834,
+    "sulphur butterfly, sulfur butterfly": 325,
+    "sulphur-crested cockatoo, Kakatoe galerita, Cacatua galerita": 89,
+    "sundial": 835,
+    "sunglass": 836,
+    "sunglasses, dark glasses, shades": 837,
+    "sunscreen, sunblock, sun blocker": 838,
+    "suspension bridge": 839,
+    "swab, swob, mop": 840,
+    "sweatshirt": 841,
+    "swimming trunks, bathing trunks": 842,
+    "swing": 843,
+    "switch, electric switch, electrical switch": 844,
+    "syringe": 845,
+    "tabby, tabby cat": 281,
+    "table lamp": 846,
+    "tailed frog, bell toad, ribbed toad, tailed toad, Ascaphus trui": 32,
+    "tank, army tank, armored combat vehicle, armoured combat vehicle": 847,
+    "tape player": 848,
+    "tarantula": 76,
+    "teapot": 849,
+    "teddy, teddy bear": 850,
+    "television, television system": 851,
+    "tench, Tinca tinca": 0,
+    "tennis ball": 852,
+    "terrapin": 36,
+    "thatch, thatched roof": 853,
+    "theater curtain, theatre curtain": 854,
+    "thimble": 855,
+    "three-toed sloth, ai, Bradypus tridactylus": 364,
+    "thresher, thrasher, threshing machine": 856,
+    "throne": 857,
+    "thunder snake, worm snake, Carphophis amoenus": 52,
+    "tick": 78,
+    "tiger beetle": 300,
+    "tiger cat": 282,
+    "tiger shark, Galeocerdo cuvieri": 3,
+    "tiger, Panthera tigris": 292,
+    "tile roof": 858,
+    "timber wolf, grey wolf, gray wolf, Canis lupus": 269,
+    "titi, titi monkey": 380,
+    "toaster": 859,
+    "tobacco shop, tobacconist shop, tobacconist": 860,
+    "toilet seat": 861,
+    "toilet tissue, toilet paper, bathroom tissue": 999,
+    "torch": 862,
+    "totem pole": 863,
+    "toucan": 96,
+    "tow truck, tow car, wrecker": 864,
+    "toy poodle": 265,
+    "toy terrier": 158,
+    "toyshop": 865,
+    "tractor": 866,
+    "traffic light, traffic signal, stoplight": 920,
+    "trailer truck, tractor trailer, trucking rig, rig, articulated lorry, semi": 867,
+    "tray": 868,
+    "tree frog, tree-frog": 31,
+    "trench coat": 869,
+    "triceratops": 51,
+    "tricycle, trike, velocipede": 870,
+    "trifle": 927,
+    "trilobite": 69,
+    "trimaran": 871,
+    "tripod": 872,
+    "triumphal arch": 873,
+    "trolleybus, trolley coach, trackless trolley": 874,
+    "trombone": 875,
+    "tub, vat": 876,
+    "turnstile": 877,
+    "tusker": 101,
+    "typewriter keyboard": 878,
+    "umbrella": 879,
+    "unicycle, monocycle": 880,
+    "upright, upright piano": 881,
+    "vacuum, vacuum cleaner": 882,
+    "valley, vale": 979,
+    "vase": 883,
+    "vault": 884,
+    "velvet": 885,
+    "vending machine": 886,
+    "vestment": 887,
+    "viaduct": 888,
+    "vine snake": 59,
+    "violin, fiddle": 889,
+    "vizsla, Hungarian pointer": 211,
+    "volcano": 980,
+    "volleyball": 890,
+    "vulture": 23,
+    "waffle iron": 891,
+    "walking stick, walkingstick, stick insect": 313,
+    "wall clock": 892,
+    "wallaby, brush kangaroo": 104,
+    "wallet, billfold, notecase, pocketbook": 893,
+    "wardrobe, closet, press": 894,
+    "warplane, military plane": 895,
+    "warthog": 343,
+    "washbasin, handbasin, washbowl, lavabo, wash-hand basin": 896,
+    "washer, automatic washer, washing machine": 897,
+    "water bottle": 898,
+    "water buffalo, water ox, Asiatic buffalo, Bubalus bubalis": 346,
+    "water jug": 899,
+    "water ouzel, dipper": 20,
+    "water snake": 58,
+    "water tower": 900,
+    "weasel": 356,
+    "web site, website, internet site, site": 916,
+    "weevil": 307,
+    "whippet": 172,
+    "whiptail, whiptail lizard": 41,
+    "whiskey jug": 901,
+    "whistle": 902,
+    "white stork, Ciconia ciconia": 127,
+    "white wolf, Arctic wolf, Canis lupus tundrarum": 270,
+    "wig": 903,
+    "wild boar, boar, Sus scrofa": 342,
+    "window screen": 904,
+    "window shade": 905,
+    "wine bottle": 907,
+    "wing": 908,
+    "wire-haired fox terrier": 188,
+    "wok": 909,
+    "wolf spider, hunting spider": 77,
+    "wombat": 106,
+    "wood rabbit, cottontail, cottontail rabbit": 330,
+    "wooden spoon": 910,
+    "wool, woolen, woollen": 911,
+    "worm fence, snake fence, snake-rail fence, Virginia fence": 912,
+    "wreck": 913,
+    "yawl": 914,
+    "yellow lady's slipper, yellow lady-slipper, Cypripedium calceolus, Cypripedium parviflorum": 986,
+    "yurt": 915,
+    "zebra": 340,
+    "zucchini, courgette": 939
+  },
+  "layer_norm_eps": 1e-12,
+  "model_type": "vit",
+  "num_attention_heads": 12,
+  "num_channels": 3,
+  "num_hidden_layers": 12,
+  "patch_size": 16,
+  "qkv_bias": true,
+  "torch_dtype": "float32",
+  "transformers_version": "4.44.2"
+}

chatbot/models/500/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d46ec8a4f83d8d75025bb7d299730df4642b3ab7082cb6594d94bcc7c69491e1
+size 343220892

chatbot/models/HuggingFaceTB/SmolLM2-1.7B-Instruct/config.json

@@ -0,0 +1,37 @@
+{
+  "_name_or_path": "HuggingFaceTB/SmolLM2-1.7B-Instruct",
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "head_dim": 64,
+  "hidden_act": "silu",
+  "hidden_size": 2048,
+  "initializer_range": 0.02,
+  "intermediate_size": 8192,
+  "max_position_embeddings": 8192,
+  "mlp_bias": false,
+  "model_type": "llama",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 24,
+  "num_key_value_heads": 32,
+  "pad_token_id": 2,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": null,
+  "rope_theta": 130000,
+  "tie_word_embeddings": true,
+  "torch_dtype": "float32",
+  "transformers.js_config": {
+    "kv_cache_dtype": {
+      "fp16": "float16",
+      "q4f16": "float16"
+    }
+  },
+  "transformers_version": "4.46.2",
+  "use_cache": true,
+  "vocab_size": 49152
+}

chatbot/models/HuggingFaceTB/SmolLM2-1.7B-Instruct/generation_config.json

@@ -0,0 +1,7 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 1,
+  "eos_token_id": 2,
+  "pad_token_id": 2,
+  "transformers_version": "4.46.2"
+}

chatbot/models/HuggingFaceTB/SmolLM2-1.7B-Instruct/model-00001-of-00002.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d21ed78acd64116143c09180c275689fd8eab22a8920455c1c64f63ceab9089a
+size 4999906800

chatbot/models/HuggingFaceTB/SmolLM2-1.7B-Instruct/model-00002-of-00002.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a1eecb36a045a2ba3deff58cfa2bbb7dddfb76f260e853b442128d1799055b61
+size 1845623728

chatbot/models/HuggingFaceTB/SmolLM2-1.7B-Instruct/model.safetensors.index.json

@@ -0,0 +1,225 @@
+{
+  "metadata": {
+    "total_size": 6845505536
+  },
+  "weight_map": {
+    "model.embed_tokens.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.0.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.1.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.10.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.11.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.12.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.13.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.14.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.15.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.16.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.17.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.17.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.17.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.17.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.17.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.17.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.17.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.17.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.17.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.18.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.18.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.18.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.18.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.18.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.18.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.18.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.18.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.18.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.19.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.19.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.19.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.19.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.19.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.19.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.19.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.19.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.19.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.2.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.2.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.20.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.20.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.20.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.20.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.20.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.20.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.20.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.20.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.20.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.21.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.21.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.21.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.21.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.21.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.21.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.21.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.21.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.21.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.22.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.22.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.22.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.22.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.22.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.22.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.22.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.22.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.22.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.23.input_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.23.mlp.down_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.23.mlp.gate_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.23.mlp.up_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.23.post_attention_layernorm.weight": "model-00002-of-00002.safetensors",
+    "model.layers.23.self_attn.k_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.23.self_attn.o_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.23.self_attn.q_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.23.self_attn.v_proj.weight": "model-00002-of-00002.safetensors",
+    "model.layers.3.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.3.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.3.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.3.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.3.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.3.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.3.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.3.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.3.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.4.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.4.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.4.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.4.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.4.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.4.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.4.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.4.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.4.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.5.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.6.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.7.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.8.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.input_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.mlp.down_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.mlp.gate_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.mlp.up_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.post_attention_layernorm.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.self_attn.k_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.self_attn.o_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.self_attn.q_proj.weight": "model-00001-of-00002.safetensors",
+    "model.layers.9.self_attn.v_proj.weight": "model-00001-of-00002.safetensors",
+    "model.norm.weight": "model-00002-of-00002.safetensors"
+  }
+}

chatbot/models/HuggingFaceTB/SmolLM2-1.7B-Instruct/special_tokens_map.json

@@ -0,0 +1,34 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "bos_token": {
+    "content": "<|im_start|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

chatbot/models/HuggingFaceTB/SmolLM2-1.7B-Instruct/tokenizer_config.json

@@ -0,0 +1,154 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "<repo_name>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "4": {
+      "content": "<reponame>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "5": {
+      "content": "<file_sep>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "6": {
+      "content": "<filename>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "7": {
+      "content": "<gh_stars>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "8": {
+      "content": "<issue_start>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "9": {
+      "content": "<issue_comment>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "10": {
+      "content": "<issue_closed>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "11": {
+      "content": "<jupyter_start>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "12": {
+      "content": "<jupyter_text>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "13": {
+      "content": "<jupyter_code>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "14": {
+      "content": "<jupyter_output>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "15": {
+      "content": "<jupyter_script>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "16": {
+      "content": "<empty_output>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "bos_token": "<|im_start|>",
+  "chat_template": "{% for message in messages %}{% if loop.first and messages[0]['role'] != 'system' %}{{ '<|im_start|>system\nYou are a helpful AI assistant named SmolLM, trained by Hugging Face<|im_end|>\n' }}{% endif %}{{'<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n'}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant\n' }}{% endif %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "model_max_length": 2048,
+  "pad_token": "<|im_end|>",
+  "tokenizer_class": "GPT2Tokenizer",
+  "unk_token": "<|endoftext|>",
+  "vocab_size": 49152
+}

chatbot/models/custom_model/config.json

@@ -0,0 +1,2026 @@
+{
+  "_name_or_path": "google/vit-base-patch16-224",
+  "architectures": [
+    "ViTForImageClassification"
+  ],
+  "attention_probs_dropout_prob": 0.0,
+  "encoder_stride": 16,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.0,
+  "hidden_size": 768,
+  "id2label": {
+    "0": "tench, Tinca tinca",
+    "1": "goldfish, Carassius auratus",
+    "2": "great white shark, white shark, man-eater, man-eating shark, Carcharodon carcharias",
+    "3": "tiger shark, Galeocerdo cuvieri",
+    "4": "hammerhead, hammerhead shark",
+    "5": "electric ray, crampfish, numbfish, torpedo",
+    "6": "stingray",
+    "7": "cock",
+    "8": "hen",
+    "9": "ostrich, Struthio camelus",
+    "10": "brambling, Fringilla montifringilla",
+    "11": "goldfinch, Carduelis carduelis",
+    "12": "house finch, linnet, Carpodacus mexicanus",
+    "13": "junco, snowbird",
+    "14": "indigo bunting, indigo finch, indigo bird, Passerina cyanea",
+    "15": "robin, American robin, Turdus migratorius",
+    "16": "bulbul",
+    "17": "jay",
+    "18": "magpie",
+    "19": "chickadee",
+    "20": "water ouzel, dipper",
+    "21": "kite",
+    "22": "bald eagle, American eagle, Haliaeetus leucocephalus",
+    "23": "vulture",
+    "24": "great grey owl, great gray owl, Strix nebulosa",
+    "25": "European fire salamander, Salamandra salamandra",
+    "26": "common newt, Triturus vulgaris",
+    "27": "eft",
+    "28": "spotted salamander, Ambystoma maculatum",
+    "29": "axolotl, mud puppy, Ambystoma mexicanum",
+    "30": "bullfrog, Rana catesbeiana",
+    "31": "tree frog, tree-frog",
+    "32": "tailed frog, bell toad, ribbed toad, tailed toad, Ascaphus trui",
+    "33": "loggerhead, loggerhead turtle, Caretta caretta",
+    "34": "leatherback turtle, leatherback, leathery turtle, Dermochelys coriacea",
+    "35": "mud turtle",
+    "36": "terrapin",
+    "37": "box turtle, box tortoise",
+    "38": "banded gecko",
+    "39": "common iguana, iguana, Iguana iguana",
+    "40": "American chameleon, anole, Anolis carolinensis",
+    "41": "whiptail, whiptail lizard",
+    "42": "agama",
+    "43": "frilled lizard, Chlamydosaurus kingi",
+    "44": "alligator lizard",
+    "45": "Gila monster, Heloderma suspectum",
+    "46": "green lizard, Lacerta viridis",
+    "47": "African chameleon, Chamaeleo chamaeleon",
+    "48": "Komodo dragon, Komodo lizard, dragon lizard, giant lizard, Varanus komodoensis",
+    "49": "African crocodile, Nile crocodile, Crocodylus niloticus",
+    "50": "American alligator, Alligator mississipiensis",
+    "51": "triceratops",
+    "52": "thunder snake, worm snake, Carphophis amoenus",
+    "53": "ringneck snake, ring-necked snake, ring snake",
+    "54": "hognose snake, puff adder, sand viper",
+    "55": "green snake, grass snake",
+    "56": "king snake, kingsnake",
+    "57": "garter snake, grass snake",
+    "58": "water snake",
+    "59": "vine snake",
+    "60": "night snake, Hypsiglena torquata",
+    "61": "boa constrictor, Constrictor constrictor",
+    "62": "rock python, rock snake, Python sebae",
+    "63": "Indian cobra, Naja naja",
+    "64": "green mamba",
+    "65": "sea snake",
+    "66": "horned viper, cerastes, sand viper, horned asp, Cerastes cornutus",
+    "67": "diamondback, diamondback rattlesnake, Crotalus adamanteus",
+    "68": "sidewinder, horned rattlesnake, Crotalus cerastes",
+    "69": "trilobite",
+    "70": "harvestman, daddy longlegs, Phalangium opilio",
+    "71": "scorpion",
+    "72": "black and gold garden spider, Argiope aurantia",
+    "73": "barn spider, Araneus cavaticus",
+    "74": "garden spider, Aranea diademata",
+    "75": "black widow, Latrodectus mactans",
+    "76": "tarantula",
+    "77": "wolf spider, hunting spider",
+    "78": "tick",
+    "79": "centipede",
+    "80": "black grouse",
+    "81": "ptarmigan",
+    "82": "ruffed grouse, partridge, Bonasa umbellus",
+    "83": "prairie chicken, prairie grouse, prairie fowl",
+    "84": "peacock",
+    "85": "quail",
+    "86": "partridge",
+    "87": "African grey, African gray, Psittacus erithacus",
+    "88": "macaw",
+    "89": "sulphur-crested cockatoo, Kakatoe galerita, Cacatua galerita",
+    "90": "lorikeet",
+    "91": "coucal",
+    "92": "bee eater",
+    "93": "hornbill",
+    "94": "hummingbird",
+    "95": "jacamar",
+    "96": "toucan",
+    "97": "drake",
+    "98": "red-breasted merganser, Mergus serrator",
+    "99": "goose",
+    "100": "black swan, Cygnus atratus",
+    "101": "tusker",
+    "102": "echidna, spiny anteater, anteater",
+    "103": "platypus, duckbill, duckbilled platypus, duck-billed platypus, Ornithorhynchus anatinus",
+    "104": "wallaby, brush kangaroo",
+    "105": "koala, koala bear, kangaroo bear, native bear, Phascolarctos cinereus",
+    "106": "wombat",
+    "107": "jellyfish",
+    "108": "sea anemone, anemone",
+    "109": "brain coral",
+    "110": "flatworm, platyhelminth",
+    "111": "nematode, nematode worm, roundworm",
+    "112": "conch",
+    "113": "snail",
+    "114": "slug",
+    "115": "sea slug, nudibranch",
+    "116": "chiton, coat-of-mail shell, sea cradle, polyplacophore",
+    "117": "chambered nautilus, pearly nautilus, nautilus",
+    "118": "Dungeness crab, Cancer magister",
+    "119": "rock crab, Cancer irroratus",
+    "120": "fiddler crab",
+    "121": "king crab, Alaska crab, Alaskan king crab, Alaska king crab, Paralithodes camtschatica",
+    "122": "American lobster, Northern lobster, Maine lobster, Homarus americanus",
+    "123": "spiny lobster, langouste, rock lobster, crawfish, crayfish, sea crawfish",
+    "124": "crayfish, crawfish, crawdad, crawdaddy",
+    "125": "hermit crab",
+    "126": "isopod",
+    "127": "white stork, Ciconia ciconia",
+    "128": "black stork, Ciconia nigra",
+    "129": "spoonbill",
+    "130": "flamingo",
+    "131": "little blue heron, Egretta caerulea",
+    "132": "American egret, great white heron, Egretta albus",
+    "133": "bittern",
+    "134": "crane",
+    "135": "limpkin, Aramus pictus",
+    "136": "European gallinule, Porphyrio porphyrio",
+    "137": "American coot, marsh hen, mud hen, water hen, Fulica americana",
+    "138": "bustard",
+    "139": "ruddy turnstone, Arenaria interpres",
+    "140": "red-backed sandpiper, dunlin, Erolia alpina",
+    "141": "redshank, Tringa totanus",
+    "142": "dowitcher",
+    "143": "oystercatcher, oyster catcher",
+    "144": "pelican",
+    "145": "king penguin, Aptenodytes patagonica",
+    "146": "albatross, mollymawk",
+    "147": "grey whale, gray whale, devilfish, Eschrichtius gibbosus, Eschrichtius robustus",
+    "148": "killer whale, killer, orca, grampus, sea wolf, Orcinus orca",
+    "149": "dugong, Dugong dugon",
+    "150": "sea lion",
+    "151": "Chihuahua",
+    "152": "Japanese spaniel",
+    "153": "Maltese dog, Maltese terrier, Maltese",
+    "154": "Pekinese, Pekingese, Peke",
+    "155": "Shih-Tzu",
+    "156": "Blenheim spaniel",
+    "157": "papillon",
+    "158": "toy terrier",
+    "159": "Rhodesian ridgeback",
+    "160": "Afghan hound, Afghan",
+    "161": "basset, basset hound",
+    "162": "beagle",
+    "163": "bloodhound, sleuthhound",
+    "164": "bluetick",
+    "165": "black-and-tan coonhound",
+    "166": "Walker hound, Walker foxhound",
+    "167": "English foxhound",
+    "168": "redbone",
+    "169": "borzoi, Russian wolfhound",
+    "170": "Irish wolfhound",
+    "171": "Italian greyhound",
+    "172": "whippet",
+    "173": "Ibizan hound, Ibizan Podenco",
+    "174": "Norwegian elkhound, elkhound",
+    "175": "otterhound, otter hound",
+    "176": "Saluki, gazelle hound",
+    "177": "Scottish deerhound, deerhound",
+    "178": "Weimaraner",
+    "179": "Staffordshire bullterrier, Staffordshire bull terrier",
+    "180": "American Staffordshire terrier, Staffordshire terrier, American pit bull terrier, pit bull terrier",
+    "181": "Bedlington terrier",
+    "182": "Border terrier",
+    "183": "Kerry blue terrier",
+    "184": "Irish terrier",
+    "185": "Norfolk terrier",
+    "186": "Norwich terrier",
+    "187": "Yorkshire terrier",
+    "188": "wire-haired fox terrier",
+    "189": "Lakeland terrier",
+    "190": "Sealyham terrier, Sealyham",
+    "191": "Airedale, Airedale terrier",
+    "192": "cairn, cairn terrier",
+    "193": "Australian terrier",
+    "194": "Dandie Dinmont, Dandie Dinmont terrier",
+    "195": "Boston bull, Boston terrier",
+    "196": "miniature schnauzer",
+    "197": "giant schnauzer",
+    "198": "standard schnauzer",
+    "199": "Scotch terrier, Scottish terrier, Scottie",
+    "200": "Tibetan terrier, chrysanthemum dog",
+    "201": "silky terrier, Sydney silky",
+    "202": "soft-coated wheaten terrier",
+    "203": "West Highland white terrier",
+    "204": "Lhasa, Lhasa apso",
+    "205": "flat-coated retriever",
+    "206": "curly-coated retriever",
+    "207": "golden retriever",
+    "208": "Labrador retriever",
+    "209": "Chesapeake Bay retriever",
+    "210": "German short-haired pointer",
+    "211": "vizsla, Hungarian pointer",
+    "212": "English setter",
+    "213": "Irish setter, red setter",
+    "214": "Gordon setter",
+    "215": "Brittany spaniel",
+    "216": "clumber, clumber spaniel",
+    "217": "English springer, English springer spaniel",
+    "218": "Welsh springer spaniel",
+    "219": "cocker spaniel, English cocker spaniel, cocker",
+    "220": "Sussex spaniel",
+    "221": "Irish water spaniel",
+    "222": "kuvasz",
+    "223": "schipperke",
+    "224": "groenendael",
+    "225": "malinois",
+    "226": "briard",
+    "227": "kelpie",
+    "228": "komondor",
+    "229": "Old English sheepdog, bobtail",
+    "230": "Shetland sheepdog, Shetland sheep dog, Shetland",
+    "231": "collie",
+    "232": "Border collie",
+    "233": "Bouvier des Flandres, Bouviers des Flandres",
+    "234": "Rottweiler",
+    "235": "German shepherd, German shepherd dog, German police dog, alsatian",
+    "236": "Doberman, Doberman pinscher",
+    "237": "miniature pinscher",
+    "238": "Greater Swiss Mountain dog",
+    "239": "Bernese mountain dog",
+    "240": "Appenzeller",
+    "241": "EntleBucher",
+    "242": "boxer",
+    "243": "bull mastiff",
+    "244": "Tibetan mastiff",
+    "245": "French bulldog",
+    "246": "Great Dane",
+    "247": "Saint Bernard, St Bernard",
+    "248": "Eskimo dog, husky",
+    "249": "malamute, malemute, Alaskan malamute",
+    "250": "Siberian husky",
+    "251": "dalmatian, coach dog, carriage dog",
+    "252": "affenpinscher, monkey pinscher, monkey dog",
+    "253": "basenji",
+    "254": "pug, pug-dog",
+    "255": "Leonberg",
+    "256": "Newfoundland, Newfoundland dog",
+    "257": "Great Pyrenees",
+    "258": "Samoyed, Samoyede",
+    "259": "Pomeranian",
+    "260": "chow, chow chow",
+    "261": "keeshond",
+    "262": "Brabancon griffon",
+    "263": "Pembroke, Pembroke Welsh corgi",
+    "264": "Cardigan, Cardigan Welsh corgi",
+    "265": "toy poodle",
+    "266": "miniature poodle",
+    "267": "standard poodle",
+    "268": "Mexican hairless",
+    "269": "timber wolf, grey wolf, gray wolf, Canis lupus",
+    "270": "white wolf, Arctic wolf, Canis lupus tundrarum",
+    "271": "red wolf, maned wolf, Canis rufus, Canis niger",
+    "272": "coyote, prairie wolf, brush wolf, Canis latrans",
+    "273": "dingo, warrigal, warragal, Canis dingo",
+    "274": "dhole, Cuon alpinus",
+    "275": "African hunting dog, hyena dog, Cape hunting dog, Lycaon pictus",
+    "276": "hyena, hyaena",
+    "277": "red fox, Vulpes vulpes",
+    "278": "kit fox, Vulpes macrotis",
+    "279": "Arctic fox, white fox, Alopex lagopus",
+    "280": "grey fox, gray fox, Urocyon cinereoargenteus",
+    "281": "tabby, tabby cat",
+    "282": "tiger cat",
+    "283": "Persian cat",
+    "284": "Siamese cat, Siamese",
+    "285": "Egyptian cat",
+    "286": "cougar, puma, catamount, mountain lion, painter, panther, Felis concolor",
+    "287": "lynx, catamount",
+    "288": "leopard, Panthera pardus",
+    "289": "snow leopard, ounce, Panthera uncia",
+    "290": "jaguar, panther, Panthera onca, Felis onca",
+    "291": "lion, king of beasts, Panthera leo",
+    "292": "tiger, Panthera tigris",
+    "293": "cheetah, chetah, Acinonyx jubatus",
+    "294": "brown bear, bruin, Ursus arctos",
+    "295": "American black bear, black bear, Ursus americanus, Euarctos americanus",
+    "296": "ice bear, polar bear, Ursus Maritimus, Thalarctos maritimus",
+    "297": "sloth bear, Melursus ursinus, Ursus ursinus",
+    "298": "mongoose",
+    "299": "meerkat, mierkat",
+    "300": "tiger beetle",
+    "301": "ladybug, ladybeetle, lady beetle, ladybird, ladybird beetle",
+    "302": "ground beetle, carabid beetle",
+    "303": "long-horned beetle, longicorn, longicorn beetle",
+    "304": "leaf beetle, chrysomelid",
+    "305": "dung beetle",
+    "306": "rhinoceros beetle",
+    "307": "weevil",
+    "308": "fly",
+    "309": "bee",
+    "310": "ant, emmet, pismire",
+    "311": "grasshopper, hopper",
+    "312": "cricket",
+    "313": "walking stick, walkingstick, stick insect",
+    "314": "cockroach, roach",
+    "315": "mantis, mantid",
+    "316": "cicada, cicala",
+    "317": "leafhopper",
+    "318": "lacewing, lacewing fly",
+    "319": "dragonfly, darning needle, devil's darning needle, sewing needle, snake feeder, snake doctor, mosquito hawk, skeeter hawk",
+    "320": "damselfly",
+    "321": "admiral",
+    "322": "ringlet, ringlet butterfly",
+    "323": "monarch, monarch butterfly, milkweed butterfly, Danaus plexippus",
+    "324": "cabbage butterfly",
+    "325": "sulphur butterfly, sulfur butterfly",
+    "326": "lycaenid, lycaenid butterfly",
+    "327": "starfish, sea star",
+    "328": "sea urchin",
+    "329": "sea cucumber, holothurian",
+    "330": "wood rabbit, cottontail, cottontail rabbit",
+    "331": "hare",
+    "332": "Angora, Angora rabbit",
+    "333": "hamster",
+    "334": "porcupine, hedgehog",
+    "335": "fox squirrel, eastern fox squirrel, Sciurus niger",
+    "336": "marmot",
+    "337": "beaver",
+    "338": "guinea pig, Cavia cobaya",
+    "339": "sorrel",
+    "340": "zebra",
+    "341": "hog, pig, grunter, squealer, Sus scrofa",
+    "342": "wild boar, boar, Sus scrofa",
+    "343": "warthog",
+    "344": "hippopotamus, hippo, river horse, Hippopotamus amphibius",
+    "345": "ox",
+    "346": "water buffalo, water ox, Asiatic buffalo, Bubalus bubalis",
+    "347": "bison",
+    "348": "ram, tup",
+    "349": "bighorn, bighorn sheep, cimarron, Rocky Mountain bighorn, Rocky Mountain sheep, Ovis canadensis",
+    "350": "ibex, Capra ibex",
+    "351": "hartebeest",
+    "352": "impala, Aepyceros melampus",
+    "353": "gazelle",
+    "354": "Arabian camel, dromedary, Camelus dromedarius",
+    "355": "llama",
+    "356": "weasel",
+    "357": "mink",
+    "358": "polecat, fitch, foulmart, foumart, Mustela putorius",
+    "359": "black-footed ferret, ferret, Mustela nigripes",
+    "360": "otter",
+    "361": "skunk, polecat, wood pussy",
+    "362": "badger",
+    "363": "armadillo",
+    "364": "three-toed sloth, ai, Bradypus tridactylus",
+    "365": "orangutan, orang, orangutang, Pongo pygmaeus",
+    "366": "gorilla, Gorilla gorilla",
+    "367": "chimpanzee, chimp, Pan troglodytes",
+    "368": "gibbon, Hylobates lar",
+    "369": "siamang, Hylobates syndactylus, Symphalangus syndactylus",
+    "370": "guenon, guenon monkey",
+    "371": "patas, hussar monkey, Erythrocebus patas",
+    "372": "baboon",
+    "373": "macaque",
+    "374": "langur",
+    "375": "colobus, colobus monkey",
+    "376": "proboscis monkey, Nasalis larvatus",
+    "377": "marmoset",
+    "378": "capuchin, ringtail, Cebus capucinus",
+    "379": "howler monkey, howler",
+    "380": "titi, titi monkey",
+    "381": "spider monkey, Ateles geoffroyi",
+    "382": "squirrel monkey, Saimiri sciureus",
+    "383": "Madagascar cat, ring-tailed lemur, Lemur catta",
+    "384": "indri, indris, Indri indri, Indri brevicaudatus",
+    "385": "Indian elephant, Elephas maximus",
+    "386": "African elephant, Loxodonta africana",
+    "387": "lesser panda, red panda, panda, bear cat, cat bear, Ailurus fulgens",
+    "388": "giant panda, panda, panda bear, coon bear, Ailuropoda melanoleuca",
+    "389": "barracouta, snoek",
+    "390": "eel",
+    "391": "coho, cohoe, coho salmon, blue jack, silver salmon, Oncorhynchus kisutch",
+    "392": "rock beauty, Holocanthus tricolor",
+    "393": "anemone fish",
+    "394": "sturgeon",
+    "395": "gar, garfish, garpike, billfish, Lepisosteus osseus",
+    "396": "lionfish",
+    "397": "puffer, pufferfish, blowfish, globefish",
+    "398": "abacus",
+    "399": "abaya",
+    "400": "academic gown, academic robe, judge's robe",
+    "401": "accordion, piano accordion, squeeze box",
+    "402": "acoustic guitar",
+    "403": "aircraft carrier, carrier, flattop, attack aircraft carrier",
+    "404": "airliner",
+    "405": "airship, dirigible",
+    "406": "altar",
+    "407": "ambulance",
+    "408": "amphibian, amphibious vehicle",
+    "409": "analog clock",
+    "410": "apiary, bee house",
+    "411": "apron",
+    "412": "ashcan, trash can, garbage can, wastebin, ash bin, ash-bin, ashbin, dustbin, trash barrel, trash bin",
+    "413": "assault rifle, assault gun",
+    "414": "backpack, back pack, knapsack, packsack, rucksack, haversack",
+    "415": "bakery, bakeshop, bakehouse",
+    "416": "balance beam, beam",
+    "417": "balloon",
+    "418": "ballpoint, ballpoint pen, ballpen, Biro",
+    "419": "Band Aid",
+    "420": "banjo",
+    "421": "bannister, banister, balustrade, balusters, handrail",
+    "422": "barbell",
+    "423": "barber chair",
+    "424": "barbershop",
+    "425": "barn",
+    "426": "barometer",
+    "427": "barrel, cask",
+    "428": "barrow, garden cart, lawn cart, wheelbarrow",
+    "429": "baseball",
+    "430": "basketball",
+    "431": "bassinet",
+    "432": "bassoon",
+    "433": "bathing cap, swimming cap",
+    "434": "bath towel",
+    "435": "bathtub, bathing tub, bath, tub",
+    "436": "beach wagon, station wagon, wagon, estate car, beach waggon, station waggon, waggon",
+    "437": "beacon, lighthouse, beacon light, pharos",
+    "438": "beaker",
+    "439": "bearskin, busby, shako",
+    "440": "beer bottle",
+    "441": "beer glass",
+    "442": "bell cote, bell cot",
+    "443": "bib",
+    "444": "bicycle-built-for-two, tandem bicycle, tandem",
+    "445": "bikini, two-piece",
+    "446": "binder, ring-binder",
+    "447": "binoculars, field glasses, opera glasses",
+    "448": "birdhouse",
+    "449": "boathouse",
+    "450": "bobsled, bobsleigh, bob",
+    "451": "bolo tie, bolo, bola tie, bola",
+    "452": "bonnet, poke bonnet",
+    "453": "bookcase",
+    "454": "bookshop, bookstore, bookstall",
+    "455": "bottlecap",
+    "456": "bow",
+    "457": "bow tie, bow-tie, bowtie",
+    "458": "brass, memorial tablet, plaque",
+    "459": "brassiere, bra, bandeau",
+    "460": "breakwater, groin, groyne, mole, bulwark, seawall, jetty",
+    "461": "breastplate, aegis, egis",
+    "462": "broom",
+    "463": "bucket, pail",
+    "464": "buckle",
+    "465": "bulletproof vest",
+    "466": "bullet train, bullet",
+    "467": "butcher shop, meat market",
+    "468": "cab, hack, taxi, taxicab",
+    "469": "caldron, cauldron",
+    "470": "candle, taper, wax light",
+    "471": "cannon",
+    "472": "canoe",
+    "473": "can opener, tin opener",
+    "474": "cardigan",
+    "475": "car mirror",
+    "476": "carousel, carrousel, merry-go-round, roundabout, whirligig",
+    "477": "carpenter's kit, tool kit",
+    "478": "carton",
+    "479": "car wheel",
+    "480": "cash machine, cash dispenser, automated teller machine, automatic teller machine, automated teller, automatic teller, ATM",
+    "481": "cassette",
+    "482": "cassette player",
+    "483": "castle",
+    "484": "catamaran",
+    "485": "CD player",
+    "486": "cello, violoncello",
+    "487": "cellular telephone, cellular phone, cellphone, cell, mobile phone",
+    "488": "chain",
+    "489": "chainlink fence",
+    "490": "chain mail, ring mail, mail, chain armor, chain armour, ring armor, ring armour",
+    "491": "chain saw, chainsaw",
+    "492": "chest",
+    "493": "chiffonier, commode",
+    "494": "chime, bell, gong",
+    "495": "china cabinet, china closet",
+    "496": "Christmas stocking",
+    "497": "church, church building",
+    "498": "cinema, movie theater, movie theatre, movie house, picture palace",
+    "499": "cleaver, meat cleaver, chopper",
+    "500": "cliff dwelling",
+    "501": "cloak",
+    "502": "clog, geta, patten, sabot",
+    "503": "cocktail shaker",
+    "504": "coffee mug",
+    "505": "coffeepot",
+    "506": "coil, spiral, volute, whorl, helix",
+    "507": "combination lock",
+    "508": "computer keyboard, keypad",
+    "509": "confectionery, confectionary, candy store",
+    "510": "container ship, containership, container vessel",
+    "511": "convertible",
+    "512": "corkscrew, bottle screw",
+    "513": "cornet, horn, trumpet, trump",
+    "514": "cowboy boot",
+    "515": "cowboy hat, ten-gallon hat",
+    "516": "cradle",
+    "517": "crane",
+    "518": "crash helmet",
+    "519": "crate",
+    "520": "crib, cot",
+    "521": "Crock Pot",
+    "522": "croquet ball",
+    "523": "crutch",
+    "524": "cuirass",
+    "525": "dam, dike, dyke",
+    "526": "desk",
+    "527": "desktop computer",
+    "528": "dial telephone, dial phone",
+    "529": "diaper, nappy, napkin",
+    "530": "digital clock",
+    "531": "digital watch",
+    "532": "dining table, board",
+    "533": "dishrag, dishcloth",
+    "534": "dishwasher, dish washer, dishwashing machine",
+    "535": "disk brake, disc brake",
+    "536": "dock, dockage, docking facility",
+    "537": "dogsled, dog sled, dog sleigh",
+    "538": "dome",
+    "539": "doormat, welcome mat",
+    "540": "drilling platform, offshore rig",
+    "541": "drum, membranophone, tympan",
+    "542": "drumstick",
+    "543": "dumbbell",
+    "544": "Dutch oven",
+    "545": "electric fan, blower",
+    "546": "electric guitar",
+    "547": "electric locomotive",
+    "548": "entertainment center",
+    "549": "envelope",
+    "550": "espresso maker",
+    "551": "face powder",
+    "552": "feather boa, boa",
+    "553": "file, file cabinet, filing cabinet",
+    "554": "fireboat",
+    "555": "fire engine, fire truck",
+    "556": "fire screen, fireguard",
+    "557": "flagpole, flagstaff",
+    "558": "flute, transverse flute",
+    "559": "folding chair",
+    "560": "football helmet",
+    "561": "forklift",
+    "562": "fountain",
+    "563": "fountain pen",
+    "564": "four-poster",
+    "565": "freight car",
+    "566": "French horn, horn",
+    "567": "frying pan, frypan, skillet",
+    "568": "fur coat",
+    "569": "garbage truck, dustcart",
+    "570": "gasmask, respirator, gas helmet",
+    "571": "gas pump, gasoline pump, petrol pump, island dispenser",
+    "572": "goblet",
+    "573": "go-kart",
+    "574": "golf ball",
+    "575": "golfcart, golf cart",
+    "576": "gondola",
+    "577": "gong, tam-tam",
+    "578": "gown",
+    "579": "grand piano, grand",
+    "580": "greenhouse, nursery, glasshouse",
+    "581": "grille, radiator grille",
+    "582": "grocery store, grocery, food market, market",
+    "583": "guillotine",
+    "584": "hair slide",
+    "585": "hair spray",
+    "586": "half track",
+    "587": "hammer",
+    "588": "hamper",
+    "589": "hand blower, blow dryer, blow drier, hair dryer, hair drier",
+    "590": "hand-held computer, hand-held microcomputer",
+    "591": "handkerchief, hankie, hanky, hankey",
+    "592": "hard disc, hard disk, fixed disk",
+    "593": "harmonica, mouth organ, harp, mouth harp",
+    "594": "harp",
+    "595": "harvester, reaper",
+    "596": "hatchet",
+    "597": "holster",
+    "598": "home theater, home theatre",
+    "599": "honeycomb",
+    "600": "hook, claw",
+    "601": "hoopskirt, crinoline",
+    "602": "horizontal bar, high bar",
+    "603": "horse cart, horse-cart",
+    "604": "hourglass",
+    "605": "iPod",
+    "606": "iron, smoothing iron",
+    "607": "jack-o'-lantern",
+    "608": "jean, blue jean, denim",
+    "609": "jeep, landrover",
+    "610": "jersey, T-shirt, tee shirt",
+    "611": "jigsaw puzzle",
+    "612": "jinrikisha, ricksha, rickshaw",
+    "613": "joystick",
+    "614": "kimono",
+    "615": "knee pad",
+    "616": "knot",
+    "617": "lab coat, laboratory coat",
+    "618": "ladle",
+    "619": "lampshade, lamp shade",
+    "620": "laptop, laptop computer",
+    "621": "lawn mower, mower",
+    "622": "lens cap, lens cover",
+    "623": "letter opener, paper knife, paperknife",
+    "624": "library",
+    "625": "lifeboat",
+    "626": "lighter, light, igniter, ignitor",
+    "627": "limousine, limo",
+    "628": "liner, ocean liner",
+    "629": "lipstick, lip rouge",
+    "630": "Loafer",
+    "631": "lotion",
+    "632": "loudspeaker, speaker, speaker unit, loudspeaker system, speaker system",
+    "633": "loupe, jeweler's loupe",
+    "634": "lumbermill, sawmill",
+    "635": "magnetic compass",
+    "636": "mailbag, postbag",
+    "637": "mailbox, letter box",
+    "638": "maillot",
+    "639": "maillot, tank suit",
+    "640": "manhole cover",
+    "641": "maraca",
+    "642": "marimba, xylophone",
+    "643": "mask",
+    "644": "matchstick",
+    "645": "maypole",
+    "646": "maze, labyrinth",
+    "647": "measuring cup",
+    "648": "medicine chest, medicine cabinet",
+    "649": "megalith, megalithic structure",
+    "650": "microphone, mike",
+    "651": "microwave, microwave oven",
+    "652": "military uniform",
+    "653": "milk can",
+    "654": "minibus",
+    "655": "miniskirt, mini",
+    "656": "minivan",
+    "657": "missile",
+    "658": "mitten",
+    "659": "mixing bowl",
+    "660": "mobile home, manufactured home",
+    "661": "Model T",
+    "662": "modem",
+    "663": "monastery",
+    "664": "monitor",
+    "665": "moped",
+    "666": "mortar",
+    "667": "mortarboard",
+    "668": "mosque",
+    "669": "mosquito net",
+    "670": "motor scooter, scooter",
+    "671": "mountain bike, all-terrain bike, off-roader",
+    "672": "mountain tent",
+    "673": "mouse, computer mouse",
+    "674": "mousetrap",
+    "675": "moving van",
+    "676": "muzzle",
+    "677": "nail",
+    "678": "neck brace",
+    "679": "necklace",
+    "680": "nipple",
+    "681": "notebook, notebook computer",
+    "682": "obelisk",
+    "683": "oboe, hautboy, hautbois",
+    "684": "ocarina, sweet potato",
+    "685": "odometer, hodometer, mileometer, milometer",
+    "686": "oil filter",
+    "687": "organ, pipe organ",
+    "688": "oscilloscope, scope, cathode-ray oscilloscope, CRO",
+    "689": "overskirt",
+    "690": "oxcart",
+    "691": "oxygen mask",
+    "692": "packet",
+    "693": "paddle, boat paddle",
+    "694": "paddlewheel, paddle wheel",
+    "695": "padlock",
+    "696": "paintbrush",
+    "697": "pajama, pyjama, pj's, jammies",
+    "698": "palace",
+    "699": "panpipe, pandean pipe, syrinx",
+    "700": "paper towel",
+    "701": "parachute, chute",
+    "702": "parallel bars, bars",
+    "703": "park bench",
+    "704": "parking meter",
+    "705": "passenger car, coach, carriage",
+    "706": "patio, terrace",
+    "707": "pay-phone, pay-station",
+    "708": "pedestal, plinth, footstall",
+    "709": "pencil box, pencil case",
+    "710": "pencil sharpener",
+    "711": "perfume, essence",
+    "712": "Petri dish",
+    "713": "photocopier",
+    "714": "pick, plectrum, plectron",
+    "715": "pickelhaube",
+    "716": "picket fence, paling",
+    "717": "pickup, pickup truck",
+    "718": "pier",
+    "719": "piggy bank, penny bank",
+    "720": "pill bottle",
+    "721": "pillow",
+    "722": "ping-pong ball",
+    "723": "pinwheel",
+    "724": "pirate, pirate ship",
+    "725": "pitcher, ewer",
+    "726": "plane, carpenter's plane, woodworking plane",
+    "727": "planetarium",
+    "728": "plastic bag",
+    "729": "plate rack",
+    "730": "plow, plough",
+    "731": "plunger, plumber's helper",
+    "732": "Polaroid camera, Polaroid Land camera",
+    "733": "pole",
+    "734": "police van, police wagon, paddy wagon, patrol wagon, wagon, black Maria",
+    "735": "poncho",
+    "736": "pool table, billiard table, snooker table",
+    "737": "pop bottle, soda bottle",
+    "738": "pot, flowerpot",
+    "739": "potter's wheel",
+    "740": "power drill",
+    "741": "prayer rug, prayer mat",
+    "742": "printer",
+    "743": "prison, prison house",
+    "744": "projectile, missile",
+    "745": "projector",
+    "746": "puck, hockey puck",
+    "747": "punching bag, punch bag, punching ball, punchball",
+    "748": "purse",
+    "749": "quill, quill pen",
+    "750": "quilt, comforter, comfort, puff",
+    "751": "racer, race car, racing car",
+    "752": "racket, racquet",
+    "753": "radiator",
+    "754": "radio, wireless",
+    "755": "radio telescope, radio reflector",
+    "756": "rain barrel",
+    "757": "recreational vehicle, RV, R.V.",
+    "758": "reel",
+    "759": "reflex camera",
+    "760": "refrigerator, icebox",
+    "761": "remote control, remote",
+    "762": "restaurant, eating house, eating place, eatery",
+    "763": "revolver, six-gun, six-shooter",
+    "764": "rifle",
+    "765": "rocking chair, rocker",
+    "766": "rotisserie",
+    "767": "rubber eraser, rubber, pencil eraser",
+    "768": "rugby ball",
+    "769": "rule, ruler",
+    "770": "running shoe",
+    "771": "safe",
+    "772": "safety pin",
+    "773": "saltshaker, salt shaker",
+    "774": "sandal",
+    "775": "sarong",
+    "776": "sax, saxophone",
+    "777": "scabbard",
+    "778": "scale, weighing machine",
+    "779": "school bus",
+    "780": "schooner",
+    "781": "scoreboard",
+    "782": "screen, CRT screen",
+    "783": "screw",
+    "784": "screwdriver",
+    "785": "seat belt, seatbelt",
+    "786": "sewing machine",
+    "787": "shield, buckler",
+    "788": "shoe shop, shoe-shop, shoe store",
+    "789": "shoji",
+    "790": "shopping basket",
+    "791": "shopping cart",
+    "792": "shovel",
+    "793": "shower cap",
+    "794": "shower curtain",
+    "795": "ski",
+    "796": "ski mask",
+    "797": "sleeping bag",
+    "798": "slide rule, slipstick",
+    "799": "sliding door",
+    "800": "slot, one-armed bandit",
+    "801": "snorkel",
+    "802": "snowmobile",
+    "803": "snowplow, snowplough",
+    "804": "soap dispenser",
+    "805": "soccer ball",
+    "806": "sock",
+    "807": "solar dish, solar collector, solar furnace",
+    "808": "sombrero",
+    "809": "soup bowl",
+    "810": "space bar",
+    "811": "space heater",
+    "812": "space shuttle",
+    "813": "spatula",
+    "814": "speedboat",
+    "815": "spider web, spider's web",
+    "816": "spindle",
+    "817": "sports car, sport car",
+    "818": "spotlight, spot",
+    "819": "stage",
+    "820": "steam locomotive",
+    "821": "steel arch bridge",
+    "822": "steel drum",
+    "823": "stethoscope",
+    "824": "stole",
+    "825": "stone wall",
+    "826": "stopwatch, stop watch",
+    "827": "stove",
+    "828": "strainer",
+    "829": "streetcar, tram, tramcar, trolley, trolley car",
+    "830": "stretcher",
+    "831": "studio couch, day bed",
+    "832": "stupa, tope",
+    "833": "submarine, pigboat, sub, U-boat",
+    "834": "suit, suit of clothes",
+    "835": "sundial",
+    "836": "sunglass",
+    "837": "sunglasses, dark glasses, shades",
+    "838": "sunscreen, sunblock, sun blocker",
+    "839": "suspension bridge",
+    "840": "swab, swob, mop",
+    "841": "sweatshirt",
+    "842": "swimming trunks, bathing trunks",
+    "843": "swing",
+    "844": "switch, electric switch, electrical switch",
+    "845": "syringe",
+    "846": "table lamp",
+    "847": "tank, army tank, armored combat vehicle, armoured combat vehicle",
+    "848": "tape player",
+    "849": "teapot",
+    "850": "teddy, teddy bear",
+    "851": "television, television system",
+    "852": "tennis ball",
+    "853": "thatch, thatched roof",
+    "854": "theater curtain, theatre curtain",
+    "855": "thimble",
+    "856": "thresher, thrasher, threshing machine",
+    "857": "throne",
+    "858": "tile roof",
+    "859": "toaster",
+    "860": "tobacco shop, tobacconist shop, tobacconist",
+    "861": "toilet seat",
+    "862": "torch",
+    "863": "totem pole",
+    "864": "tow truck, tow car, wrecker",
+    "865": "toyshop",
+    "866": "tractor",
+    "867": "trailer truck, tractor trailer, trucking rig, rig, articulated lorry, semi",
+    "868": "tray",
+    "869": "trench coat",
+    "870": "tricycle, trike, velocipede",
+    "871": "trimaran",
+    "872": "tripod",
+    "873": "triumphal arch",
+    "874": "trolleybus, trolley coach, trackless trolley",
+    "875": "trombone",
+    "876": "tub, vat",
+    "877": "turnstile",
+    "878": "typewriter keyboard",
+    "879": "umbrella",
+    "880": "unicycle, monocycle",
+    "881": "upright, upright piano",
+    "882": "vacuum, vacuum cleaner",
+    "883": "vase",
+    "884": "vault",
+    "885": "velvet",
+    "886": "vending machine",
+    "887": "vestment",
+    "888": "viaduct",
+    "889": "violin, fiddle",
+    "890": "volleyball",
+    "891": "waffle iron",
+    "892": "wall clock",
+    "893": "wallet, billfold, notecase, pocketbook",
+    "894": "wardrobe, closet, press",
+    "895": "warplane, military plane",
+    "896": "washbasin, handbasin, washbowl, lavabo, wash-hand basin",
+    "897": "washer, automatic washer, washing machine",
+    "898": "water bottle",
+    "899": "water jug",
+    "900": "water tower",
+    "901": "whiskey jug",
+    "902": "whistle",
+    "903": "wig",
+    "904": "window screen",
+    "905": "window shade",
+    "906": "Windsor tie",
+    "907": "wine bottle",
+    "908": "wing",
+    "909": "wok",
+    "910": "wooden spoon",
+    "911": "wool, woolen, woollen",
+    "912": "worm fence, snake fence, snake-rail fence, Virginia fence",
+    "913": "wreck",
+    "914": "yawl",
+    "915": "yurt",
+    "916": "web site, website, internet site, site",
+    "917": "comic book",
+    "918": "crossword puzzle, crossword",
+    "919": "street sign",
+    "920": "traffic light, traffic signal, stoplight",
+    "921": "book jacket, dust cover, dust jacket, dust wrapper",
+    "922": "menu",
+    "923": "plate",
+    "924": "guacamole",
+    "925": "consomme",
+    "926": "hot pot, hotpot",
+    "927": "trifle",
+    "928": "ice cream, icecream",
+    "929": "ice lolly, lolly, lollipop, popsicle",
+    "930": "French loaf",
+    "931": "bagel, beigel",
+    "932": "pretzel",
+    "933": "cheeseburger",
+    "934": "hotdog, hot dog, red hot",
+    "935": "mashed potato",
+    "936": "head cabbage",
+    "937": "broccoli",
+    "938": "cauliflower",
+    "939": "zucchini, courgette",
+    "940": "spaghetti squash",
+    "941": "acorn squash",
+    "942": "butternut squash",
+    "943": "cucumber, cuke",
+    "944": "artichoke, globe artichoke",
+    "945": "bell pepper",
+    "946": "cardoon",
+    "947": "mushroom",
+    "948": "Granny Smith",
+    "949": "strawberry",
+    "950": "orange",
+    "951": "lemon",
+    "952": "fig",
+    "953": "pineapple, ananas",
+    "954": "banana",
+    "955": "jackfruit, jak, jack",
+    "956": "custard apple",
+    "957": "pomegranate",
+    "958": "hay",
+    "959": "carbonara",
+    "960": "chocolate sauce, chocolate syrup",
+    "961": "dough",
+    "962": "meat loaf, meatloaf",
+    "963": "pizza, pizza pie",
+    "964": "potpie",
+    "965": "burrito",
+    "966": "red wine",
+    "967": "espresso",
+    "968": "cup",
+    "969": "eggnog",
+    "970": "alp",
+    "971": "bubble",
+    "972": "cliff, drop, drop-off",
+    "973": "coral reef",
+    "974": "geyser",
+    "975": "lakeside, lakeshore",
+    "976": "promontory, headland, head, foreland",
+    "977": "sandbar, sand bar",
+    "978": "seashore, coast, seacoast, sea-coast",
+    "979": "valley, vale",
+    "980": "volcano",
+    "981": "ballplayer, baseball player",
+    "982": "groom, bridegroom",
+    "983": "scuba diver",
+    "984": "rapeseed",
+    "985": "daisy",
+    "986": "yellow lady's slipper, yellow lady-slipper, Cypripedium calceolus, Cypripedium parviflorum",
+    "987": "corn",
+    "988": "acorn",
+    "989": "hip, rose hip, rosehip",
+    "990": "buckeye, horse chestnut, conker",
+    "991": "coral fungus",
+    "992": "agaric",
+    "993": "gyromitra",
+    "994": "stinkhorn, carrion fungus",
+    "995": "earthstar",
+    "996": "hen-of-the-woods, hen of the woods, Polyporus frondosus, Grifola frondosa",
+    "997": "bolete",
+    "998": "ear, spike, capitulum",
+    "999": "toilet tissue, toilet paper, bathroom tissue"
+  },
+  "image_size": 224,
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "label2id": {
+    "Afghan hound, Afghan": 160,
+    "African chameleon, Chamaeleo chamaeleon": 47,
+    "African crocodile, Nile crocodile, Crocodylus niloticus": 49,
+    "African elephant, Loxodonta africana": 386,
+    "African grey, African gray, Psittacus erithacus": 87,
+    "African hunting dog, hyena dog, Cape hunting dog, Lycaon pictus": 275,
+    "Airedale, Airedale terrier": 191,
+    "American Staffordshire terrier, Staffordshire terrier, American pit bull terrier, pit bull terrier": 180,
+    "American alligator, Alligator mississipiensis": 50,
+    "American black bear, black bear, Ursus americanus, Euarctos americanus": 295,
+    "American chameleon, anole, Anolis carolinensis": 40,
+    "American coot, marsh hen, mud hen, water hen, Fulica americana": 137,
+    "American egret, great white heron, Egretta albus": 132,
+    "American lobster, Northern lobster, Maine lobster, Homarus americanus": 122,
+    "Angora, Angora rabbit": 332,
+    "Appenzeller": 240,
+    "Arabian camel, dromedary, Camelus dromedarius": 354,
+    "Arctic fox, white fox, Alopex lagopus": 279,
+    "Australian terrier": 193,
+    "Band Aid": 419,
+    "Bedlington terrier": 181,
+    "Bernese mountain dog": 239,
+    "Blenheim spaniel": 156,
+    "Border collie": 232,
+    "Border terrier": 182,
+    "Boston bull, Boston terrier": 195,
+    "Bouvier des Flandres, Bouviers des Flandres": 233,
+    "Brabancon griffon": 262,
+    "Brittany spaniel": 215,
+    "CD player": 485,
+    "Cardigan, Cardigan Welsh corgi": 264,
+    "Chesapeake Bay retriever": 209,
+    "Chihuahua": 151,
+    "Christmas stocking": 496,
+    "Crock Pot": 521,
+    "Dandie Dinmont, Dandie Dinmont terrier": 194,
+    "Doberman, Doberman pinscher": 236,
+    "Dungeness crab, Cancer magister": 118,
+    "Dutch oven": 544,
+    "Egyptian cat": 285,
+    "English foxhound": 167,
+    "English setter": 212,
+    "English springer, English springer spaniel": 217,
+    "EntleBucher": 241,
+    "Eskimo dog, husky": 248,
+    "European fire salamander, Salamandra salamandra": 25,
+    "European gallinule, Porphyrio porphyrio": 136,
+    "French bulldog": 245,
+    "French horn, horn": 566,
+    "French loaf": 930,
+    "German shepherd, German shepherd dog, German police dog, alsatian": 235,
+    "German short-haired pointer": 210,
+    "Gila monster, Heloderma suspectum": 45,
+    "Gordon setter": 214,
+    "Granny Smith": 948,
+    "Great Dane": 246,
+    "Great Pyrenees": 257,
+    "Greater Swiss Mountain dog": 238,
+    "Ibizan hound, Ibizan Podenco": 173,
+    "Indian cobra, Naja naja": 63,
+    "Indian elephant, Elephas maximus": 385,
+    "Irish setter, red setter": 213,
+    "Irish terrier": 184,
+    "Irish water spaniel": 221,
+    "Irish wolfhound": 170,
+    "Italian greyhound": 171,
+    "Japanese spaniel": 152,
+    "Kerry blue terrier": 183,
+    "Komodo dragon, Komodo lizard, dragon lizard, giant lizard, Varanus komodoensis": 48,
+    "Labrador retriever": 208,
+    "Lakeland terrier": 189,
+    "Leonberg": 255,
+    "Lhasa, Lhasa apso": 204,
+    "Loafer": 630,
+    "Madagascar cat, ring-tailed lemur, Lemur catta": 383,
+    "Maltese dog, Maltese terrier, Maltese": 153,
+    "Mexican hairless": 268,
+    "Model T": 661,
+    "Newfoundland, Newfoundland dog": 256,
+    "Norfolk terrier": 185,
+    "Norwegian elkhound, elkhound": 174,
+    "Norwich terrier": 186,
+    "Old English sheepdog, bobtail": 229,
+    "Pekinese, Pekingese, Peke": 154,
+    "Pembroke, Pembroke Welsh corgi": 263,
+    "Persian cat": 283,
+    "Petri dish": 712,
+    "Polaroid camera, Polaroid Land camera": 732,
+    "Pomeranian": 259,
+    "Rhodesian ridgeback": 159,
+    "Rottweiler": 234,
+    "Saint Bernard, St Bernard": 247,
+    "Saluki, gazelle hound": 176,
+    "Samoyed, Samoyede": 258,
+    "Scotch terrier, Scottish terrier, Scottie": 199,
+    "Scottish deerhound, deerhound": 177,
+    "Sealyham terrier, Sealyham": 190,
+    "Shetland sheepdog, Shetland sheep dog, Shetland": 230,
+    "Shih-Tzu": 155,
+    "Siamese cat, Siamese": 284,
+    "Siberian husky": 250,
+    "Staffordshire bullterrier, Staffordshire bull terrier": 179,
+    "Sussex spaniel": 220,
+    "Tibetan mastiff": 244,
+    "Tibetan terrier, chrysanthemum dog": 200,
+    "Walker hound, Walker foxhound": 166,
+    "Weimaraner": 178,
+    "Welsh springer spaniel": 218,
+    "West Highland white terrier": 203,
+    "Windsor tie": 906,
+    "Yorkshire terrier": 187,
+    "abacus": 398,
+    "abaya": 399,
+    "academic gown, academic robe, judge's robe": 400,
+    "accordion, piano accordion, squeeze box": 401,
+    "acorn": 988,
+    "acorn squash": 941,
+    "acoustic guitar": 402,
+    "admiral": 321,
+    "affenpinscher, monkey pinscher, monkey dog": 252,
+    "agama": 42,
+    "agaric": 992,
+    "aircraft carrier, carrier, flattop, attack aircraft carrier": 403,
+    "airliner": 404,
+    "airship, dirigible": 405,
+    "albatross, mollymawk": 146,
+    "alligator lizard": 44,
+    "alp": 970,
+    "altar": 406,
+    "ambulance": 407,
+    "amphibian, amphibious vehicle": 408,
+    "analog clock": 409,
+    "anemone fish": 393,
+    "ant, emmet, pismire": 310,
+    "apiary, bee house": 410,
+    "apron": 411,
+    "armadillo": 363,
+    "artichoke, globe artichoke": 944,
+    "ashcan, trash can, garbage can, wastebin, ash bin, ash-bin, ashbin, dustbin, trash barrel, trash bin": 412,
+    "assault rifle, assault gun": 413,
+    "axolotl, mud puppy, Ambystoma mexicanum": 29,
+    "baboon": 372,
+    "backpack, back pack, knapsack, packsack, rucksack, haversack": 414,
+    "badger": 362,
+    "bagel, beigel": 931,
+    "bakery, bakeshop, bakehouse": 415,
+    "balance beam, beam": 416,
+    "bald eagle, American eagle, Haliaeetus leucocephalus": 22,
+    "balloon": 417,
+    "ballplayer, baseball player": 981,
+    "ballpoint, ballpoint pen, ballpen, Biro": 418,
+    "banana": 954,
+    "banded gecko": 38,
+    "banjo": 420,
+    "bannister, banister, balustrade, balusters, handrail": 421,
+    "barbell": 422,
+    "barber chair": 423,
+    "barbershop": 424,
+    "barn": 425,
+    "barn spider, Araneus cavaticus": 73,
+    "barometer": 426,
+    "barracouta, snoek": 389,
+    "barrel, cask": 427,
+    "barrow, garden cart, lawn cart, wheelbarrow": 428,
+    "baseball": 429,
+    "basenji": 253,
+    "basketball": 430,
+    "basset, basset hound": 161,
+    "bassinet": 431,
+    "bassoon": 432,
+    "bath towel": 434,
+    "bathing cap, swimming cap": 433,
+    "bathtub, bathing tub, bath, tub": 435,
+    "beach wagon, station wagon, wagon, estate car, beach waggon, station waggon, waggon": 436,
+    "beacon, lighthouse, beacon light, pharos": 437,
+    "beagle": 162,
+    "beaker": 438,
+    "bearskin, busby, shako": 439,
+    "beaver": 337,
+    "bee": 309,
+    "bee eater": 92,
+    "beer bottle": 440,
+    "beer glass": 441,
+    "bell cote, bell cot": 442,
+    "bell pepper": 945,
+    "bib": 443,
+    "bicycle-built-for-two, tandem bicycle, tandem": 444,
+    "bighorn, bighorn sheep, cimarron, Rocky Mountain bighorn, Rocky Mountain sheep, Ovis canadensis": 349,
+    "bikini, two-piece": 445,
+    "binder, ring-binder": 446,
+    "binoculars, field glasses, opera glasses": 447,
+    "birdhouse": 448,
+    "bison": 347,
+    "bittern": 133,
+    "black and gold garden spider, Argiope aurantia": 72,
+    "black grouse": 80,
+    "black stork, Ciconia nigra": 128,
+    "black swan, Cygnus atratus": 100,
+    "black widow, Latrodectus mactans": 75,
+    "black-and-tan coonhound": 165,
+    "black-footed ferret, ferret, Mustela nigripes": 359,
+    "bloodhound, sleuthhound": 163,
+    "bluetick": 164,
+    "boa constrictor, Constrictor constrictor": 61,
+    "boathouse": 449,
+    "bobsled, bobsleigh, bob": 450,
+    "bolete": 997,
+    "bolo tie, bolo, bola tie, bola": 451,
+    "bonnet, poke bonnet": 452,
+    "book jacket, dust cover, dust jacket, dust wrapper": 921,
+    "bookcase": 453,
+    "bookshop, bookstore, bookstall": 454,
+    "borzoi, Russian wolfhound": 169,
+    "bottlecap": 455,
+    "bow": 456,
+    "bow tie, bow-tie, bowtie": 457,
+    "box turtle, box tortoise": 37,
+    "boxer": 242,
+    "brain coral": 109,
+    "brambling, Fringilla montifringilla": 10,
+    "brass, memorial tablet, plaque": 458,
+    "brassiere, bra, bandeau": 459,
+    "breakwater, groin, groyne, mole, bulwark, seawall, jetty": 460,
+    "breastplate, aegis, egis": 461,
+    "briard": 226,
+    "broccoli": 937,
+    "broom": 462,
+    "brown bear, bruin, Ursus arctos": 294,
+    "bubble": 971,
+    "bucket, pail": 463,
+    "buckeye, horse chestnut, conker": 990,
+    "buckle": 464,
+    "bulbul": 16,
+    "bull mastiff": 243,
+    "bullet train, bullet": 466,
+    "bulletproof vest": 465,
+    "bullfrog, Rana catesbeiana": 30,
+    "burrito": 965,
+    "bustard": 138,
+    "butcher shop, meat market": 467,
+    "butternut squash": 942,
+    "cab, hack, taxi, taxicab": 468,
+    "cabbage butterfly": 324,
+    "cairn, cairn terrier": 192,
+    "caldron, cauldron": 469,
+    "can opener, tin opener": 473,
+    "candle, taper, wax light": 470,
+    "cannon": 471,
+    "canoe": 472,
+    "capuchin, ringtail, Cebus capucinus": 378,
+    "car mirror": 475,
+    "car wheel": 479,
+    "carbonara": 959,
+    "cardigan": 474,
+    "cardoon": 946,
+    "carousel, carrousel, merry-go-round, roundabout, whirligig": 476,
+    "carpenter's kit, tool kit": 477,
+    "carton": 478,
+    "cash machine, cash dispenser, automated teller machine, automatic teller machine, automated teller, automatic teller, ATM": 480,
+    "cassette": 481,
+    "cassette player": 482,
+    "castle": 483,
+    "catamaran": 484,
+    "cauliflower": 938,
+    "cello, violoncello": 486,
+    "cellular telephone, cellular phone, cellphone, cell, mobile phone": 487,
+    "centipede": 79,
+    "chain": 488,
+    "chain mail, ring mail, mail, chain armor, chain armour, ring armor, ring armour": 490,
+    "chain saw, chainsaw": 491,
+    "chainlink fence": 489,
+    "chambered nautilus, pearly nautilus, nautilus": 117,
+    "cheeseburger": 933,
+    "cheetah, chetah, Acinonyx jubatus": 293,
+    "chest": 492,
+    "chickadee": 19,
+    "chiffonier, commode": 493,
+    "chime, bell, gong": 494,
+    "chimpanzee, chimp, Pan troglodytes": 367,
+    "china cabinet, china closet": 495,
+    "chiton, coat-of-mail shell, sea cradle, polyplacophore": 116,
+    "chocolate sauce, chocolate syrup": 960,
+    "chow, chow chow": 260,
+    "church, church building": 497,
+    "cicada, cicala": 316,
+    "cinema, movie theater, movie theatre, movie house, picture palace": 498,
+    "cleaver, meat cleaver, chopper": 499,
+    "cliff dwelling": 500,
+    "cliff, drop, drop-off": 972,
+    "cloak": 501,
+    "clog, geta, patten, sabot": 502,
+    "clumber, clumber spaniel": 216,
+    "cock": 7,
+    "cocker spaniel, English cocker spaniel, cocker": 219,
+    "cockroach, roach": 314,
+    "cocktail shaker": 503,
+    "coffee mug": 504,
+    "coffeepot": 505,
+    "coho, cohoe, coho salmon, blue jack, silver salmon, Oncorhynchus kisutch": 391,
+    "coil, spiral, volute, whorl, helix": 506,
+    "collie": 231,
+    "colobus, colobus monkey": 375,
+    "combination lock": 507,
+    "comic book": 917,
+    "common iguana, iguana, Iguana iguana": 39,
+    "common newt, Triturus vulgaris": 26,
+    "computer keyboard, keypad": 508,
+    "conch": 112,
+    "confectionery, confectionary, candy store": 509,
+    "consomme": 925,
+    "container ship, containership, container vessel": 510,
+    "convertible": 511,
+    "coral fungus": 991,
+    "coral reef": 973,
+    "corkscrew, bottle screw": 512,
+    "corn": 987,
+    "cornet, horn, trumpet, trump": 513,
+    "coucal": 91,
+    "cougar, puma, catamount, mountain lion, painter, panther, Felis concolor": 286,
+    "cowboy boot": 514,
+    "cowboy hat, ten-gallon hat": 515,
+    "coyote, prairie wolf, brush wolf, Canis latrans": 272,
+    "cradle": 516,
+    "crane": 517,
+    "crash helmet": 518,
+    "crate": 519,
+    "crayfish, crawfish, crawdad, crawdaddy": 124,
+    "crib, cot": 520,
+    "cricket": 312,
+    "croquet ball": 522,
+    "crossword puzzle, crossword": 918,
+    "crutch": 523,
+    "cucumber, cuke": 943,
+    "cuirass": 524,
+    "cup": 968,
+    "curly-coated retriever": 206,
+    "custard apple": 956,
+    "daisy": 985,
+    "dalmatian, coach dog, carriage dog": 251,
+    "dam, dike, dyke": 525,
+    "damselfly": 320,
+    "desk": 526,
+    "desktop computer": 527,
+    "dhole, Cuon alpinus": 274,
+    "dial telephone, dial phone": 528,
+    "diamondback, diamondback rattlesnake, Crotalus adamanteus": 67,
+    "diaper, nappy, napkin": 529,
+    "digital clock": 530,
+    "digital watch": 531,
+    "dingo, warrigal, warragal, Canis dingo": 273,
+    "dining table, board": 532,
+    "dishrag, dishcloth": 533,
+    "dishwasher, dish washer, dishwashing machine": 534,
+    "disk brake, disc brake": 535,
+    "dock, dockage, docking facility": 536,
+    "dogsled, dog sled, dog sleigh": 537,
+    "dome": 538,
+    "doormat, welcome mat": 539,
+    "dough": 961,
+    "dowitcher": 142,
+    "dragonfly, darning needle, devil's darning needle, sewing needle, snake feeder, snake doctor, mosquito hawk, skeeter hawk": 319,
+    "drake": 97,
+    "drilling platform, offshore rig": 540,
+    "drum, membranophone, tympan": 541,
+    "drumstick": 542,
+    "dugong, Dugong dugon": 149,
+    "dumbbell": 543,
+    "dung beetle": 305,
+    "ear, spike, capitulum": 998,
+    "earthstar": 995,
+    "echidna, spiny anteater, anteater": 102,
+    "eel": 390,
+    "eft": 27,
+    "eggnog": 969,
+    "electric fan, blower": 545,
+    "electric guitar": 546,
+    "electric locomotive": 547,
+    "electric ray, crampfish, numbfish, torpedo": 5,
+    "entertainment center": 548,
+    "envelope": 549,
+    "espresso": 967,
+    "espresso maker": 550,
+    "face powder": 551,
+    "feather boa, boa": 552,
+    "fiddler crab": 120,
+    "fig": 952,
+    "file, file cabinet, filing cabinet": 553,
+    "fire engine, fire truck": 555,
+    "fire screen, fireguard": 556,
+    "fireboat": 554,
+    "flagpole, flagstaff": 557,
+    "flamingo": 130,
+    "flat-coated retriever": 205,
+    "flatworm, platyhelminth": 110,
+    "flute, transverse flute": 558,
+    "fly": 308,
+    "folding chair": 559,
+    "football helmet": 560,
+    "forklift": 561,
+    "fountain": 562,
+    "fountain pen": 563,
+    "four-poster": 564,
+    "fox squirrel, eastern fox squirrel, Sciurus niger": 335,
+    "freight car": 565,
+    "frilled lizard, Chlamydosaurus kingi": 43,
+    "frying pan, frypan, skillet": 567,
+    "fur coat": 568,
+    "gar, garfish, garpike, billfish, Lepisosteus osseus": 395,
+    "garbage truck, dustcart": 569,
+    "garden spider, Aranea diademata": 74,
+    "garter snake, grass snake": 57,
+    "gas pump, gasoline pump, petrol pump, island dispenser": 571,
+    "gasmask, respirator, gas helmet": 570,
+    "gazelle": 353,
+    "geyser": 974,
+    "giant panda, panda, panda bear, coon bear, Ailuropoda melanoleuca": 388,
+    "giant schnauzer": 197,
+    "gibbon, Hylobates lar": 368,
+    "go-kart": 573,
+    "goblet": 572,
+    "golden retriever": 207,
+    "goldfinch, Carduelis carduelis": 11,
+    "goldfish, Carassius auratus": 1,
+    "golf ball": 574,
+    "golfcart, golf cart": 575,
+    "gondola": 576,
+    "gong, tam-tam": 577,
+    "goose": 99,
+    "gorilla, Gorilla gorilla": 366,
+    "gown": 578,
+    "grand piano, grand": 579,
+    "grasshopper, hopper": 311,
+    "great grey owl, great gray owl, Strix nebulosa": 24,
+    "great white shark, white shark, man-eater, man-eating shark, Carcharodon carcharias": 2,
+    "green lizard, Lacerta viridis": 46,
+    "green mamba": 64,
+    "green snake, grass snake": 55,
+    "greenhouse, nursery, glasshouse": 580,
+    "grey fox, gray fox, Urocyon cinereoargenteus": 280,
+    "grey whale, gray whale, devilfish, Eschrichtius gibbosus, Eschrichtius robustus": 147,
+    "grille, radiator grille": 581,
+    "grocery store, grocery, food market, market": 582,
+    "groenendael": 224,
+    "groom, bridegroom": 982,
+    "ground beetle, carabid beetle": 302,
+    "guacamole": 924,
+    "guenon, guenon monkey": 370,
+    "guillotine": 583,
+    "guinea pig, Cavia cobaya": 338,
+    "gyromitra": 993,
+    "hair slide": 584,
+    "hair spray": 585,
+    "half track": 586,
+    "hammer": 587,
+    "hammerhead, hammerhead shark": 4,
+    "hamper": 588,
+    "hamster": 333,
+    "hand blower, blow dryer, blow drier, hair dryer, hair drier": 589,
+    "hand-held computer, hand-held microcomputer": 590,
+    "handkerchief, hankie, hanky, hankey": 591,
+    "hard disc, hard disk, fixed disk": 592,
+    "hare": 331,
+    "harmonica, mouth organ, harp, mouth harp": 593,
+    "harp": 594,
+    "hartebeest": 351,
+    "harvester, reaper": 595,
+    "harvestman, daddy longlegs, Phalangium opilio": 70,
+    "hatchet": 596,
+    "hay": 958,
+    "head cabbage": 936,
+    "hen": 8,
+    "hen-of-the-woods, hen of the woods, Polyporus frondosus, Grifola frondosa": 996,
+    "hermit crab": 125,
+    "hip, rose hip, rosehip": 989,
+    "hippopotamus, hippo, river horse, Hippopotamus amphibius": 344,
+    "hog, pig, grunter, squealer, Sus scrofa": 341,
+    "hognose snake, puff adder, sand viper": 54,
+    "holster": 597,
+    "home theater, home theatre": 598,
+    "honeycomb": 599,
+    "hook, claw": 600,
+    "hoopskirt, crinoline": 601,
+    "horizontal bar, high bar": 602,
+    "hornbill": 93,
+    "horned viper, cerastes, sand viper, horned asp, Cerastes cornutus": 66,
+    "horse cart, horse-cart": 603,
+    "hot pot, hotpot": 926,
+    "hotdog, hot dog, red hot": 934,
+    "hourglass": 604,
+    "house finch, linnet, Carpodacus mexicanus": 12,
+    "howler monkey, howler": 379,
+    "hummingbird": 94,
+    "hyena, hyaena": 276,
+    "iPod": 605,
+    "ibex, Capra ibex": 350,
+    "ice bear, polar bear, Ursus Maritimus, Thalarctos maritimus": 296,
+    "ice cream, icecream": 928,
+    "ice lolly, lolly, lollipop, popsicle": 929,
+    "impala, Aepyceros melampus": 352,
+    "indigo bunting, indigo finch, indigo bird, Passerina cyanea": 14,
+    "indri, indris, Indri indri, Indri brevicaudatus": 384,
+    "iron, smoothing iron": 606,
+    "isopod": 126,
+    "jacamar": 95,
+    "jack-o'-lantern": 607,
+    "jackfruit, jak, jack": 955,
+    "jaguar, panther, Panthera onca, Felis onca": 290,
+    "jay": 17,
+    "jean, blue jean, denim": 608,
+    "jeep, landrover": 609,
+    "jellyfish": 107,
+    "jersey, T-shirt, tee shirt": 610,
+    "jigsaw puzzle": 611,
+    "jinrikisha, ricksha, rickshaw": 612,
+    "joystick": 613,
+    "junco, snowbird": 13,
+    "keeshond": 261,
+    "kelpie": 227,
+    "killer whale, killer, orca, grampus, sea wolf, Orcinus orca": 148,
+    "kimono": 614,
+    "king crab, Alaska crab, Alaskan king crab, Alaska king crab, Paralithodes camtschatica": 121,
+    "king penguin, Aptenodytes patagonica": 145,
+    "king snake, kingsnake": 56,
+    "kit fox, Vulpes macrotis": 278,
+    "kite": 21,
+    "knee pad": 615,
+    "knot": 616,
+    "koala, koala bear, kangaroo bear, native bear, Phascolarctos cinereus": 105,
+    "komondor": 228,
+    "kuvasz": 222,
+    "lab coat, laboratory coat": 617,
+    "lacewing, lacewing fly": 318,
+    "ladle": 618,
+    "ladybug, ladybeetle, lady beetle, ladybird, ladybird beetle": 301,
+    "lakeside, lakeshore": 975,
+    "lampshade, lamp shade": 619,
+    "langur": 374,
+    "laptop, laptop computer": 620,
+    "lawn mower, mower": 621,
+    "leaf beetle, chrysomelid": 304,
+    "leafhopper": 317,
+    "leatherback turtle, leatherback, leathery turtle, Dermochelys coriacea": 34,
+    "lemon": 951,
+    "lens cap, lens cover": 622,
+    "leopard, Panthera pardus": 288,
+    "lesser panda, red panda, panda, bear cat, cat bear, Ailurus fulgens": 387,
+    "letter opener, paper knife, paperknife": 623,
+    "library": 624,
+    "lifeboat": 625,
+    "lighter, light, igniter, ignitor": 626,
+    "limousine, limo": 627,
+    "limpkin, Aramus pictus": 135,
+    "liner, ocean liner": 628,
+    "lion, king of beasts, Panthera leo": 291,
+    "lionfish": 396,
+    "lipstick, lip rouge": 629,
+    "little blue heron, Egretta caerulea": 131,
+    "llama": 355,
+    "loggerhead, loggerhead turtle, Caretta caretta": 33,
+    "long-horned beetle, longicorn, longicorn beetle": 303,
+    "lorikeet": 90,
+    "lotion": 631,
+    "loudspeaker, speaker, speaker unit, loudspeaker system, speaker system": 632,
+    "loupe, jeweler's loupe": 633,
+    "lumbermill, sawmill": 634,
+    "lycaenid, lycaenid butterfly": 326,
+    "lynx, catamount": 287,
+    "macaque": 373,
+    "macaw": 88,
+    "magnetic compass": 635,
+    "magpie": 18,
+    "mailbag, postbag": 636,
+    "mailbox, letter box": 637,
+    "maillot": 638,
+    "maillot, tank suit": 639,
+    "malamute, malemute, Alaskan malamute": 249,
+    "malinois": 225,
+    "manhole cover": 640,
+    "mantis, mantid": 315,
+    "maraca": 641,
+    "marimba, xylophone": 642,
+    "marmoset": 377,
+    "marmot": 336,
+    "mashed potato": 935,
+    "mask": 643,
+    "matchstick": 644,
+    "maypole": 645,
+    "maze, labyrinth": 646,
+    "measuring cup": 647,
+    "meat loaf, meatloaf": 962,
+    "medicine chest, medicine cabinet": 648,
+    "meerkat, mierkat": 299,
+    "megalith, megalithic structure": 649,
+    "menu": 922,
+    "microphone, mike": 650,
+    "microwave, microwave oven": 651,
+    "military uniform": 652,
+    "milk can": 653,
+    "miniature pinscher": 237,
+    "miniature poodle": 266,
+    "miniature schnauzer": 196,
+    "minibus": 654,
+    "miniskirt, mini": 655,
+    "minivan": 656,
+    "mink": 357,
+    "missile": 657,
+    "mitten": 658,
+    "mixing bowl": 659,
+    "mobile home, manufactured home": 660,
+    "modem": 662,
+    "monarch, monarch butterfly, milkweed butterfly, Danaus plexippus": 323,
+    "monastery": 663,
+    "mongoose": 298,
+    "monitor": 664,
+    "moped": 665,
+    "mortar": 666,
+    "mortarboard": 667,
+    "mosque": 668,
+    "mosquito net": 669,
+    "motor scooter, scooter": 670,
+    "mountain bike, all-terrain bike, off-roader": 671,
+    "mountain tent": 672,
+    "mouse, computer mouse": 673,
+    "mousetrap": 674,
+    "moving van": 675,
+    "mud turtle": 35,
+    "mushroom": 947,
+    "muzzle": 676,
+    "nail": 677,
+    "neck brace": 678,
+    "necklace": 679,
+    "nematode, nematode worm, roundworm": 111,
+    "night snake, Hypsiglena torquata": 60,
+    "nipple": 680,
+    "notebook, notebook computer": 681,
+    "obelisk": 682,
+    "oboe, hautboy, hautbois": 683,
+    "ocarina, sweet potato": 684,
+    "odometer, hodometer, mileometer, milometer": 685,
+    "oil filter": 686,
+    "orange": 950,
+    "orangutan, orang, orangutang, Pongo pygmaeus": 365,
+    "organ, pipe organ": 687,
+    "oscilloscope, scope, cathode-ray oscilloscope, CRO": 688,
+    "ostrich, Struthio camelus": 9,
+    "otter": 360,
+    "otterhound, otter hound": 175,
+    "overskirt": 689,
+    "ox": 345,
+    "oxcart": 690,
+    "oxygen mask": 691,
+    "oystercatcher, oyster catcher": 143,
+    "packet": 692,
+    "paddle, boat paddle": 693,
+    "paddlewheel, paddle wheel": 694,
+    "padlock": 695,
+    "paintbrush": 696,
+    "pajama, pyjama, pj's, jammies": 697,
+    "palace": 698,
+    "panpipe, pandean pipe, syrinx": 699,
+    "paper towel": 700,
+    "papillon": 157,
+    "parachute, chute": 701,
+    "parallel bars, bars": 702,
+    "park bench": 703,
+    "parking meter": 704,
+    "partridge": 86,
+    "passenger car, coach, carriage": 705,
+    "patas, hussar monkey, Erythrocebus patas": 371,
+    "patio, terrace": 706,
+    "pay-phone, pay-station": 707,
+    "peacock": 84,
+    "pedestal, plinth, footstall": 708,
+    "pelican": 144,
+    "pencil box, pencil case": 709,
+    "pencil sharpener": 710,
+    "perfume, essence": 711,
+    "photocopier": 713,
+    "pick, plectrum, plectron": 714,
+    "pickelhaube": 715,
+    "picket fence, paling": 716,
+    "pickup, pickup truck": 717,
+    "pier": 718,
+    "piggy bank, penny bank": 719,
+    "pill bottle": 720,
+    "pillow": 721,
+    "pineapple, ananas": 953,
+    "ping-pong ball": 722,
+    "pinwheel": 723,
+    "pirate, pirate ship": 724,
+    "pitcher, ewer": 725,
+    "pizza, pizza pie": 963,
+    "plane, carpenter's plane, woodworking plane": 726,
+    "planetarium": 727,
+    "plastic bag": 728,
+    "plate": 923,
+    "plate rack": 729,
+    "platypus, duckbill, duckbilled platypus, duck-billed platypus, Ornithorhynchus anatinus": 103,
+    "plow, plough": 730,
+    "plunger, plumber's helper": 731,
+    "pole": 733,
+    "polecat, fitch, foulmart, foumart, Mustela putorius": 358,
+    "police van, police wagon, paddy wagon, patrol wagon, wagon, black Maria": 734,
+    "pomegranate": 957,
+    "poncho": 735,
+    "pool table, billiard table, snooker table": 736,
+    "pop bottle, soda bottle": 737,
+    "porcupine, hedgehog": 334,
+    "pot, flowerpot": 738,
+    "potpie": 964,
+    "potter's wheel": 739,
+    "power drill": 740,
+    "prairie chicken, prairie grouse, prairie fowl": 83,
+    "prayer rug, prayer mat": 741,
+    "pretzel": 932,
+    "printer": 742,
+    "prison, prison house": 743,
+    "proboscis monkey, Nasalis larvatus": 376,
+    "projectile, missile": 744,
+    "projector": 745,
+    "promontory, headland, head, foreland": 976,
+    "ptarmigan": 81,
+    "puck, hockey puck": 746,
+    "puffer, pufferfish, blowfish, globefish": 397,
+    "pug, pug-dog": 254,
+    "punching bag, punch bag, punching ball, punchball": 747,
+    "purse": 748,
+    "quail": 85,
+    "quill, quill pen": 749,
+    "quilt, comforter, comfort, puff": 750,
+    "racer, race car, racing car": 751,
+    "racket, racquet": 752,
+    "radiator": 753,
+    "radio telescope, radio reflector": 755,
+    "radio, wireless": 754,
+    "rain barrel": 756,
+    "ram, tup": 348,
+    "rapeseed": 984,
+    "recreational vehicle, RV, R.V.": 757,
+    "red fox, Vulpes vulpes": 277,
+    "red wine": 966,
+    "red wolf, maned wolf, Canis rufus, Canis niger": 271,
+    "red-backed sandpiper, dunlin, Erolia alpina": 140,
+    "red-breasted merganser, Mergus serrator": 98,
+    "redbone": 168,
+    "redshank, Tringa totanus": 141,
+    "reel": 758,
+    "reflex camera": 759,
+    "refrigerator, icebox": 760,
+    "remote control, remote": 761,
+    "restaurant, eating house, eating place, eatery": 762,
+    "revolver, six-gun, six-shooter": 763,
+    "rhinoceros beetle": 306,
+    "rifle": 764,
+    "ringlet, ringlet butterfly": 322,
+    "ringneck snake, ring-necked snake, ring snake": 53,
+    "robin, American robin, Turdus migratorius": 15,
+    "rock beauty, Holocanthus tricolor": 392,
+    "rock crab, Cancer irroratus": 119,
+    "rock python, rock snake, Python sebae": 62,
+    "rocking chair, rocker": 765,
+    "rotisserie": 766,
+    "rubber eraser, rubber, pencil eraser": 767,
+    "ruddy turnstone, Arenaria interpres": 139,
+    "ruffed grouse, partridge, Bonasa umbellus": 82,
+    "rugby ball": 768,
+    "rule, ruler": 769,
+    "running shoe": 770,
+    "safe": 771,
+    "safety pin": 772,
+    "saltshaker, salt shaker": 773,
+    "sandal": 774,
+    "sandbar, sand bar": 977,
+    "sarong": 775,
+    "sax, saxophone": 776,
+    "scabbard": 777,
+    "scale, weighing machine": 778,
+    "schipperke": 223,
+    "school bus": 779,
+    "schooner": 780,
+    "scoreboard": 781,
+    "scorpion": 71,
+    "screen, CRT screen": 782,
+    "screw": 783,
+    "screwdriver": 784,
+    "scuba diver": 983,
+    "sea anemone, anemone": 108,
+    "sea cucumber, holothurian": 329,
+    "sea lion": 150,
+    "sea slug, nudibranch": 115,
+    "sea snake": 65,
+    "sea urchin": 328,
+    "seashore, coast, seacoast, sea-coast": 978,
+    "seat belt, seatbelt": 785,
+    "sewing machine": 786,
+    "shield, buckler": 787,
+    "shoe shop, shoe-shop, shoe store": 788,
+    "shoji": 789,
+    "shopping basket": 790,
+    "shopping cart": 791,
+    "shovel": 792,
+    "shower cap": 793,
+    "shower curtain": 794,
+    "siamang, Hylobates syndactylus, Symphalangus syndactylus": 369,
+    "sidewinder, horned rattlesnake, Crotalus cerastes": 68,
+    "silky terrier, Sydney silky": 201,
+    "ski": 795,
+    "ski mask": 796,
+    "skunk, polecat, wood pussy": 361,
+    "sleeping bag": 797,
+    "slide rule, slipstick": 798,
+    "sliding door": 799,
+    "slot, one-armed bandit": 800,
+    "sloth bear, Melursus ursinus, Ursus ursinus": 297,
+    "slug": 114,
+    "snail": 113,
+    "snorkel": 801,
+    "snow leopard, ounce, Panthera uncia": 289,
+    "snowmobile": 802,
+    "snowplow, snowplough": 803,
+    "soap dispenser": 804,
+    "soccer ball": 805,
+    "sock": 806,
+    "soft-coated wheaten terrier": 202,
+    "solar dish, solar collector, solar furnace": 807,
+    "sombrero": 808,
+    "sorrel": 339,
+    "soup bowl": 809,
+    "space bar": 810,
+    "space heater": 811,
+    "space shuttle": 812,
+    "spaghetti squash": 940,
+    "spatula": 813,
+    "speedboat": 814,
+    "spider monkey, Ateles geoffroyi": 381,
+    "spider web, spider's web": 815,
+    "spindle": 816,
+    "spiny lobster, langouste, rock lobster, crawfish, crayfish, sea crawfish": 123,
+    "spoonbill": 129,
+    "sports car, sport car": 817,
+    "spotlight, spot": 818,
+    "spotted salamander, Ambystoma maculatum": 28,
+    "squirrel monkey, Saimiri sciureus": 382,
+    "stage": 819,
+    "standard poodle": 267,
+    "standard schnauzer": 198,
+    "starfish, sea star": 327,
+    "steam locomotive": 820,
+    "steel arch bridge": 821,
+    "steel drum": 822,
+    "stethoscope": 823,
+    "stingray": 6,
+    "stinkhorn, carrion fungus": 994,
+    "stole": 824,
+    "stone wall": 825,
+    "stopwatch, stop watch": 826,
+    "stove": 827,
+    "strainer": 828,
+    "strawberry": 949,
+    "street sign": 919,
+    "streetcar, tram, tramcar, trolley, trolley car": 829,
+    "stretcher": 830,
+    "studio couch, day bed": 831,
+    "stupa, tope": 832,
+    "sturgeon": 394,
+    "submarine, pigboat, sub, U-boat": 833,
+    "suit, suit of clothes": 834,
+    "sulphur butterfly, sulfur butterfly": 325,
+    "sulphur-crested cockatoo, Kakatoe galerita, Cacatua galerita": 89,
+    "sundial": 835,
+    "sunglass": 836,
+    "sunglasses, dark glasses, shades": 837,
+    "sunscreen, sunblock, sun blocker": 838,
+    "suspension bridge": 839,
+    "swab, swob, mop": 840,
+    "sweatshirt": 841,
+    "swimming trunks, bathing trunks": 842,
+    "swing": 843,
+    "switch, electric switch, electrical switch": 844,
+    "syringe": 845,
+    "tabby, tabby cat": 281,
+    "table lamp": 846,
+    "tailed frog, bell toad, ribbed toad, tailed toad, Ascaphus trui": 32,
+    "tank, army tank, armored combat vehicle, armoured combat vehicle": 847,
+    "tape player": 848,
+    "tarantula": 76,
+    "teapot": 849,
+    "teddy, teddy bear": 850,
+    "television, television system": 851,
+    "tench, Tinca tinca": 0,
+    "tennis ball": 852,
+    "terrapin": 36,
+    "thatch, thatched roof": 853,
+    "theater curtain, theatre curtain": 854,
+    "thimble": 855,
+    "three-toed sloth, ai, Bradypus tridactylus": 364,
+    "thresher, thrasher, threshing machine": 856,
+    "throne": 857,
+    "thunder snake, worm snake, Carphophis amoenus": 52,
+    "tick": 78,
+    "tiger beetle": 300,
+    "tiger cat": 282,
+    "tiger shark, Galeocerdo cuvieri": 3,
+    "tiger, Panthera tigris": 292,
+    "tile roof": 858,
+    "timber wolf, grey wolf, gray wolf, Canis lupus": 269,
+    "titi, titi monkey": 380,
+    "toaster": 859,
+    "tobacco shop, tobacconist shop, tobacconist": 860,
+    "toilet seat": 861,
+    "toilet tissue, toilet paper, bathroom tissue": 999,
+    "torch": 862,
+    "totem pole": 863,
+    "toucan": 96,
+    "tow truck, tow car, wrecker": 864,
+    "toy poodle": 265,
+    "toy terrier": 158,
+    "toyshop": 865,
+    "tractor": 866,
+    "traffic light, traffic signal, stoplight": 920,
+    "trailer truck, tractor trailer, trucking rig, rig, articulated lorry, semi": 867,
+    "tray": 868,
+    "tree frog, tree-frog": 31,
+    "trench coat": 869,
+    "triceratops": 51,
+    "tricycle, trike, velocipede": 870,
+    "trifle": 927,
+    "trilobite": 69,
+    "trimaran": 871,
+    "tripod": 872,
+    "triumphal arch": 873,
+    "trolleybus, trolley coach, trackless trolley": 874,
+    "trombone": 875,
+    "tub, vat": 876,
+    "turnstile": 877,
+    "tusker": 101,
+    "typewriter keyboard": 878,
+    "umbrella": 879,
+    "unicycle, monocycle": 880,
+    "upright, upright piano": 881,
+    "vacuum, vacuum cleaner": 882,
+    "valley, vale": 979,
+    "vase": 883,
+    "vault": 884,
+    "velvet": 885,
+    "vending machine": 886,
+    "vestment": 887,
+    "viaduct": 888,
+    "vine snake": 59,
+    "violin, fiddle": 889,
+    "vizsla, Hungarian pointer": 211,
+    "volcano": 980,
+    "volleyball": 890,
+    "vulture": 23,
+    "waffle iron": 891,
+    "walking stick, walkingstick, stick insect": 313,
+    "wall clock": 892,
+    "wallaby, brush kangaroo": 104,
+    "wallet, billfold, notecase, pocketbook": 893,
+    "wardrobe, closet, press": 894,
+    "warplane, military plane": 895,
+    "warthog": 343,
+    "washbasin, handbasin, washbowl, lavabo, wash-hand basin": 896,
+    "washer, automatic washer, washing machine": 897,
+    "water bottle": 898,
+    "water buffalo, water ox, Asiatic buffalo, Bubalus bubalis": 346,
+    "water jug": 899,
+    "water ouzel, dipper": 20,
+    "water snake": 58,
+    "water tower": 900,
+    "weasel": 356,
+    "web site, website, internet site, site": 916,
+    "weevil": 307,
+    "whippet": 172,
+    "whiptail, whiptail lizard": 41,
+    "whiskey jug": 901,
+    "whistle": 902,
+    "white stork, Ciconia ciconia": 127,
+    "white wolf, Arctic wolf, Canis lupus tundrarum": 270,
+    "wig": 903,
+    "wild boar, boar, Sus scrofa": 342,
+    "window screen": 904,
+    "window shade": 905,
+    "wine bottle": 907,
+    "wing": 908,
+    "wire-haired fox terrier": 188,
+    "wok": 909,
+    "wolf spider, hunting spider": 77,
+    "wombat": 106,
+    "wood rabbit, cottontail, cottontail rabbit": 330,
+    "wooden spoon": 910,
+    "wool, woolen, woollen": 911,
+    "worm fence, snake fence, snake-rail fence, Virginia fence": 912,
+    "wreck": 913,
+    "yawl": 914,
+    "yellow lady's slipper, yellow lady-slipper, Cypripedium calceolus, Cypripedium parviflorum": 986,
+    "yurt": 915,
+    "zebra": 340,
+    "zucchini, courgette": 939
+  },
+  "layer_norm_eps": 1e-12,
+  "model_type": "vit",
+  "num_attention_heads": 12,
+  "num_channels": 3,
+  "num_hidden_layers": 12,
+  "patch_size": 16,
+  "qkv_bias": true,
+  "torch_dtype": "float32",
+  "transformers_version": "4.44.2"
+}

chatbot/models/custom_model/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e8a70954c2250bc785d2259892c0ead906d25f5f16bbea62f55f668d286a6cfa
+size 343220892

chatbot/models/microsoft/Phi-3.5-mini-instruct/added_tokens.json

@@ -0,0 +1,13 @@
+{
+  "<|assistant|>": 32001,
+  "<|endoftext|>": 32000,
+  "<|end|>": 32007,
+  "<|placeholder1|>": 32002,
+  "<|placeholder2|>": 32003,
+  "<|placeholder3|>": 32004,
+  "<|placeholder4|>": 32005,
+  "<|placeholder5|>": 32008,
+  "<|placeholder6|>": 32009,
+  "<|system|>": 32006,
+  "<|user|>": 32010
+}

chatbot/models/microsoft/Phi-3.5-mini-instruct/config.json

@@ -0,0 +1,138 @@
+{
+  "_name_or_path": "microsoft/Phi-3.5-mini-instruct",
+  "architectures": [
+    "Phi3ForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "auto_map": {
+    "AutoConfig": "microsoft/Phi-3.5-mini-instruct--configuration_phi3.Phi3Config",
+    "AutoModelForCausalLM": "microsoft/Phi-3.5-mini-instruct--modeling_phi3.Phi3ForCausalLM"
+  },
+  "bos_token_id": 1,
+  "embd_pdrop": 0.0,
+  "eos_token_id": 32000,
+  "hidden_act": "silu",
+  "hidden_size": 3072,
+  "initializer_range": 0.02,
+  "intermediate_size": 8192,
+  "max_position_embeddings": 131072,
+  "model_type": "phi3",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 32,
+  "num_key_value_heads": 32,
+  "original_max_position_embeddings": 4096,
+  "pad_token_id": 32000,
+  "resid_pdrop": 0.0,
+  "rms_norm_eps": 1e-05,
+  "rope_scaling": {
+    "long_factor": [
+      1.0800000429153442,
+      1.1100000143051147,
+      1.1399999856948853,
+      1.340000033378601,
+      1.5899999141693115,
+      1.600000023841858,
+      1.6200000047683716,
+      2.620000123977661,
+      3.2300000190734863,
+      3.2300000190734863,
+      4.789999961853027,
+      7.400000095367432,
+      7.700000286102295,
+      9.09000015258789,
+      12.199999809265137,
+      17.670000076293945,
+      24.46000099182129,
+      28.57000160217285,
+      30.420001983642578,
+      30.840002059936523,
+      32.590003967285156,
+      32.93000411987305,
+      42.320003509521484,
+      44.96000289916992,
+      50.340003967285156,
+      50.45000457763672,
+      57.55000305175781,
+      57.93000411987305,
+      58.21000289916992,
+      60.1400032043457,
+      62.61000442504883,
+      62.62000274658203,
+      62.71000289916992,
+      63.1400032043457,
+      63.1400032043457,
+      63.77000427246094,
+      63.93000411987305,
+      63.96000289916992,
+      63.970001220703125,
+      64.02999877929688,
+      64.06999969482422,
+      64.08000183105469,
+      64.12000274658203,
+      64.41000366210938,
+      64.4800033569336,
+      64.51000213623047,
+      64.52999877929688,
+      64.83999633789062
+    ],
+    "short_factor": [
+      1.0,
+      1.0199999809265137,
+      1.0299999713897705,
+      1.0299999713897705,
+      1.0499999523162842,
+      1.0499999523162842,
+      1.0499999523162842,
+      1.0499999523162842,
+      1.0499999523162842,
+      1.0699999332427979,
+      1.0999999046325684,
+      1.1099998950958252,
+      1.1599998474121094,
+      1.1599998474121094,
+      1.1699998378753662,
+      1.2899998426437378,
+      1.339999794960022,
+      1.679999828338623,
+      1.7899998426437378,
+      1.8199998140335083,
+      1.8499997854232788,
+      1.8799997568130493,
+      1.9099997282028198,
+      1.9399996995925903,
+      1.9899996519088745,
+      2.0199997425079346,
+      2.0199997425079346,
+      2.0199997425079346,
+      2.0199997425079346,
+      2.0199997425079346,
+      2.0199997425079346,
+      2.0299997329711914,
+      2.0299997329711914,
+      2.0299997329711914,
+      2.0299997329711914,
+      2.0299997329711914,
+      2.0299997329711914,
+      2.0299997329711914,
+      2.0299997329711914,
+      2.0299997329711914,
+      2.0799996852874756,
+      2.0899996757507324,
+      2.189999580383301,
+      2.2199995517730713,
+      2.5899994373321533,
+      2.729999542236328,
+      2.749999523162842,
+      2.8399994373321533
+    ],
+    "type": "longrope"
+  },
+  "rope_theta": 10000.0,
+  "sliding_window": 262144,
+  "tie_word_embeddings": false,
+  "torch_dtype": "float32",
+  "transformers_version": "4.46.2",
+  "use_cache": true,
+  "vocab_size": 32064
+}

chatbot/models/microsoft/Phi-3.5-mini-instruct/generation_config.json

@@ -0,0 +1,11 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 1,
+  "eos_token_id": [
+    32007,
+    32001,
+    32000
+  ],
+  "pad_token_id": 32000,
+  "transformers_version": "4.46.2"
+}

chatbot/models/microsoft/Phi-3.5-mini-instruct/model-00001-of-00004.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:69f5298198adf824addbd073539922a669c5405154b9b8e5cf1b9f6f9829cee4
+size 4961852416

chatbot/models/microsoft/Phi-3.5-mini-instruct/model-00002-of-00004.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8f182e86b3535d1baf86e190963197c8860911c68b6d8613776aca890731f6c0
+size 4983111176

chatbot/models/microsoft/Phi-3.5-mini-instruct/model-00003-of-00004.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d672464098f4fa6927ff602d8de7c69e9c26dced17a5530a335e51ca95caaf43
+size 4945374704

chatbot/models/microsoft/Phi-3.5-mini-instruct/model-00004-of-00004.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2445155b9094892fed9c94bac15a5f750a041a814824a4770e4acf3d454380a3
+size 394002560

chatbot/models/microsoft/Phi-3.5-mini-instruct/model.safetensors.index.json

@@ -0,0 +1,202 @@
+{
+  "metadata": {
+    "total_size": 15284318208
+  },
+  "weight_map": {
+    "lm_head.weight": "model-00004-of-00004.safetensors",
+    "model.embed_tokens.weight": "model-00001-of-00004.safetensors",
+    "model.layers.0.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.0.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.0.mlp.gate_up_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.0.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.0.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.0.self_attn.qkv_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.1.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.1.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.1.mlp.gate_up_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.1.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.1.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.1.self_attn.qkv_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.10.input_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.10.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.10.mlp.gate_up_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.10.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.10.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.10.self_attn.qkv_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.11.input_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.11.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.11.mlp.gate_up_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.11.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.11.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.11.self_attn.qkv_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.12.input_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.12.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.12.mlp.gate_up_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.12.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.12.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.12.self_attn.qkv_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.13.input_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.13.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.13.mlp.gate_up_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.13.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.13.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.13.self_attn.qkv_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.14.input_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.14.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.14.mlp.gate_up_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.14.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.14.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.14.self_attn.qkv_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.15.input_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.15.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.15.mlp.gate_up_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.15.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.15.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.15.self_attn.qkv_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.16.input_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.16.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.16.mlp.gate_up_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.16.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.16.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.16.self_attn.qkv_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.17.input_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.17.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.17.mlp.gate_up_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.17.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.17.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.17.self_attn.qkv_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.18.input_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.18.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.18.mlp.gate_up_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.18.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.18.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.18.self_attn.qkv_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.19.input_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.19.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.19.mlp.gate_up_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.19.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.19.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.19.self_attn.qkv_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.2.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.2.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.2.mlp.gate_up_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.2.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.2.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.2.self_attn.qkv_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.20.input_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.20.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.20.mlp.gate_up_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.20.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.20.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.20.self_attn.qkv_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.21.input_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.21.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.21.mlp.gate_up_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.21.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.21.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.21.self_attn.qkv_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.22.input_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.22.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.22.mlp.gate_up_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.22.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.22.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.22.self_attn.qkv_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.23.input_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.23.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.23.mlp.gate_up_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.23.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.23.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.23.self_attn.qkv_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.24.input_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.24.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.24.mlp.gate_up_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.24.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.24.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.24.self_attn.qkv_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.25.input_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.25.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.25.mlp.gate_up_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.25.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.25.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.25.self_attn.qkv_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.26.input_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.26.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.26.mlp.gate_up_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.26.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.26.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.26.self_attn.qkv_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.27.input_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.27.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.27.mlp.gate_up_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.27.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.27.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.27.self_attn.qkv_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.28.input_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.28.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.28.mlp.gate_up_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.28.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.28.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.28.self_attn.qkv_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.29.input_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.29.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.29.mlp.gate_up_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.29.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.29.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.29.self_attn.qkv_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.3.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.3.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.3.mlp.gate_up_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.3.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.3.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.3.self_attn.qkv_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.30.input_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.30.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.30.mlp.gate_up_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.30.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.30.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.30.self_attn.qkv_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.31.input_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.31.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.31.mlp.gate_up_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.31.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.31.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.31.self_attn.qkv_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.4.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.4.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.4.mlp.gate_up_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.4.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.4.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.4.self_attn.qkv_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.5.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.5.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.5.mlp.gate_up_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.5.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.5.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.5.self_attn.qkv_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.6.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.6.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.6.mlp.gate_up_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.6.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.6.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.6.self_attn.qkv_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.7.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.7.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.7.mlp.gate_up_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.7.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.7.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.7.self_attn.qkv_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.8.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.8.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.8.mlp.gate_up_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.8.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.8.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.8.self_attn.qkv_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.9.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.9.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.9.mlp.gate_up_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.9.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.9.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.9.self_attn.qkv_proj.weight": "model-00001-of-00004.safetensors",
+    "model.norm.weight": "model-00003-of-00004.safetensors"
+  }
+}