{ "cells": [ { "cell_type": "markdown", "source": [ "# Fine-Tuning GPT-2 on Drugs.com Reviews for Depression\n", "\n", "**Author**: Zakia Salod\n", "\n", "**Affiliation**: University of KwaZulu-Natal (UKZN), Durban, South Africa\n", "\n", "**Contact**: zakia.salod@gmail.com\n", "\n", "**Machine Used**: Google Colab T4 GPU\n", "\n", "**Last Updated**: 9 December 2023\n", "\n", "**Description**:\n", "This notebook demonstrates the process of fine-tuning the GPT-2 model using the Drugs.com reviews dataset, focusing specifically on reviews related to depression. The notebook covers the steps of loading and preprocessing the dataset, initializing and preparing the GPT-2 model for fine-tuning, and the fine-tuning process itself. Post-training, the notebook illustrates how to save and push the fine-tuned model to the Hugging Face Hub. This exercise aims to enhance GPT-2's ability to generate more relevant and insightful text related to depression drug reviews.\n", "\n", "\n", "**License**:\n", "This work is licensed under the [Apache 2.0 License](https://www.apache.org/licenses/LICENSE-2.0). Free for educational and research use.\n", "\n" ], "metadata": { "id": "SlML_A8WXRKX" } }, { "cell_type": "markdown", "metadata": { "id": "kJK4WrJB4JR1" }, "source": [ "## STEP 1: SETTING UP THE ENVIRONMENT" ] }, { "cell_type": "markdown", "metadata": { "id": "zzrRM4P14M7-" }, "source": [ "### Load Necessary Libraries" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "zbzxLSs_tKQ4" }, "outputs": [], "source": [ "# Enable automatic module reloading to reflect changes in external .py files\n", "%load_ext autoreload\n", "# Reload all modules before executing code, keeping modules up-to-date\n", "%autoreload 2" ] }, { "cell_type": "markdown", "source": [ "### Install Required Packages" ], "metadata": { "id": "fq1xfyKlf_Ea" } }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "Bv85wGPxto3O", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "dab3659e-59ab-41a4-9a70-838773c85c5a" }, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "Collecting git+https://github.com/huggingface/transformers\n", " Cloning https://github.com/huggingface/transformers to /tmp/pip-req-build-44ijr861\n", " Running command git clone --filter=blob:none --quiet https://github.com/huggingface/transformers /tmp/pip-req-build-44ijr861\n", " Resolved https://github.com/huggingface/transformers to commit df5c5c62ae253055336f5bb0828ca8e3e15ab6bd\n", " Installing build dependencies ... \u001b[?25l\u001b[?25hdone\n", " Getting requirements to build wheel ... \u001b[?25l\u001b[?25hdone\n", " Preparing metadata (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n", "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from transformers==4.36.0.dev0) (3.13.1)\n", "Requirement already satisfied: huggingface-hub<1.0,>=0.19.3 in /usr/local/lib/python3.10/dist-packages (from transformers==4.36.0.dev0) (0.19.4)\n", "Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.10/dist-packages (from transformers==4.36.0.dev0) (1.23.5)\n", "Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.10/dist-packages (from transformers==4.36.0.dev0) (23.2)\n", "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from transformers==4.36.0.dev0) (6.0.1)\n", "Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.10/dist-packages (from transformers==4.36.0.dev0) (2023.6.3)\n", "Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from transformers==4.36.0.dev0) (2.31.0)\n", "Requirement already satisfied: tokenizers<0.19,>=0.14 in /usr/local/lib/python3.10/dist-packages (from transformers==4.36.0.dev0) (0.15.0)\n", "Requirement already satisfied: safetensors>=0.3.1 in /usr/local/lib/python3.10/dist-packages (from transformers==4.36.0.dev0) (0.4.1)\n", "Requirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.10/dist-packages (from transformers==4.36.0.dev0) (4.66.1)\n", "Requirement already satisfied: fsspec>=2023.5.0 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub<1.0,>=0.19.3->transformers==4.36.0.dev0) (2023.6.0)\n", "Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub<1.0,>=0.19.3->transformers==4.36.0.dev0) (4.5.0)\n", "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->transformers==4.36.0.dev0) (3.3.2)\n", "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->transformers==4.36.0.dev0) (3.6)\n", "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->transformers==4.36.0.dev0) (2.0.7)\n", "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->transformers==4.36.0.dev0) (2023.11.17)\n", "Building wheels for collected packages: transformers\n", " Building wheel for transformers (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n", " Created wheel for transformers: filename=transformers-4.36.0.dev0-py3-none-any.whl size=8195433 sha256=79b5b0d64790d529619e06ffc24ad2f2a2692b0a7d84d20a1e0c4dae56550d23\n", " Stored in directory: /tmp/pip-ephem-wheel-cache-ogqdggii/wheels/c0/14/d6/6c9a5582d2ac191ec0a483be151a4495fe1eb2a6706ca49f1b\n", "Successfully built transformers\n", "Installing collected packages: transformers\n", " Attempting uninstall: transformers\n", " Found existing installation: transformers 4.35.2\n", " Uninstalling transformers-4.35.2:\n", " Successfully uninstalled transformers-4.35.2\n", "Successfully installed transformers-4.36.0.dev0\n" ] } ], "source": [ "!pip install git+https://github.com/huggingface/transformers" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "ZTZrScZY6NB1", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "5342e849-426d-437f-a9c2-b75adfbcb4bf" }, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "Collecting datasets\n", " Downloading datasets-2.15.0-py3-none-any.whl (521 kB)\n", "\u001b[?25l \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m0.0/521.2 kB\u001b[0m \u001b[31m?\u001b[0m eta \u001b[36m-:--:--\u001b[0m\r\u001b[2K \u001b[91m━━━━━━━━━━━\u001b[0m\u001b[90m╺\u001b[0m\u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m143.4/521.2 kB\u001b[0m \u001b[31m4.6 MB/s\u001b[0m eta \u001b[36m0:00:01\u001b[0m\r\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m521.2/521.2 kB\u001b[0m \u001b[31m8.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hCollecting evaluate\n", " Downloading evaluate-0.4.1-py3-none-any.whl (84 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m84.1/84.1 kB\u001b[0m \u001b[31m11.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hRequirement already satisfied: torch in /usr/local/lib/python3.10/dist-packages (2.1.0+cu118)\n", "Collecting wandb\n", " Downloading wandb-0.16.1-py3-none-any.whl (2.1 MB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.1/2.1 MB\u001b[0m \u001b[31m31.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hRequirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.10/dist-packages (from datasets) (1.23.5)\n", "Requirement already satisfied: pyarrow>=8.0.0 in /usr/local/lib/python3.10/dist-packages (from datasets) (9.0.0)\n", "Collecting pyarrow-hotfix (from datasets)\n", " Downloading pyarrow_hotfix-0.6-py3-none-any.whl (7.9 kB)\n", "Collecting dill<0.3.8,>=0.3.0 (from datasets)\n", " Downloading dill-0.3.7-py3-none-any.whl (115 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m115.3/115.3 kB\u001b[0m \u001b[31m15.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hRequirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from datasets) (1.5.3)\n", "Requirement already satisfied: requests>=2.19.0 in /usr/local/lib/python3.10/dist-packages (from datasets) (2.31.0)\n", "Requirement already satisfied: tqdm>=4.62.1 in /usr/local/lib/python3.10/dist-packages (from datasets) (4.66.1)\n", "Requirement already satisfied: xxhash in /usr/local/lib/python3.10/dist-packages (from datasets) (3.4.1)\n", "Collecting multiprocess (from datasets)\n", " Downloading multiprocess-0.70.15-py310-none-any.whl (134 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m134.8/134.8 kB\u001b[0m \u001b[31m18.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hRequirement already satisfied: fsspec[http]<=2023.10.0,>=2023.1.0 in /usr/local/lib/python3.10/dist-packages (from datasets) (2023.6.0)\n", "Requirement already satisfied: aiohttp in /usr/local/lib/python3.10/dist-packages (from datasets) (3.9.1)\n", "Requirement already satisfied: huggingface-hub>=0.18.0 in /usr/local/lib/python3.10/dist-packages (from datasets) (0.19.4)\n", "Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from datasets) (23.2)\n", "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from datasets) (6.0.1)\n", "Collecting responses<0.19 (from evaluate)\n", " Downloading responses-0.18.0-py3-none-any.whl (38 kB)\n", "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from torch) (3.13.1)\n", "Requirement already satisfied: typing-extensions in /usr/local/lib/python3.10/dist-packages (from torch) (4.5.0)\n", "Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch) (1.12)\n", "Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch) (3.2.1)\n", "Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from torch) (3.1.2)\n", "Requirement already satisfied: triton==2.1.0 in /usr/local/lib/python3.10/dist-packages (from torch) (2.1.0)\n", "Requirement already satisfied: Click!=8.0.0,>=7.1 in /usr/local/lib/python3.10/dist-packages (from wandb) (8.1.7)\n", "Collecting GitPython!=3.1.29,>=1.0.0 (from wandb)\n", " Downloading GitPython-3.1.40-py3-none-any.whl (190 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m190.6/190.6 kB\u001b[0m \u001b[31m23.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hRequirement already satisfied: psutil>=5.0.0 in /usr/local/lib/python3.10/dist-packages (from wandb) (5.9.5)\n", "Collecting sentry-sdk>=1.0.0 (from wandb)\n", " Downloading sentry_sdk-1.38.0-py2.py3-none-any.whl (252 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m252.8/252.8 kB\u001b[0m \u001b[31m28.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hCollecting docker-pycreds>=0.4.0 (from wandb)\n", " Downloading docker_pycreds-0.4.0-py2.py3-none-any.whl (9.0 kB)\n", "Collecting setproctitle (from wandb)\n", " Downloading setproctitle-1.3.3-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (30 kB)\n", "Requirement already satisfied: setuptools in /usr/local/lib/python3.10/dist-packages (from wandb) (67.7.2)\n", "Requirement already satisfied: appdirs>=1.4.3 in /usr/local/lib/python3.10/dist-packages (from wandb) (1.4.4)\n", "Requirement already satisfied: protobuf!=4.21.0,<5,>=3.19.0 in /usr/local/lib/python3.10/dist-packages (from wandb) (3.20.3)\n", "Requirement already satisfied: six>=1.4.0 in /usr/local/lib/python3.10/dist-packages (from docker-pycreds>=0.4.0->wandb) (1.16.0)\n", "Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (23.1.0)\n", "Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (6.0.4)\n", "Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (1.9.3)\n", "Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (1.4.0)\n", "Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (1.3.1)\n", "Requirement already satisfied: async-timeout<5.0,>=4.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp->datasets) (4.0.3)\n", "Collecting gitdb<5,>=4.0.1 (from GitPython!=3.1.29,>=1.0.0->wandb)\n", " Downloading gitdb-4.0.11-py3-none-any.whl (62 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m62.7/62.7 kB\u001b[0m \u001b[31m9.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hRequirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests>=2.19.0->datasets) (3.3.2)\n", "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests>=2.19.0->datasets) (3.6)\n", "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests>=2.19.0->datasets) (2.0.7)\n", "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests>=2.19.0->datasets) (2023.11.17)\n", "Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->torch) (2.1.3)\n", "Requirement already satisfied: python-dateutil>=2.8.1 in /usr/local/lib/python3.10/dist-packages (from pandas->datasets) (2.8.2)\n", "Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas->datasets) (2023.3.post1)\n", "Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->torch) (1.3.0)\n", "Collecting smmap<6,>=3.0.1 (from gitdb<5,>=4.0.1->GitPython!=3.1.29,>=1.0.0->wandb)\n", " Downloading smmap-5.0.1-py3-none-any.whl (24 kB)\n", "Installing collected packages: smmap, setproctitle, sentry-sdk, pyarrow-hotfix, docker-pycreds, dill, responses, multiprocess, gitdb, GitPython, wandb, datasets, evaluate\n", "Successfully installed GitPython-3.1.40 datasets-2.15.0 dill-0.3.7 docker-pycreds-0.4.0 evaluate-0.4.1 gitdb-4.0.11 multiprocess-0.70.15 pyarrow-hotfix-0.6 responses-0.18.0 sentry-sdk-1.38.0 setproctitle-1.3.3 smmap-5.0.1 wandb-0.16.1\n" ] } ], "source": [ "!pip install datasets evaluate torch wandb" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "CHbwWz_gH1qc", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "18979ebb-9fce-4d87-dccf-ddae7d9df25c" }, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "Collecting accelerate\n", " Downloading accelerate-0.25.0-py3-none-any.whl (265 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m265.7/265.7 kB\u001b[0m \u001b[31m5.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hRequirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.10/dist-packages (from accelerate) (1.23.5)\n", "Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.10/dist-packages (from accelerate) (23.2)\n", "Requirement already satisfied: psutil in /usr/local/lib/python3.10/dist-packages (from accelerate) (5.9.5)\n", "Requirement already satisfied: pyyaml in /usr/local/lib/python3.10/dist-packages (from accelerate) (6.0.1)\n", "Requirement already satisfied: torch>=1.10.0 in /usr/local/lib/python3.10/dist-packages (from accelerate) (2.1.0+cu118)\n", "Requirement already satisfied: huggingface-hub in /usr/local/lib/python3.10/dist-packages (from accelerate) (0.19.4)\n", "Requirement already satisfied: safetensors>=0.3.1 in /usr/local/lib/python3.10/dist-packages (from accelerate) (0.4.1)\n", "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate) (3.13.1)\n", "Requirement already satisfied: typing-extensions in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate) (4.5.0)\n", "Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate) (1.12)\n", "Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate) (3.2.1)\n", "Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate) (3.1.2)\n", "Requirement already satisfied: fsspec in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate) (2023.6.0)\n", "Requirement already satisfied: triton==2.1.0 in /usr/local/lib/python3.10/dist-packages (from torch>=1.10.0->accelerate) (2.1.0)\n", "Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from huggingface-hub->accelerate) (2.31.0)\n", "Requirement already satisfied: tqdm>=4.42.1 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub->accelerate) (4.66.1)\n", "Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->torch>=1.10.0->accelerate) (2.1.3)\n", "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface-hub->accelerate) (3.3.2)\n", "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface-hub->accelerate) (3.6)\n", "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface-hub->accelerate) (2.0.7)\n", "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface-hub->accelerate) (2023.11.17)\n", "Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->torch>=1.10.0->accelerate) (1.3.0)\n", "Installing collected packages: accelerate\n", "Successfully installed accelerate-0.25.0\n" ] } ], "source": [ "!pip install accelerate -U" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "8ClR-_ktH6-3", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "1bc58861-c84b-4b0d-bd02-87f8fb39fbf0" }, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "Requirement already satisfied: transformers[torch] in /usr/local/lib/python3.10/dist-packages (4.36.0.dev0)\n", "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from transformers[torch]) (3.13.1)\n", "Requirement already satisfied: huggingface-hub<1.0,>=0.19.3 in /usr/local/lib/python3.10/dist-packages (from transformers[torch]) (0.19.4)\n", "Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.10/dist-packages (from transformers[torch]) (1.23.5)\n", "Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.10/dist-packages (from transformers[torch]) (23.2)\n", "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from transformers[torch]) (6.0.1)\n", "Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.10/dist-packages (from transformers[torch]) (2023.6.3)\n", "Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from transformers[torch]) (2.31.0)\n", "Requirement already satisfied: tokenizers<0.19,>=0.14 in /usr/local/lib/python3.10/dist-packages (from transformers[torch]) (0.15.0)\n", "Requirement already satisfied: safetensors>=0.3.1 in /usr/local/lib/python3.10/dist-packages (from transformers[torch]) (0.4.1)\n", "Requirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.10/dist-packages (from transformers[torch]) (4.66.1)\n", "Requirement already satisfied: torch!=1.12.0,>=1.10 in /usr/local/lib/python3.10/dist-packages (from transformers[torch]) (2.1.0+cu118)\n", "Requirement already satisfied: accelerate>=0.21.0 in /usr/local/lib/python3.10/dist-packages (from transformers[torch]) (0.25.0)\n", "Requirement already satisfied: psutil in /usr/local/lib/python3.10/dist-packages (from accelerate>=0.21.0->transformers[torch]) (5.9.5)\n", "Requirement already satisfied: fsspec>=2023.5.0 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub<1.0,>=0.19.3->transformers[torch]) (2023.6.0)\n", "Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub<1.0,>=0.19.3->transformers[torch]) (4.5.0)\n", "Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch!=1.12.0,>=1.10->transformers[torch]) (1.12)\n", "Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch!=1.12.0,>=1.10->transformers[torch]) (3.2.1)\n", "Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from torch!=1.12.0,>=1.10->transformers[torch]) (3.1.2)\n", "Requirement already satisfied: triton==2.1.0 in /usr/local/lib/python3.10/dist-packages (from torch!=1.12.0,>=1.10->transformers[torch]) (2.1.0)\n", "Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->transformers[torch]) (3.3.2)\n", "Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/dist-packages (from requests->transformers[torch]) (3.6)\n", "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->transformers[torch]) (2.0.7)\n", "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/dist-packages (from requests->transformers[torch]) (2023.11.17)\n", "Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->torch!=1.12.0,>=1.10->transformers[torch]) (2.1.3)\n", "Requirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->torch!=1.12.0,>=1.10->transformers[torch]) (1.3.0)\n" ] } ], "source": [ "!pip install transformers[torch]" ] }, { "cell_type": "markdown", "source": [ "### Import Necessary Libraries" ], "metadata": { "id": "GXDAGYt9gFFM" } }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "ZPQo3nOFtpX-" }, "outputs": [], "source": [ "import numpy as np\n", "from datasets import load_dataset\n", "from datasets import concatenate_datasets\n", "from transformers import GPT2Tokenizer\n", "from transformers import GPT2LMHeadModel\n", "import html\n", "import re\n", "from collections import Counter\n", "import pandas as pd" ] }, { "cell_type": "markdown", "metadata": { "id": "CAc15TpfT0m6" }, "source": [ "### Initialize Weights & Biases for Tracking" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "gu1-uIyZT179" }, "outputs": [], "source": [ "# Import Weights & Biases for experiment tracking\n", "import wandb" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "FAcoX4eniLWc", "colab": { "base_uri": "https://localhost:8080/", "height": 211 }, "outputId": "5726f50d-2232-44bc-c4fb-a8cd9c8fe69f" }, "outputs": [ { "output_type": "display_data", "data": { "text/plain": [ "" ], "application/javascript": [ "\n", " window._wandbApiKey = new Promise((resolve, reject) => {\n", " function loadScript(url) {\n", " return new Promise(function(resolve, reject) {\n", " let newScript = document.createElement(\"script\");\n", " newScript.onerror = reject;\n", " newScript.onload = resolve;\n", " document.body.appendChild(newScript);\n", " newScript.src = url;\n", " });\n", " }\n", " loadScript(\"https://cdn.jsdelivr.net/npm/postmate/build/postmate.min.js\").then(() => {\n", " const iframe = document.createElement('iframe')\n", " iframe.style.cssText = \"width:0;height:0;border:none\"\n", " document.body.appendChild(iframe)\n", " const handshake = new Postmate({\n", " container: iframe,\n", " url: 'https://wandb.ai/authorize'\n", " });\n", " const timeout = setTimeout(() => reject(\"Couldn't auto authenticate\"), 5000)\n", " handshake.then(function(child) {\n", " child.on('authorize', data => {\n", " clearTimeout(timeout)\n", " resolve(data)\n", " });\n", " });\n", " })\n", " });\n", " " ] }, "metadata": {} }, { "output_type": "stream", "name": "stderr", "text": [ "\u001b[34m\u001b[1mwandb\u001b[0m: Logging into wandb.ai. (Learn how to deploy a W&B server locally: https://wandb.me/wandb-server)\n", "\u001b[34m\u001b[1mwandb\u001b[0m: You can find your API key in your browser here: https://wandb.ai/authorize\n", "wandb: Paste an API key from your profile and hit enter, or press ctrl+c to quit:" ] }, { "name": "stdout", "output_type": "stream", "text": [ " ··········\n" ] }, { "output_type": "stream", "name": "stderr", "text": [ "\u001b[34m\u001b[1mwandb\u001b[0m: Appending key for api.wandb.ai to your netrc file: /root/.netrc\n" ] }, { "output_type": "display_data", "data": { "text/plain": [ "" ], "text/html": [ "Tracking run with wandb version 0.16.1" ] }, "metadata": {} }, { "output_type": "display_data", "data": { "text/plain": [ "" ], "text/html": [ "Run data is saved locally in /content/wandb/run-20231209_123209-kfb5cx1j" ] }, "metadata": {} }, { "output_type": "display_data", "data": { "text/plain": [ "" ], "text/html": [ "Syncing run jumping-energy-8 to Weights & Biases (docs)
" ] }, "metadata": {} }, { "output_type": "display_data", "data": { "text/plain": [ "" ], "text/html": [ " View project at https://wandb.ai/team-zakia/gpt2-drugscom_depression_reviews" ] }, "metadata": {} }, { "output_type": "display_data", "data": { "text/plain": [ "" ], "text/html": [ " View run at https://wandb.ai/team-zakia/gpt2-drugscom_depression_reviews/runs/kfb5cx1j" ] }, "metadata": {} }, { "output_type": "execute_result", "data": { "text/html": [ "" ], "text/plain": [ "" ] }, "metadata": {}, "execution_count": 8 } ], "source": [ "# Initialize wandb for tracking and visualizing the training\n", "wandb.init(project=\"gpt2-drugscom_depression_reviews\")" ] }, { "cell_type": "markdown", "metadata": { "id": "LGYKu-eF4ZvJ" }, "source": [ "## STEP 2: LOAD AND PRE-PROCESS DRUG REVIEWS DATASET" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "v_FNuZmvtsY1" }, "outputs": [], "source": [ "dataset_name = \"Zakia/drugscom_reviews\"" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "DyK71VIut7kf", "colab": { "base_uri": "https://localhost:8080/", "height": 241, "referenced_widgets": [ "b45c8c73698048aeaee4d34b28d6a27a", "90e79e1e73ae489199ff3811dca9adae", "e59b83647a7f483da05b37429322c469", "66ec2664647846d8b4da85fee9e32df1", "6b6eec8ea25445a6bc24b88778e1edbf", "f2722d9ef4484a2a90ca9d6319d04345", "9cc770f5d35e421cb07b44c9f7e2a730", "2c52eb43c73e4c7f8831bd6f71809570", "02587cf05a5547c4ba874c34bf4cec4b", "0707b92bf1a84a9384beba55789ce6ba", "57e880d55b0548a0b3a0f61ccdcc930f", "10311a17cdf645f69a3edd08ca2cad62", "a6000f68e93d4bf1a343629032dcef04", "ac869d78cfe146879a2494579c77277d", "2e35f836eaed419a8431e0c68e8d2cc1", "a3d694bb06194b138a7abdbf199a1d81", "5bdaf9e1d6da4b808b2a704b3c8d973b", "2e5a51a0fb4c4249b55d8af4b0506eca", "b4fcbf969be2427b92046a1f528db2a6", "43c31088bf444fd39578f2fe630dc46e", "3f7cb58407064ebaa1ec3943309c95c3", "ce17ffc4033b4866b48d5e61467db6c3", "0ffc9ba03f924bc0ad48fa82c2e711e7", "0abb262f82df4d728a8f05bf992a5ec3", "09e3afd0ba9e4b5298d741ce825fdbfb", "f069002ce14648c3b0269ae3e377879c", "1b6b3d597a024cfb9a15573cea12891e", "e48c988e59b8462babda6b5173a337a6", "43f34a6373a7449a83d86705351d4957", "3794962a26714723a2b7d345336db7ec", "19ee2670c44b46e2906b21a73b27ba88", "16a81359fc4a40dbb83d185c1db81ebf", "8970468c315b413abb6541205fc3174d", "84ba5a9fb669445d88a8f18c86986a65", "74e6479c15a843c196b26eae181f3b50", "24f3bb13e552446ca3dbf1d5acdd306e", "adf7780fbbed4baf90f08b113553e0a3", "59b48ccd44734f2fb869291b1c52a56b", "bff2b32d8b624af98814c950bf2934d9", "8b5cf7a76357475d8156886940ad2790", "a8a19e8b09a84ea9a29a6dd8312d2110", "654b39a5572f42bfa93ba73c4b251fbd", "76464bf094c74cd68f3fb5e9975ff56e", "1a3a890317814810834c117ffe62f390", "a6687ff1f33a42988536dc04ccf824ad", "6e79a05a0f6f4e4c8b0ce54d47180df1", "c7f71309b9a14d9b85a9953f1c5ef919", "ad491e5e0a8d4106b6942f20206081f4", "a08bd346d672441994b14340f57fa298", "d9166639b16c423fa902ef9a23a6582f", "0e564d1c6ae8447fb431032ee5d64199", "c22ad507940c46d0bf34dc5efe8ffee3", "38331b41172b47fa96b2b7950dd4f94b", "15038217fa3c4c43990aeb222792feed", "8bad5a126935481497fdb16ad2dacbb5", "28b5d0115eb44b9483c9f7d345d644e1", "5aa2e3b0f8024725a260898904739acb", "bdb27820a5204529987b9e2df414df78", "f4f30d7e47864e4c9515ef6195661782", "c4230c8b74d540afa5d2e885310dbf6b", "bf9000b17de04dd1bd522f94a3eb1ed9", "97411c1fec18481cbd5bcab1e794cc7f", "875e1cbc8e634cbaa00a22ff3c9d62f8", "52d6693766714e91b126d9180dd03346", "f247e069bb4147e0b6019d66d851ede9", "6a509c81bfff4f30bc7d2de029bdc93c", "36a765c57fc349eab616355610980324", "210adeaaf64e40db8323e085794e7d7a", "b8b1a3dfbb984ba08b1c862b26af8dbe", "304df8debe3f4662a4171221d669798e", "266e9676bce140b69ad73456851ff499", "9b0b095167de49f4bc5efb5502b3d41c", "af16b06528824cc9b0ce63fc4b4738ba", "daa61ca2c88c49be9a115fb81f01caac", "5edea6130ae64675a5aecbbaffcc5821", "1b55987264384ee3bd90ba9e97842b61", "2a4ea7765a984268a58766f089744cdb" ] }, "outputId": "bd5d0427-c53e-4168-efba-44f64eadee42" }, "outputs": [ { "output_type": "display_data", "data": { "text/plain": [ "Downloading readme: 0%| | 0.00/6.72k [00:00', '', text) # Remove HTML tags\n", " return text" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "jSEt0xnfuSQu" }, "outputs": [], "source": [ "# Clean the reviews of the train set\n", "# Apply the clean_review function in a batched manner\n", "def clean_reviews(batch):\n", " # Apply clean_review to each review in the batch and return the modified batch\n", " return {\"review\": [clean_review(review) for review in batch[\"review\"]]}" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "llYAOpmpuTad", "colab": { "base_uri": "https://localhost:8080/", "height": 49, "referenced_widgets": [ "48f3b3e586f74174a0232079ec543e2c", "57403d31cdb64f11add87566d4f5a325", "f9e6f05aeb904395a5d5a8a3ec3b9756", "5b787cdd23f44cb8805e055ccffdfe40", "e00ea9a3b8cc433a8dffc1cf9d99f609", "c6cae5d28bd9480eb1eb5642132d4dd5", "90e73a35476f4e74aba7c02f329a3536", "dea3da72e6d946029a3760a594356701", "5c7f4c3d4ebc47a6ba27aeed52a89add", "23396629e70847a28c247157d27587e7", "2262d8d8318e453a92e4ae9b37fa484c" ] }, "outputId": "3815254b-c672-41ad-8227-5a925d00d4d7" }, "outputs": [ { "output_type": "display_data", "data": { "text/plain": [ "Map: 0%| | 0/9069 [00:00,\n", "ignore_data_skip=False,\n", "include_inputs_for_metrics=False,\n", "include_num_input_tokens_seen=False,\n", "include_tokens_per_second=False,\n", "jit_mode_eval=False,\n", "label_names=None,\n", "label_smoothing_factor=0.0,\n", "learning_rate=5e-05,\n", "length_column_name=length,\n", "load_best_model_at_end=False,\n", "local_rank=0,\n", "log_level=passive,\n", "log_level_replica=warning,\n", "log_on_each_node=True,\n", "logging_dir=./gpt2-drugscom_depression_reviews/runs/Dec09_12-33-34_8acb048be5fa,\n", "logging_first_step=False,\n", "logging_nan_inf_filter=True,\n", "logging_steps=500,\n", "logging_strategy=steps,\n", "lr_scheduler_kwargs={},\n", "lr_scheduler_type=linear,\n", "max_grad_norm=1.0,\n", "max_steps=-1,\n", "metric_for_best_model=None,\n", "mp_parameters=,\n", "neftune_noise_alpha=None,\n", "no_cuda=False,\n", "num_train_epochs=5.0,\n", "optim=adamw_torch,\n", "optim_args=None,\n", "output_dir=./gpt2-drugscom_depression_reviews,\n", "overwrite_output_dir=True,\n", "past_index=-1,\n", "per_device_eval_batch_size=8,\n", "per_device_train_batch_size=2,\n", "prediction_loss_only=False,\n", "push_to_hub=False,\n", "push_to_hub_model_id=None,\n", "push_to_hub_organization=None,\n", "push_to_hub_token=,\n", "ray_scope=last,\n", "remove_unused_columns=True,\n", "report_to=['wandb'],\n", "resume_from_checkpoint=None,\n", "run_name=./gpt2-drugscom_depression_reviews,\n", "save_on_each_node=False,\n", "save_only_model=False,\n", "save_safetensors=True,\n", "save_steps=500,\n", "save_strategy=steps,\n", "save_total_limit=None,\n", "seed=42,\n", "skip_memory_metrics=True,\n", "split_batches=False,\n", "tf32=None,\n", "torch_compile=False,\n", "torch_compile_backend=None,\n", "torch_compile_mode=None,\n", "torchdynamo=None,\n", "tpu_metrics_debug=False,\n", "tpu_num_cores=None,\n", "use_cpu=False,\n", "use_ipex=False,\n", "use_legacy_prediction_loop=False,\n", "use_mps_device=False,\n", "warmup_ratio=0.0,\n", "warmup_steps=0,\n", "weight_decay=0.0,\n", ")\n", "Using custom data configuration default-99d8365ea3008c01\n", "12/09/2023 12:33:40 - INFO - datasets.builder - Using custom data configuration default-99d8365ea3008c01\n", "Loading Dataset Infos from /usr/local/lib/python3.10/dist-packages/datasets/packaged_modules/text\n", "12/09/2023 12:33:40 - INFO - datasets.info - Loading Dataset Infos from /usr/local/lib/python3.10/dist-packages/datasets/packaged_modules/text\n", "Generating dataset text (/root/.cache/huggingface/datasets/text/default-99d8365ea3008c01/0.0.0/c4a140d10f020282918b5dd1b8a49f0104729c6177f60a6b49ec2a365ec69f34)\n", "12/09/2023 12:33:40 - INFO - datasets.builder - Generating dataset text (/root/.cache/huggingface/datasets/text/default-99d8365ea3008c01/0.0.0/c4a140d10f020282918b5dd1b8a49f0104729c6177f60a6b49ec2a365ec69f34)\n", "Downloading and preparing dataset text/default to /root/.cache/huggingface/datasets/text/default-99d8365ea3008c01/0.0.0/c4a140d10f020282918b5dd1b8a49f0104729c6177f60a6b49ec2a365ec69f34...\n", "12/09/2023 12:33:40 - INFO - datasets.builder - Downloading and preparing dataset text/default to /root/.cache/huggingface/datasets/text/default-99d8365ea3008c01/0.0.0/c4a140d10f020282918b5dd1b8a49f0104729c6177f60a6b49ec2a365ec69f34...\n", "Downloading data files: 100% 1/1 [00:00<00:00, 7256.58it/s]\n", "Downloading took 0.0 min\n", "12/09/2023 12:33:40 - INFO - datasets.download.download_manager - Downloading took 0.0 min\n", "Checksum Computation took 0.0 min\n", "12/09/2023 12:33:40 - INFO - datasets.download.download_manager - Checksum Computation took 0.0 min\n", "Extracting data files: 100% 1/1 [00:00<00:00, 1198.72it/s]\n", "Generating train split\n", "12/09/2023 12:33:40 - INFO - datasets.builder - Generating train split\n", "Generating train split: 12943 examples [00:00, 19902.65 examples/s]\n", "Unable to verify splits sizes.\n", "12/09/2023 12:33:41 - INFO - datasets.utils.info_utils - Unable to verify splits sizes.\n", "Dataset text downloaded and prepared to /root/.cache/huggingface/datasets/text/default-99d8365ea3008c01/0.0.0/c4a140d10f020282918b5dd1b8a49f0104729c6177f60a6b49ec2a365ec69f34. Subsequent calls will reuse this data.\n", "12/09/2023 12:33:41 - INFO - datasets.builder - Dataset text downloaded and prepared to /root/.cache/huggingface/datasets/text/default-99d8365ea3008c01/0.0.0/c4a140d10f020282918b5dd1b8a49f0104729c6177f60a6b49ec2a365ec69f34. Subsequent calls will reuse this data.\n", "Using custom data configuration default-99d8365ea3008c01\n", "12/09/2023 12:33:41 - INFO - datasets.builder - Using custom data configuration default-99d8365ea3008c01\n", "Loading Dataset Infos from /usr/local/lib/python3.10/dist-packages/datasets/packaged_modules/text\n", "12/09/2023 12:33:41 - INFO - datasets.info - Loading Dataset Infos from /usr/local/lib/python3.10/dist-packages/datasets/packaged_modules/text\n", "Overwrite dataset info from restored data version if exists.\n", "12/09/2023 12:33:41 - INFO - datasets.builder - Overwrite dataset info from restored data version if exists.\n", "Loading Dataset info from /root/.cache/huggingface/datasets/text/default-99d8365ea3008c01/0.0.0/c4a140d10f020282918b5dd1b8a49f0104729c6177f60a6b49ec2a365ec69f34\n", "12/09/2023 12:33:41 - INFO - datasets.info - Loading Dataset info from /root/.cache/huggingface/datasets/text/default-99d8365ea3008c01/0.0.0/c4a140d10f020282918b5dd1b8a49f0104729c6177f60a6b49ec2a365ec69f34\n", "Found cached dataset text (/root/.cache/huggingface/datasets/text/default-99d8365ea3008c01/0.0.0/c4a140d10f020282918b5dd1b8a49f0104729c6177f60a6b49ec2a365ec69f34)\n", "12/09/2023 12:33:41 - INFO - datasets.builder - Found cached dataset text (/root/.cache/huggingface/datasets/text/default-99d8365ea3008c01/0.0.0/c4a140d10f020282918b5dd1b8a49f0104729c6177f60a6b49ec2a365ec69f34)\n", "Loading Dataset info from /root/.cache/huggingface/datasets/text/default-99d8365ea3008c01/0.0.0/c4a140d10f020282918b5dd1b8a49f0104729c6177f60a6b49ec2a365ec69f34\n", "12/09/2023 12:33:41 - INFO - datasets.info - Loading Dataset info from /root/.cache/huggingface/datasets/text/default-99d8365ea3008c01/0.0.0/c4a140d10f020282918b5dd1b8a49f0104729c6177f60a6b49ec2a365ec69f34\n", "Using custom data configuration default-99d8365ea3008c01\n", "12/09/2023 12:33:41 - INFO - datasets.builder - Using custom data configuration default-99d8365ea3008c01\n", "Loading Dataset Infos from /usr/local/lib/python3.10/dist-packages/datasets/packaged_modules/text\n", "12/09/2023 12:33:41 - INFO - datasets.info - Loading Dataset Infos from /usr/local/lib/python3.10/dist-packages/datasets/packaged_modules/text\n", "Overwrite dataset info from restored data version if exists.\n", "12/09/2023 12:33:41 - INFO - datasets.builder - Overwrite dataset info from restored data version if exists.\n", "Loading Dataset info from /root/.cache/huggingface/datasets/text/default-99d8365ea3008c01/0.0.0/c4a140d10f020282918b5dd1b8a49f0104729c6177f60a6b49ec2a365ec69f34\n", "12/09/2023 12:33:41 - INFO - datasets.info - Loading Dataset info from /root/.cache/huggingface/datasets/text/default-99d8365ea3008c01/0.0.0/c4a140d10f020282918b5dd1b8a49f0104729c6177f60a6b49ec2a365ec69f34\n", "Found cached dataset text (/root/.cache/huggingface/datasets/text/default-99d8365ea3008c01/0.0.0/c4a140d10f020282918b5dd1b8a49f0104729c6177f60a6b49ec2a365ec69f34)\n", "12/09/2023 12:33:41 - INFO - datasets.builder - Found cached dataset text (/root/.cache/huggingface/datasets/text/default-99d8365ea3008c01/0.0.0/c4a140d10f020282918b5dd1b8a49f0104729c6177f60a6b49ec2a365ec69f34)\n", "Loading Dataset info from /root/.cache/huggingface/datasets/text/default-99d8365ea3008c01/0.0.0/c4a140d10f020282918b5dd1b8a49f0104729c6177f60a6b49ec2a365ec69f34\n", "12/09/2023 12:33:41 - INFO - datasets.info - Loading Dataset info from /root/.cache/huggingface/datasets/text/default-99d8365ea3008c01/0.0.0/c4a140d10f020282918b5dd1b8a49f0104729c6177f60a6b49ec2a365ec69f34\n", "[INFO|configuration_utils.py:739] 2023-12-09 12:33:41,848 >> loading configuration file config.json from cache at /root/.cache/huggingface/hub/models--gpt2/snapshots/11c5a3d5811f50298f278a704980280950aedb10/config.json\n", "[INFO|configuration_utils.py:802] 2023-12-09 12:33:41,850 >> Model config GPT2Config {\n", " \"_name_or_path\": \"gpt2\",\n", " \"activation_function\": \"gelu_new\",\n", " \"architectures\": [\n", " \"GPT2LMHeadModel\"\n", " ],\n", " \"attn_pdrop\": 0.1,\n", " \"bos_token_id\": 50256,\n", " \"embd_pdrop\": 0.1,\n", " \"eos_token_id\": 50256,\n", " \"initializer_range\": 0.02,\n", " \"layer_norm_epsilon\": 1e-05,\n", " \"model_type\": \"gpt2\",\n", " \"n_ctx\": 1024,\n", " \"n_embd\": 768,\n", " \"n_head\": 12,\n", " \"n_inner\": null,\n", " \"n_layer\": 12,\n", " \"n_positions\": 1024,\n", " \"reorder_and_upcast_attn\": false,\n", " \"resid_pdrop\": 0.1,\n", " \"scale_attn_by_inverse_layer_idx\": false,\n", " \"scale_attn_weights\": true,\n", " \"summary_activation\": null,\n", " \"summary_first_dropout\": 0.1,\n", " \"summary_proj_to_labels\": true,\n", " \"summary_type\": \"cls_index\",\n", " \"summary_use_proj\": true,\n", " \"task_specific_params\": {\n", " \"text-generation\": {\n", " \"do_sample\": true,\n", " \"max_length\": 50\n", " }\n", " },\n", " \"transformers_version\": \"4.36.0.dev0\",\n", " \"use_cache\": true,\n", " \"vocab_size\": 50257\n", "}\n", "\n", "[INFO|tokenization_auto.py:578] 2023-12-09 12:33:41,919 >> Could not locate the tokenizer configuration file, will try to use the model config instead.\n", "[INFO|configuration_utils.py:739] 2023-12-09 12:33:41,991 >> loading configuration file config.json from cache at /root/.cache/huggingface/hub/models--gpt2/snapshots/11c5a3d5811f50298f278a704980280950aedb10/config.json\n", "[INFO|configuration_utils.py:802] 2023-12-09 12:33:41,992 >> Model config GPT2Config {\n", " \"_name_or_path\": \"gpt2\",\n", " \"activation_function\": \"gelu_new\",\n", " \"architectures\": [\n", " \"GPT2LMHeadModel\"\n", " ],\n", " \"attn_pdrop\": 0.1,\n", " \"bos_token_id\": 50256,\n", " \"embd_pdrop\": 0.1,\n", " \"eos_token_id\": 50256,\n", " \"initializer_range\": 0.02,\n", " \"layer_norm_epsilon\": 1e-05,\n", " \"model_type\": \"gpt2\",\n", " \"n_ctx\": 1024,\n", " \"n_embd\": 768,\n", " \"n_head\": 12,\n", " \"n_inner\": null,\n", " \"n_layer\": 12,\n", " \"n_positions\": 1024,\n", " \"reorder_and_upcast_attn\": false,\n", " \"resid_pdrop\": 0.1,\n", " \"scale_attn_by_inverse_layer_idx\": false,\n", " \"scale_attn_weights\": true,\n", " \"summary_activation\": null,\n", " \"summary_first_dropout\": 0.1,\n", " \"summary_proj_to_labels\": true,\n", " \"summary_type\": \"cls_index\",\n", " \"summary_use_proj\": true,\n", " \"task_specific_params\": {\n", " \"text-generation\": {\n", " \"do_sample\": true,\n", " \"max_length\": 50\n", " }\n", " },\n", " \"transformers_version\": \"4.36.0.dev0\",\n", " \"use_cache\": true,\n", " \"vocab_size\": 50257\n", "}\n", "\n", "[INFO|tokenization_utils_base.py:2026] 2023-12-09 12:33:42,155 >> loading file vocab.json from cache at /root/.cache/huggingface/hub/models--gpt2/snapshots/11c5a3d5811f50298f278a704980280950aedb10/vocab.json\n", "[INFO|tokenization_utils_base.py:2026] 2023-12-09 12:33:42,155 >> loading file merges.txt from cache at /root/.cache/huggingface/hub/models--gpt2/snapshots/11c5a3d5811f50298f278a704980280950aedb10/merges.txt\n", "[INFO|tokenization_utils_base.py:2026] 2023-12-09 12:33:42,155 >> loading file tokenizer.json from cache at /root/.cache/huggingface/hub/models--gpt2/snapshots/11c5a3d5811f50298f278a704980280950aedb10/tokenizer.json\n", "[INFO|tokenization_utils_base.py:2026] 2023-12-09 12:33:42,155 >> loading file added_tokens.json from cache at None\n", "[INFO|tokenization_utils_base.py:2026] 2023-12-09 12:33:42,155 >> loading file special_tokens_map.json from cache at None\n", "[INFO|tokenization_utils_base.py:2026] 2023-12-09 12:33:42,155 >> loading file tokenizer_config.json from cache at None\n", "[INFO|configuration_utils.py:739] 2023-12-09 12:33:42,156 >> loading configuration file config.json from cache at /root/.cache/huggingface/hub/models--gpt2/snapshots/11c5a3d5811f50298f278a704980280950aedb10/config.json\n", "[INFO|configuration_utils.py:802] 2023-12-09 12:33:42,157 >> Model config GPT2Config {\n", " \"_name_or_path\": \"gpt2\",\n", " \"activation_function\": \"gelu_new\",\n", " \"architectures\": [\n", " \"GPT2LMHeadModel\"\n", " ],\n", " \"attn_pdrop\": 0.1,\n", " \"bos_token_id\": 50256,\n", " \"embd_pdrop\": 0.1,\n", " \"eos_token_id\": 50256,\n", " \"initializer_range\": 0.02,\n", " \"layer_norm_epsilon\": 1e-05,\n", " \"model_type\": \"gpt2\",\n", " \"n_ctx\": 1024,\n", " \"n_embd\": 768,\n", " \"n_head\": 12,\n", " \"n_inner\": null,\n", " \"n_layer\": 12,\n", " \"n_positions\": 1024,\n", " \"reorder_and_upcast_attn\": false,\n", " \"resid_pdrop\": 0.1,\n", " \"scale_attn_by_inverse_layer_idx\": false,\n", " \"scale_attn_weights\": true,\n", " \"summary_activation\": null,\n", " \"summary_first_dropout\": 0.1,\n", " \"summary_proj_to_labels\": true,\n", " \"summary_type\": \"cls_index\",\n", " \"summary_use_proj\": true,\n", " \"task_specific_params\": {\n", " \"text-generation\": {\n", " \"do_sample\": true,\n", " \"max_length\": 50\n", " }\n", " },\n", " \"transformers_version\": \"4.36.0.dev0\",\n", " \"use_cache\": true,\n", " \"vocab_size\": 50257\n", "}\n", "\n", "model.safetensors: 100% 548M/548M [00:06<00:00, 78.6MB/s]\n", "[INFO|modeling_utils.py:3299] 2023-12-09 12:33:49,589 >> loading weights file model.safetensors from cache at /root/.cache/huggingface/hub/models--gpt2/snapshots/11c5a3d5811f50298f278a704980280950aedb10/model.safetensors\n", "[INFO|configuration_utils.py:826] 2023-12-09 12:33:49,639 >> Generate config GenerationConfig {\n", " \"bos_token_id\": 50256,\n", " \"eos_token_id\": 50256\n", "}\n", "\n", "[INFO|modeling_utils.py:4139] 2023-12-09 12:33:51,494 >> All model checkpoint weights were used when initializing GPT2LMHeadModel.\n", "\n", "[INFO|modeling_utils.py:4147] 2023-12-09 12:33:51,494 >> All the weights of GPT2LMHeadModel were initialized from the model checkpoint at gpt2.\n", "If your task is similar to the task the model of the checkpoint was trained on, you can already use GPT2LMHeadModel for predictions without further training.\n", "generation_config.json: 100% 124/124 [00:00<00:00, 629kB/s]\n", "[INFO|configuration_utils.py:781] 2023-12-09 12:33:51,630 >> loading configuration file generation_config.json from cache at /root/.cache/huggingface/hub/models--gpt2/snapshots/11c5a3d5811f50298f278a704980280950aedb10/generation_config.json\n", "[INFO|configuration_utils.py:826] 2023-12-09 12:33:51,630 >> Generate config GenerationConfig {\n", " \"bos_token_id\": 50256,\n", " \"eos_token_id\": 50256\n", "}\n", "\n", "Running tokenizer on dataset: 0% 0/12296 [00:00> ***** Running training *****\n", "[INFO|trainer.py:1707] 2023-12-09 12:34:09,306 >> Num examples = 4,308\n", "[INFO|trainer.py:1708] 2023-12-09 12:34:09,306 >> Num Epochs = 5\n", "[INFO|trainer.py:1709] 2023-12-09 12:34:09,306 >> Instantaneous batch size per device = 2\n", "[INFO|trainer.py:1712] 2023-12-09 12:34:09,307 >> Total train batch size (w. parallel, distributed & accumulation) = 2\n", "[INFO|trainer.py:1713] 2023-12-09 12:34:09,307 >> Gradient Accumulation steps = 1\n", "[INFO|trainer.py:1714] 2023-12-09 12:34:09,307 >> Total optimization steps = 10,770\n", "[INFO|trainer.py:1715] 2023-12-09 12:34:09,307 >> Number of trainable parameters = 124,439,808\n", "[INFO|integration_utils.py:722] 2023-12-09 12:34:09,308 >> Automatic Weights & Biases logging enabled, to disable set os.environ[\"WANDB_DISABLED\"] = \"true\"\n", "\u001b[34m\u001b[1mwandb\u001b[0m: Currently logged in as: \u001b[33mzakia-salod\u001b[0m (\u001b[33mteam-zakia\u001b[0m). Use \u001b[1m`wandb login --relogin`\u001b[0m to force relogin\n", "\u001b[34m\u001b[1mwandb\u001b[0m: Tracking run with wandb version 0.16.1\n", "\u001b[34m\u001b[1mwandb\u001b[0m: Run data is saved locally in \u001b[35m\u001b[1m/content/wandb/run-20231209_123409-bsxtbazc\u001b[0m\n", "\u001b[34m\u001b[1mwandb\u001b[0m: Run \u001b[1m`wandb offline`\u001b[0m to turn off syncing.\n", "\u001b[34m\u001b[1mwandb\u001b[0m: Syncing run \u001b[33mradiant-firebrand-8\u001b[0m\n", "\u001b[34m\u001b[1mwandb\u001b[0m: ⭐️ View project at \u001b[34m\u001b[4mhttps://wandb.ai/team-zakia/huggingface\u001b[0m\n", "\u001b[34m\u001b[1mwandb\u001b[0m: 🚀 View run at \u001b[34m\u001b[4mhttps://wandb.ai/team-zakia/huggingface/runs/bsxtbazc\u001b[0m\n", "{'loss': 0.7369, 'learning_rate': 4.767873723305478e-05, 'epoch': 0.23}\n", " 5% 500/10770 [06:09<2:07:10, 1.35it/s][INFO|trainer.py:2877] 2023-12-09 12:40:20,104 >> Saving model checkpoint to ./gpt2-drugscom_depression_reviews/tmp-checkpoint-500\n", "[INFO|configuration_utils.py:483] 2023-12-09 12:40:20,106 >> Configuration saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-500/config.json\n", "[INFO|configuration_utils.py:594] 2023-12-09 12:40:20,107 >> Configuration saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-500/generation_config.json\n", "[INFO|modeling_utils.py:2349] 2023-12-09 12:40:22,146 >> Model weights saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-500/pytorch_model.bin\n", "[INFO|tokenization_utils_base.py:2432] 2023-12-09 12:40:22,148 >> tokenizer config file saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-500/tokenizer_config.json\n", "[INFO|tokenization_utils_base.py:2441] 2023-12-09 12:40:22,148 >> Special tokens file saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-500/special_tokens_map.json\n", "{'loss': 0.6937, 'learning_rate': 4.535747446610957e-05, 'epoch': 0.46}\n", " 9% 1000/10770 [12:29<2:00:13, 1.35it/s][INFO|trainer.py:2877] 2023-12-09 12:46:40,220 >> Saving model checkpoint to ./gpt2-drugscom_depression_reviews/tmp-checkpoint-1000\n", "[INFO|configuration_utils.py:483] 2023-12-09 12:46:40,221 >> Configuration saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-1000/config.json\n", "[INFO|configuration_utils.py:594] 2023-12-09 12:46:40,222 >> Configuration saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-1000/generation_config.json\n", "[INFO|modeling_utils.py:2349] 2023-12-09 12:46:44,783 >> Model weights saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-1000/pytorch_model.bin\n", "[INFO|tokenization_utils_base.py:2432] 2023-12-09 12:46:44,785 >> tokenizer config file saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-1000/tokenizer_config.json\n", "[INFO|tokenization_utils_base.py:2441] 2023-12-09 12:46:44,788 >> Special tokens file saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-1000/special_tokens_map.json\n", "{'loss': 0.6743, 'learning_rate': 4.303621169916435e-05, 'epoch': 0.7}\n", " 14% 1500/10770 [18:49<1:54:18, 1.35it/s][INFO|trainer.py:2877] 2023-12-09 12:52:59,800 >> Saving model checkpoint to ./gpt2-drugscom_depression_reviews/tmp-checkpoint-1500\n", "[INFO|configuration_utils.py:483] 2023-12-09 12:52:59,801 >> Configuration saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-1500/config.json\n", "[INFO|configuration_utils.py:594] 2023-12-09 12:52:59,802 >> Configuration saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-1500/generation_config.json\n", "[INFO|modeling_utils.py:2349] 2023-12-09 12:53:01,644 >> Model weights saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-1500/pytorch_model.bin\n", "[INFO|tokenization_utils_base.py:2432] 2023-12-09 12:53:01,645 >> tokenizer config file saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-1500/tokenizer_config.json\n", "[INFO|tokenization_utils_base.py:2441] 2023-12-09 12:53:01,646 >> Special tokens file saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-1500/special_tokens_map.json\n", "{'loss': 0.6565, 'learning_rate': 4.071494893221913e-05, 'epoch': 0.93}\n", " 19% 2000/10770 [25:05<1:48:39, 1.35it/s][INFO|trainer.py:2877] 2023-12-09 12:59:15,817 >> Saving model checkpoint to ./gpt2-drugscom_depression_reviews/tmp-checkpoint-2000\n", "[INFO|configuration_utils.py:483] 2023-12-09 12:59:15,818 >> Configuration saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-2000/config.json\n", "[INFO|configuration_utils.py:594] 2023-12-09 12:59:15,819 >> Configuration saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-2000/generation_config.json\n", "[INFO|modeling_utils.py:2349] 2023-12-09 12:59:17,871 >> Model weights saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-2000/pytorch_model.bin\n", "[INFO|tokenization_utils_base.py:2432] 2023-12-09 12:59:17,872 >> tokenizer config file saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-2000/tokenizer_config.json\n", "[INFO|tokenization_utils_base.py:2441] 2023-12-09 12:59:17,873 >> Special tokens file saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-2000/special_tokens_map.json\n", "{'loss': 0.6237, 'learning_rate': 3.8393686165273915e-05, 'epoch': 1.16}\n", " 23% 2500/10770 [31:29<1:41:45, 1.35it/s][INFO|trainer.py:2877] 2023-12-09 13:05:39,969 >> Saving model checkpoint to ./gpt2-drugscom_depression_reviews/tmp-checkpoint-2500\n", "[INFO|configuration_utils.py:483] 2023-12-09 13:05:39,971 >> Configuration saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-2500/config.json\n", "[INFO|configuration_utils.py:594] 2023-12-09 13:05:39,971 >> Configuration saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-2500/generation_config.json\n", "[INFO|modeling_utils.py:2349] 2023-12-09 13:05:43,191 >> Model weights saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-2500/pytorch_model.bin\n", "[INFO|tokenization_utils_base.py:2432] 2023-12-09 13:05:43,192 >> tokenizer config file saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-2500/tokenizer_config.json\n", "[INFO|tokenization_utils_base.py:2441] 2023-12-09 13:05:43,192 >> Special tokens file saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-2500/special_tokens_map.json\n", "{'loss': 0.6237, 'learning_rate': 3.607242339832869e-05, 'epoch': 1.39}\n", " 28% 3000/10770 [37:48<1:35:53, 1.35it/s][INFO|trainer.py:2877] 2023-12-09 13:11:58,874 >> Saving model checkpoint to ./gpt2-drugscom_depression_reviews/tmp-checkpoint-3000\n", "[INFO|configuration_utils.py:483] 2023-12-09 13:11:58,876 >> Configuration saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-3000/config.json\n", "[INFO|configuration_utils.py:594] 2023-12-09 13:11:58,877 >> Configuration saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-3000/generation_config.json\n", "[INFO|modeling_utils.py:2349] 2023-12-09 13:12:00,785 >> Model weights saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-3000/pytorch_model.bin\n", "[INFO|tokenization_utils_base.py:2432] 2023-12-09 13:12:00,787 >> tokenizer config file saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-3000/tokenizer_config.json\n", "[INFO|tokenization_utils_base.py:2441] 2023-12-09 13:12:00,788 >> Special tokens file saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-3000/special_tokens_map.json\n", "{'loss': 0.6167, 'learning_rate': 3.375116063138347e-05, 'epoch': 1.62}\n", " 32% 3500/10770 [44:04<1:29:41, 1.35it/s][INFO|trainer.py:2877] 2023-12-09 13:18:15,187 >> Saving model checkpoint to ./gpt2-drugscom_depression_reviews/tmp-checkpoint-3500\n", "[INFO|configuration_utils.py:483] 2023-12-09 13:18:15,188 >> Configuration saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-3500/config.json\n", "[INFO|configuration_utils.py:594] 2023-12-09 13:18:15,189 >> Configuration saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-3500/generation_config.json\n", "[INFO|modeling_utils.py:2349] 2023-12-09 13:18:19,719 >> Model weights saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-3500/pytorch_model.bin\n", "[INFO|tokenization_utils_base.py:2432] 2023-12-09 13:18:19,720 >> tokenizer config file saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-3500/tokenizer_config.json\n", "[INFO|tokenization_utils_base.py:2441] 2023-12-09 13:18:19,721 >> Special tokens file saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-3500/special_tokens_map.json\n", "{'loss': 0.6143, 'learning_rate': 3.1429897864438255e-05, 'epoch': 1.86}\n", " 37% 4000/10770 [50:25<1:23:55, 1.34it/s][INFO|trainer.py:2877] 2023-12-09 13:24:35,665 >> Saving model checkpoint to ./gpt2-drugscom_depression_reviews/tmp-checkpoint-4000\n", "[INFO|configuration_utils.py:483] 2023-12-09 13:24:35,667 >> Configuration saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-4000/config.json\n", "[INFO|configuration_utils.py:594] 2023-12-09 13:24:35,668 >> Configuration saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-4000/generation_config.json\n", "[INFO|modeling_utils.py:2349] 2023-12-09 13:24:37,671 >> Model weights saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-4000/pytorch_model.bin\n", "[INFO|tokenization_utils_base.py:2432] 2023-12-09 13:24:37,672 >> tokenizer config file saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-4000/tokenizer_config.json\n", "[INFO|tokenization_utils_base.py:2441] 2023-12-09 13:24:37,673 >> Special tokens file saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-4000/special_tokens_map.json\n", "{'loss': 0.6003, 'learning_rate': 2.9108635097493035e-05, 'epoch': 2.09}\n", " 42% 4500/10770 [56:41<1:17:15, 1.35it/s][INFO|trainer.py:2877] 2023-12-09 13:30:52,074 >> Saving model checkpoint to ./gpt2-drugscom_depression_reviews/tmp-checkpoint-4500\n", "[INFO|configuration_utils.py:483] 2023-12-09 13:30:52,076 >> Configuration saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-4500/config.json\n", "[INFO|configuration_utils.py:594] 2023-12-09 13:30:52,079 >> Configuration saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-4500/generation_config.json\n", "[INFO|modeling_utils.py:2349] 2023-12-09 13:30:54,225 >> Model weights saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-4500/pytorch_model.bin\n", "[INFO|tokenization_utils_base.py:2432] 2023-12-09 13:30:54,227 >> tokenizer config file saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-4500/tokenizer_config.json\n", "[INFO|tokenization_utils_base.py:2441] 2023-12-09 13:30:54,228 >> Special tokens file saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-4500/special_tokens_map.json\n", "{'loss': 0.5719, 'learning_rate': 2.678737233054782e-05, 'epoch': 2.32}\n", " 46% 5000/10770 [1:02:58<1:11:11, 1.35it/s][INFO|trainer.py:2877] 2023-12-09 13:37:08,692 >> Saving model checkpoint to ./gpt2-drugscom_depression_reviews/tmp-checkpoint-5000\n", "[INFO|configuration_utils.py:483] 2023-12-09 13:37:08,693 >> Configuration saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-5000/config.json\n", "[INFO|configuration_utils.py:594] 2023-12-09 13:37:08,694 >> Configuration saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-5000/generation_config.json\n", "[INFO|modeling_utils.py:2349] 2023-12-09 13:37:10,841 >> Model weights saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-5000/pytorch_model.bin\n", "[INFO|tokenization_utils_base.py:2432] 2023-12-09 13:37:10,843 >> tokenizer config file saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-5000/tokenizer_config.json\n", "[INFO|tokenization_utils_base.py:2441] 2023-12-09 13:37:10,846 >> Special tokens file saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-5000/special_tokens_map.json\n", "{'loss': 0.5824, 'learning_rate': 2.44661095636026e-05, 'epoch': 2.55}\n", " 51% 5500/10770 [1:09:14<1:04:55, 1.35it/s][INFO|trainer.py:2877] 2023-12-09 13:43:25,363 >> Saving model checkpoint to ./gpt2-drugscom_depression_reviews/tmp-checkpoint-5500\n", "[INFO|configuration_utils.py:483] 2023-12-09 13:43:25,364 >> Configuration saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-5500/config.json\n", "[INFO|configuration_utils.py:594] 2023-12-09 13:43:25,365 >> Configuration saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-5500/generation_config.json\n", "[INFO|modeling_utils.py:2349] 2023-12-09 13:43:27,212 >> Model weights saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-5500/pytorch_model.bin\n", "[INFO|tokenization_utils_base.py:2432] 2023-12-09 13:43:27,214 >> tokenizer config file saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-5500/tokenizer_config.json\n", "[INFO|tokenization_utils_base.py:2441] 2023-12-09 13:43:27,214 >> Special tokens file saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-5500/special_tokens_map.json\n", "{'loss': 0.5766, 'learning_rate': 2.214484679665738e-05, 'epoch': 2.79}\n", " 56% 6000/10770 [1:15:31<58:47, 1.35it/s][INFO|trainer.py:2877] 2023-12-09 13:49:41,984 >> Saving model checkpoint to ./gpt2-drugscom_depression_reviews/tmp-checkpoint-6000\n", "[INFO|configuration_utils.py:483] 2023-12-09 13:49:41,986 >> Configuration saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-6000/config.json\n", "[INFO|configuration_utils.py:594] 2023-12-09 13:49:41,987 >> Configuration saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-6000/generation_config.json\n", "[INFO|modeling_utils.py:2349] 2023-12-09 13:49:43,886 >> Model weights saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-6000/pytorch_model.bin\n", "[INFO|tokenization_utils_base.py:2432] 2023-12-09 13:49:43,887 >> tokenizer config file saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-6000/tokenizer_config.json\n", "[INFO|tokenization_utils_base.py:2441] 2023-12-09 13:49:43,888 >> Special tokens file saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-6000/special_tokens_map.json\n", "{'loss': 0.5798, 'learning_rate': 1.9823584029712165e-05, 'epoch': 3.02}\n", " 60% 6500/10770 [1:21:47<52:35, 1.35it/s][INFO|trainer.py:2877] 2023-12-09 13:55:58,283 >> Saving model checkpoint to ./gpt2-drugscom_depression_reviews/tmp-checkpoint-6500\n", "[INFO|configuration_utils.py:483] 2023-12-09 13:55:58,284 >> Configuration saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-6500/config.json\n", "[INFO|configuration_utils.py:594] 2023-12-09 13:55:58,286 >> Configuration saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-6500/generation_config.json\n", "[INFO|modeling_utils.py:2349] 2023-12-09 13:56:00,185 >> Model weights saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-6500/pytorch_model.bin\n", "[INFO|tokenization_utils_base.py:2432] 2023-12-09 13:56:00,187 >> tokenizer config file saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-6500/tokenizer_config.json\n", "[INFO|tokenization_utils_base.py:2441] 2023-12-09 13:56:00,187 >> Special tokens file saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-6500/special_tokens_map.json\n", "{'loss': 0.5537, 'learning_rate': 1.7502321262766948e-05, 'epoch': 3.25}\n", " 65% 7000/10770 [1:28:03<46:29, 1.35it/s][INFO|trainer.py:2877] 2023-12-09 14:02:14,355 >> Saving model checkpoint to ./gpt2-drugscom_depression_reviews/tmp-checkpoint-7000\n", "[INFO|configuration_utils.py:483] 2023-12-09 14:02:14,357 >> Configuration saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-7000/config.json\n", "[INFO|configuration_utils.py:594] 2023-12-09 14:02:14,358 >> Configuration saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-7000/generation_config.json\n", "[INFO|modeling_utils.py:2349] 2023-12-09 14:02:17,406 >> Model weights saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-7000/pytorch_model.bin\n", "[INFO|tokenization_utils_base.py:2432] 2023-12-09 14:02:17,407 >> tokenizer config file saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-7000/tokenizer_config.json\n", "[INFO|tokenization_utils_base.py:2441] 2023-12-09 14:02:17,408 >> Special tokens file saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-7000/special_tokens_map.json\n", "{'loss': 0.5553, 'learning_rate': 1.518105849582173e-05, 'epoch': 3.48}\n", " 70% 7500/10770 [1:34:22<40:34, 1.34it/s][INFO|trainer.py:2877] 2023-12-09 14:08:33,229 >> Saving model checkpoint to ./gpt2-drugscom_depression_reviews/tmp-checkpoint-7500\n", "[INFO|configuration_utils.py:483] 2023-12-09 14:08:33,231 >> Configuration saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-7500/config.json\n", "[INFO|configuration_utils.py:594] 2023-12-09 14:08:33,232 >> Configuration saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-7500/generation_config.json\n", "[INFO|modeling_utils.py:2349] 2023-12-09 14:08:35,154 >> Model weights saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-7500/pytorch_model.bin\n", "[INFO|tokenization_utils_base.py:2432] 2023-12-09 14:08:35,155 >> tokenizer config file saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-7500/tokenizer_config.json\n", "[INFO|tokenization_utils_base.py:2441] 2023-12-09 14:08:35,155 >> Special tokens file saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-7500/special_tokens_map.json\n", "{'loss': 0.5479, 'learning_rate': 1.2859795728876508e-05, 'epoch': 3.71}\n", " 74% 8000/10770 [1:40:38<34:27, 1.34it/s][INFO|trainer.py:2877] 2023-12-09 14:14:49,206 >> Saving model checkpoint to ./gpt2-drugscom_depression_reviews/tmp-checkpoint-8000\n", "[INFO|configuration_utils.py:483] 2023-12-09 14:14:49,209 >> Configuration saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-8000/config.json\n", "[INFO|configuration_utils.py:594] 2023-12-09 14:14:49,210 >> Configuration saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-8000/generation_config.json\n", "[INFO|modeling_utils.py:2349] 2023-12-09 14:14:51,216 >> Model weights saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-8000/pytorch_model.bin\n", "[INFO|tokenization_utils_base.py:2432] 2023-12-09 14:14:51,218 >> tokenizer config file saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-8000/tokenizer_config.json\n", "[INFO|tokenization_utils_base.py:2441] 2023-12-09 14:14:51,219 >> Special tokens file saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-8000/special_tokens_map.json\n", "{'loss': 0.5538, 'learning_rate': 1.0538532961931291e-05, 'epoch': 3.95}\n", " 79% 8500/10770 [1:46:55<27:57, 1.35it/s][INFO|trainer.py:2877] 2023-12-09 14:21:06,112 >> Saving model checkpoint to ./gpt2-drugscom_depression_reviews/tmp-checkpoint-8500\n", "[INFO|configuration_utils.py:483] 2023-12-09 14:21:06,114 >> Configuration saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-8500/config.json\n", "[INFO|configuration_utils.py:594] 2023-12-09 14:21:06,115 >> Configuration saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-8500/generation_config.json\n", "[INFO|modeling_utils.py:2349] 2023-12-09 14:21:08,009 >> Model weights saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-8500/pytorch_model.bin\n", "[INFO|tokenization_utils_base.py:2432] 2023-12-09 14:21:08,011 >> tokenizer config file saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-8500/tokenizer_config.json\n", "[INFO|tokenization_utils_base.py:2441] 2023-12-09 14:21:08,012 >> Special tokens file saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-8500/special_tokens_map.json\n", "{'loss': 0.5429, 'learning_rate': 8.217270194986073e-06, 'epoch': 4.18}\n", " 84% 9000/10770 [1:53:11<21:43, 1.36it/s][INFO|trainer.py:2877] 2023-12-09 14:27:22,134 >> Saving model checkpoint to ./gpt2-drugscom_depression_reviews/tmp-checkpoint-9000\n", "[INFO|configuration_utils.py:483] 2023-12-09 14:27:22,135 >> Configuration saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-9000/config.json\n", "[INFO|configuration_utils.py:594] 2023-12-09 14:27:22,136 >> Configuration saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-9000/generation_config.json\n", "[INFO|modeling_utils.py:2349] 2023-12-09 14:27:24,023 >> Model weights saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-9000/pytorch_model.bin\n", "[INFO|tokenization_utils_base.py:2432] 2023-12-09 14:27:24,024 >> tokenizer config file saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-9000/tokenizer_config.json\n", "[INFO|tokenization_utils_base.py:2441] 2023-12-09 14:27:24,025 >> Special tokens file saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-9000/special_tokens_map.json\n", "{'loss': 0.5361, 'learning_rate': 5.896007428040855e-06, 'epoch': 4.41}\n", " 88% 9500/10770 [1:59:27<15:39, 1.35it/s][INFO|trainer.py:2877] 2023-12-09 14:33:38,093 >> Saving model checkpoint to ./gpt2-drugscom_depression_reviews/tmp-checkpoint-9500\n", "[INFO|configuration_utils.py:483] 2023-12-09 14:33:38,095 >> Configuration saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-9500/config.json\n", "[INFO|configuration_utils.py:594] 2023-12-09 14:33:38,096 >> Configuration saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-9500/generation_config.json\n", "[INFO|modeling_utils.py:2349] 2023-12-09 14:33:39,987 >> Model weights saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-9500/pytorch_model.bin\n", "[INFO|tokenization_utils_base.py:2432] 2023-12-09 14:33:39,988 >> tokenizer config file saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-9500/tokenizer_config.json\n", "[INFO|tokenization_utils_base.py:2441] 2023-12-09 14:33:39,988 >> Special tokens file saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-9500/special_tokens_map.json\n", "{'loss': 0.5425, 'learning_rate': 3.574744661095636e-06, 'epoch': 4.64}\n", " 93% 10000/10770 [2:05:51<09:27, 1.36it/s][INFO|trainer.py:2877] 2023-12-09 14:40:02,357 >> Saving model checkpoint to ./gpt2-drugscom_depression_reviews/tmp-checkpoint-10000\n", "[INFO|configuration_utils.py:483] 2023-12-09 14:40:02,358 >> Configuration saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-10000/config.json\n", "[INFO|configuration_utils.py:594] 2023-12-09 14:40:02,359 >> Configuration saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-10000/generation_config.json\n", "[INFO|modeling_utils.py:2349] 2023-12-09 14:40:08,494 >> Model weights saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-10000/pytorch_model.bin\n", "[INFO|tokenization_utils_base.py:2432] 2023-12-09 14:40:08,501 >> tokenizer config file saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-10000/tokenizer_config.json\n", "[INFO|tokenization_utils_base.py:2441] 2023-12-09 14:40:08,502 >> Special tokens file saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-10000/special_tokens_map.json\n", "{'loss': 0.5388, 'learning_rate': 1.253481894150418e-06, 'epoch': 4.87}\n", " 97% 10500/10770 [2:12:12<03:20, 1.35it/s][INFO|trainer.py:2877] 2023-12-09 14:46:23,328 >> Saving model checkpoint to ./gpt2-drugscom_depression_reviews/tmp-checkpoint-10500\n", "[INFO|configuration_utils.py:483] 2023-12-09 14:46:23,330 >> Configuration saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-10500/config.json\n", "[INFO|configuration_utils.py:594] 2023-12-09 14:46:23,331 >> Configuration saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-10500/generation_config.json\n", "[INFO|modeling_utils.py:2349] 2023-12-09 14:46:25,338 >> Model weights saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-10500/pytorch_model.bin\n", "[INFO|tokenization_utils_base.py:2432] 2023-12-09 14:46:25,339 >> tokenizer config file saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-10500/tokenizer_config.json\n", "[INFO|tokenization_utils_base.py:2441] 2023-12-09 14:46:25,339 >> Special tokens file saved in ./gpt2-drugscom_depression_reviews/tmp-checkpoint-10500/special_tokens_map.json\n", "100% 10770/10770 [2:15:39<00:00, 1.36it/s][INFO|trainer.py:1947] 2023-12-09 14:49:49,384 >> \n", "\n", "Training completed. Do not forget to share your model on huggingface.co/models =)\n", "\n", "\n", "{'train_runtime': 8140.1135, 'train_samples_per_second': 2.646, 'train_steps_per_second': 1.323, 'train_loss': 0.5943892072502285, 'epoch': 5.0}\n", "100% 10770/10770 [2:15:39<00:00, 1.32it/s]\n", "[INFO|trainer.py:2877] 2023-12-09 14:49:49,424 >> Saving model checkpoint to ./gpt2-drugscom_depression_reviews\n", "[INFO|configuration_utils.py:483] 2023-12-09 14:49:49,429 >> Configuration saved in ./gpt2-drugscom_depression_reviews/config.json\n", "[INFO|configuration_utils.py:594] 2023-12-09 14:49:49,430 >> Configuration saved in ./gpt2-drugscom_depression_reviews/generation_config.json\n", "[INFO|modeling_utils.py:2349] 2023-12-09 14:49:51,305 >> Model weights saved in ./gpt2-drugscom_depression_reviews/pytorch_model.bin\n", "[INFO|tokenization_utils_base.py:2432] 2023-12-09 14:49:51,306 >> tokenizer config file saved in ./gpt2-drugscom_depression_reviews/tokenizer_config.json\n", "[INFO|tokenization_utils_base.py:2441] 2023-12-09 14:49:51,307 >> Special tokens file saved in ./gpt2-drugscom_depression_reviews/special_tokens_map.json\n", "***** train metrics *****\n", " epoch = 5.0\n", " train_loss = 0.5944\n", " train_runtime = 2:15:40.11\n", " train_samples = 4308\n", " train_samples_per_second = 2.646\n", " train_steps_per_second = 1.323\n", "[INFO|modelcard.py:452] 2023-12-09 14:49:51,959 >> Dropping the following result as it does not have all the necessary fields:\n", "{'task': {'name': 'Causal Language Modeling', 'type': 'text-generation'}}\n" ] } ], "source": [ "# Fine-tuning command for Google Colab\n", "# Note: run_clm.py should be present in the working directory\n", "!python run_clm.py \\\n", " --train_file train_dataset.txt \\\n", " --model_name_or_path gpt2 \\\n", " --output_dir ./gpt2-drugscom_depression_reviews \\\n", " --do_train \\\n", " --per_device_train_batch_size 2 \\\n", " --num_train_epochs 5 \\\n", " --overwrite_output_dir \\\n", " --use_fast_tokenizer True \\\n", " --report_to \"wandb\"" ] }, { "cell_type": "markdown", "metadata": { "id": "EFH89vCQ0pMf" }, "source": [ "## STEP 5: SAVE THE FINE-TUNED GPT-2 MODEL: gpt2-drugscom_depression_reviews" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "LthX-XqW5Agq" }, "outputs": [], "source": [ "from huggingface_hub import notebook_login # To log to our Hugging Face account to be able to upload models to the Hub." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "X_g76SB35CDE", "colab": { "base_uri": "https://localhost:8080/", "height": 145, "referenced_widgets": [ "9290ccab8e20428a8c6a61e2ce67e0c7", "95db8459bbae45c9ac3b05898c06dc70", "983e2f804ad24fe3b34a33c800417ca2", "231839169648413991d11e48bb43a7a7", "bd9f134683a84df09aadcab94fad8831", "f6933a18f84f4678a360ffcc3e13f083", "15fdfbc426924d5091533d4462c9b53b", "44aa945dc5e743b8a6aa3f3078caac77", "fa8d2f6085ab4a31b61270aa6a1ea4c5", "7f3d379eb619439c98379c9b4f0795ec", "6b048e6506aa454b9c5858828fbcd40c", "4690ab39fdb348359cb42526a3ebeb86", "0c3bc5d3a65d4a8980e321caf5a5d4f6", "b5ea618a490448b4aa95637a03f9adb0", "379537673a7b46a8bdcdbfdf346be8dd", "bef797322e0940a0a4b0116bc502480d", "8f9941f463cc4e7688daf9d70fb8937e", "9502b7944c4948cd9850aa515ba5f2d3", "56eeae6e18784dd2903addb05112e495", "8bc3ad27022a45c29c0d7eac1da9956f", "9fec786322d047299a289cb80487ed52", "99801da567c5403284ac7555ea967dd3", "d2dfcff538f54c449386cc843c2e7c6d", "7728b2f00d96487a9397a55c3cd336ce", "77dfd24b45f24da09dfe0c426d741aa7", "c5313e81a1784396afd3cdc03f0920ea", "65e11d956f2b4f59a9ccc26176cd4a58", "452967ad3cd94a21bcde37e3ea2784eb", "9fbe272599ed450e840a918a8b6a1fbc", "d3c9c4c34afd4ebfb8b911c2aac756b5", "1852d779e7c8448d87ce381d4348868d", "6a2cbfb5e7b64c8cac850d0b05ce8f47" ] }, "outputId": "52df4679-af8f-4961-b006-6d11ee1ec163" }, "outputs": [ { "output_type": "display_data", "data": { "text/plain": [ "VBox(children=(HTML(value='

Copy a token from your Hugging Face\ntokens page and paste it below.
Immediately click login after copying\nyour token or it might be stored in plain text in this notebook file.
" } }, "983e2f804ad24fe3b34a33c800417ca2": { "model_module": "@jupyter-widgets/controls", "model_name": "PasswordModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "PasswordModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "PasswordView", "continuous_update": true, "description": "Token:", "description_tooltip": null, "disabled": false, "layout": "IPY_MODEL_7f3d379eb619439c98379c9b4f0795ec", "placeholder": "​", "style": "IPY_MODEL_6b048e6506aa454b9c5858828fbcd40c", "value": "" } }, "231839169648413991d11e48bb43a7a7": { "model_module": "@jupyter-widgets/controls", "model_name": "CheckboxModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "CheckboxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "CheckboxView", "description": "Add token as git credential?", "description_tooltip": null, "disabled": false, "indent": true, "layout": "IPY_MODEL_4690ab39fdb348359cb42526a3ebeb86", "style": "IPY_MODEL_0c3bc5d3a65d4a8980e321caf5a5d4f6", "value": true } }, "bd9f134683a84df09aadcab94fad8831": { "model_module": "@jupyter-widgets/controls", "model_name": "ButtonModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ButtonModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ButtonView", "button_style": "", "description": "Login", "disabled": false, "icon": "", "layout": "IPY_MODEL_b5ea618a490448b4aa95637a03f9adb0", "style": "IPY_MODEL_379537673a7b46a8bdcdbfdf346be8dd", "tooltip": "" } }, "f6933a18f84f4678a360ffcc3e13f083": { "model_module": "@jupyter-widgets/controls", "model_name": "HTMLModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_bef797322e0940a0a4b0116bc502480d", "placeholder": "​", "style": "IPY_MODEL_8f9941f463cc4e7688daf9d70fb8937e", "value": "\nPro Tip: If you don't already have one, you can create a dedicated\n'notebooks' token with 'write' access, that you can then easily reuse for all\nnotebooks. " } }, "15fdfbc426924d5091533d4462c9b53b": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": "center", "align_self": null, "border": null, "bottom": null, "display": "flex", "flex": null, "flex_flow": "column", "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": "50%" } }, "44aa945dc5e743b8a6aa3f3078caac77": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "fa8d2f6085ab4a31b61270aa6a1ea4c5": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "7f3d379eb619439c98379c9b4f0795ec": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "6b048e6506aa454b9c5858828fbcd40c": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "4690ab39fdb348359cb42526a3ebeb86": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "0c3bc5d3a65d4a8980e321caf5a5d4f6": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "b5ea618a490448b4aa95637a03f9adb0": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "379537673a7b46a8bdcdbfdf346be8dd": { "model_module": "@jupyter-widgets/controls", "model_name": "ButtonStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ButtonStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "button_color": null, "font_weight": "" } }, "bef797322e0940a0a4b0116bc502480d": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "8f9941f463cc4e7688daf9d70fb8937e": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "9502b7944c4948cd9850aa515ba5f2d3": { "model_module": "@jupyter-widgets/controls", "model_name": "LabelModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "LabelModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "LabelView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_56eeae6e18784dd2903addb05112e495", "placeholder": "​", "style": "IPY_MODEL_8bc3ad27022a45c29c0d7eac1da9956f", "value": "Connecting..." } }, "56eeae6e18784dd2903addb05112e495": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "8bc3ad27022a45c29c0d7eac1da9956f": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "9fec786322d047299a289cb80487ed52": { "model_module": "@jupyter-widgets/controls", "model_name": "LabelModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "LabelModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "LabelView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_77dfd24b45f24da09dfe0c426d741aa7", "placeholder": "​", "style": "IPY_MODEL_c5313e81a1784396afd3cdc03f0920ea", "value": "Token is valid (permission: write)." } }, "99801da567c5403284ac7555ea967dd3": { "model_module": "@jupyter-widgets/controls", "model_name": "LabelModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "LabelModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "LabelView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_65e11d956f2b4f59a9ccc26176cd4a58", "placeholder": "​", "style": "IPY_MODEL_452967ad3cd94a21bcde37e3ea2784eb", "value": "Your token has been saved in your configured git credential helpers (store)." } }, "d2dfcff538f54c449386cc843c2e7c6d": { "model_module": "@jupyter-widgets/controls", "model_name": "LabelModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "LabelModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "LabelView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_9fbe272599ed450e840a918a8b6a1fbc", "placeholder": "​", "style": "IPY_MODEL_d3c9c4c34afd4ebfb8b911c2aac756b5", "value": "Your token has been saved to /root/.cache/huggingface/token" } }, "7728b2f00d96487a9397a55c3cd336ce": { "model_module": "@jupyter-widgets/controls", "model_name": "LabelModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "LabelModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "LabelView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_1852d779e7c8448d87ce381d4348868d", "placeholder": "​", "style": "IPY_MODEL_6a2cbfb5e7b64c8cac850d0b05ce8f47", "value": "Login successful" } }, "77dfd24b45f24da09dfe0c426d741aa7": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "c5313e81a1784396afd3cdc03f0920ea": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "65e11d956f2b4f59a9ccc26176cd4a58": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "452967ad3cd94a21bcde37e3ea2784eb": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "9fbe272599ed450e840a918a8b6a1fbc": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "d3c9c4c34afd4ebfb8b911c2aac756b5": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "1852d779e7c8448d87ce381d4348868d": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "6a2cbfb5e7b64c8cac850d0b05ce8f47": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "d2ce25900eae4b41b8107401f174b78d": { "model_module": "@jupyter-widgets/controls", "model_name": "HBoxModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_84433208e40d4271b4db101602d267ef", "IPY_MODEL_c58122cb38ef4486a96966ac861cf913", "IPY_MODEL_121add4c3f014c04926701a1da4782f0" ], "layout": "IPY_MODEL_74c8e8e472904e6c9651509d8bfc6a50" } }, "84433208e40d4271b4db101602d267ef": { "model_module": "@jupyter-widgets/controls", "model_name": "HTMLModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_9ada60f4d6374e3ab1d616970fea8b57", "placeholder": "​", "style": "IPY_MODEL_7ce21dd248a047fe8b4a8be37a05bfac", "value": "model.safetensors: 100%" } }, "c58122cb38ef4486a96966ac861cf913": { "model_module": "@jupyter-widgets/controls", "model_name": "FloatProgressModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_a3a9ece98352459baa56c051886a8924", "max": 497774208, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_ea47214f28704e6ea43cb064bf9f5294", "value": 497774208 } }, "121add4c3f014c04926701a1da4782f0": { "model_module": "@jupyter-widgets/controls", "model_name": "HTMLModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_c43afb5ad3ed429285c3a6d871a734d7", "placeholder": "​", "style": "IPY_MODEL_62a83b49f10f49ee9a8b85ad601b45c0", "value": " 498M/498M [00:17<00:00, 26.5MB/s]" } }, "74c8e8e472904e6c9651509d8bfc6a50": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "9ada60f4d6374e3ab1d616970fea8b57": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "7ce21dd248a047fe8b4a8be37a05bfac": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "a3a9ece98352459baa56c051886a8924": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "ea47214f28704e6ea43cb064bf9f5294": { "model_module": "@jupyter-widgets/controls", "model_name": "ProgressStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "c43afb5ad3ed429285c3a6d871a734d7": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "62a83b49f10f49ee9a8b85ad601b45c0": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } } } } }, "nbformat": 4, "nbformat_minor": 0 }