{ "cells": [ { "cell_type": "markdown", "metadata": { "id": "GZiMfnKVCniS" }, "source": [ "# PROYECTO III PROGRAMA DE FORMACIÓN MLDS AVANZADO\n", "## Daniel F. Benavides R. \n", "## Módulo VI - Entrenamiento de modelo de red neuronal y disposición del mismo a nivel local. \n", "\n", "### OBJETIVO\n", "\n", "El objetivo de este proyecto es realizar el despliegue de un modelo a nivel local. El mismo se llevará a cabo en dos partes: La primera en la cual se realiza el entrenamiento del modelo. El mismo se guarda a nivel local para su posterior uso. \n", "\n", "Es así como a continuación se ve el ejercicio de fine-tuning del modelo preentrenado de transformers [_'distilbert-base-uncased'_](https://huggingface.co/distilbert-base-uncased?text=Paris+is+the+%5BMASK%5D+of+France.). Este modelo inicialmente fue entrenado para labores de _fill mask_ y se adaptará como modelo clasificación de **SMS** no deseado. \n" ] }, { "cell_type": "code", "execution_count": 33, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "gTYjO-PJIUT-", "outputId": "27f1d993-d34f-4f4b-bbe1-ab6c96bc2825" }, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "Drive already mounted at /content/drive; to attempt to forcibly remount, call drive.mount(\"/content/drive\", force_remount=True).\n" ] } ], "source": [ "from google.colab import drive\n", "drive.mount('/content/drive')" ] }, { "cell_type": "markdown", "metadata": { "id": "GRFpQTqzDCLT" }, "source": [ "### Carga y manipulación de los datos \n", "\n", "A continuación importamos pandas, por medio del cual hacemos el respectivo cargue del dataset, delimitamos por el espacio la etiqueta del mensaje.\n", "\n", "Luego por medio de la función _list_ convertimos el mensaje y las etiquetas en un par de listas. luego convertimos las etiquetas en una variable dummie, debido a que tenemos una salida binaria _(el mensaje es spam o no lo es)_\n", "\n", "## Importamos el dataset de entrenamiento" ] }, { "cell_type": "code", "execution_count": 34, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "3Dt9fFKs74zR", "outputId": "271ff620-a449-4e87-a163-14621651b48e" }, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "Drive already mounted at /content/drive; to attempt to forcibly remount, call drive.mount(\"/content/drive\", force_remount=True).\n" ] } ], "source": [ "from google.colab import drive\n", "drive.mount('/content/drive')\n", "path= \"/content/drive/MyDrive/MLDS-2/MODULO II/Talleres/SMSSpamCollection\"" ] }, { "cell_type": "code", "execution_count": 35, "metadata": { "id": "awPXefiYqQsF" }, "outputs": [], "source": [ "\n", "import pandas as pd\n", "df=messages = pd.read_csv(path, sep='\\t',\n", " names=[\"label\", \"message\"])\n", "X=list(df['message'])\n", "y=list(df['label'])\n", "y=list(pd.get_dummies(y,drop_first=True)['spam'])\n" ] }, { "cell_type": "markdown", "metadata": { "id": "1T8CpN3YDar6" }, "source": [ "### Preprocesamiento \n", "\n", "Ahora importamos la función *train_test_split* del módulo *model_selection* de la librería *scikit-learn* y por medio de este dividimos en set de entrenamiento y prueba. Definimos el tamaño de set de prueba en 20% de la muestra. También definimos el parámetro *random_state* para efectos de controlar la generación de los dos conjuntos de tal manera que no sean aleatorios. \n", "\n", "Luego instalamos la librería transformers, aunque en mi caso ya lo había realizado. \n" ] }, { "cell_type": "code", "execution_count": 36, "metadata": { "id": "dLFDWda0rIKw" }, "outputs": [], "source": [ "from sklearn.model_selection import train_test_split\n", "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.20, random_state = 0)" ] }, { "cell_type": "code", "execution_count": 37, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "AqOBGiGErZgj", "outputId": "1a461d33-55ae-4a22-9746-8c01e99d49bd" }, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n", "Requirement already satisfied: transformers in /usr/local/lib/python3.8/dist-packages (4.25.1)\n", "Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.8/dist-packages (from transformers) (21.3)\n", "Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.8/dist-packages (from transformers) (2022.6.2)\n", "Requirement already satisfied: huggingface-hub<1.0,>=0.10.0 in /usr/local/lib/python3.8/dist-packages (from transformers) (0.11.1)\n", "Requirement already satisfied: filelock in /usr/local/lib/python3.8/dist-packages (from transformers) (3.8.2)\n", "Requirement already satisfied: requests in /usr/local/lib/python3.8/dist-packages (from transformers) (2.23.0)\n", "Requirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.8/dist-packages (from transformers) (4.64.1)\n", "Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.8/dist-packages (from transformers) (1.21.6)\n", "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.8/dist-packages (from transformers) (6.0)\n", "Requirement already satisfied: tokenizers!=0.11.3,<0.14,>=0.11.1 in /usr/local/lib/python3.8/dist-packages (from transformers) (0.13.2)\n", "Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.8/dist-packages (from huggingface-hub<1.0,>=0.10.0->transformers) (4.4.0)\n", "Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.8/dist-packages (from packaging>=20.0->transformers) (3.0.9)\n", "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.8/dist-packages (from requests->transformers) (2022.12.7)\n", "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.8/dist-packages (from requests->transformers) (1.24.3)\n", "Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.8/dist-packages (from requests->transformers) (2.10)\n", "Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.8/dist-packages (from requests->transformers) (3.0.4)\n" ] } ], "source": [ "!pip install transformers" ] }, { "cell_type": "markdown", "metadata": { "id": "31aeQ6u-Dq-K" }, "source": [ "Ahora debemos invocar los modelos de que vamos a utilizar de la librería transformers en los siguientes pasos: \n", "\n", "* Llamamos el modelo preentrenado\n", "* Llamamos el tokenizador \n", "\n", "Necesitamos aplicar el tokenizador sobre nuestro conjunto de datos. \n", "\n", "Así que acontinuación llamamos de la librería transformers el tokenizador _\"DistilBertTokenizerFast\"_ luego lo definimos como nuestro **tokenizer** indicando que el mismo proviene del modelo preentrenado [_'distilbert-base-uncased'_](https://huggingface.co/distilbert-base-uncased?text=Paris+is+the+%5BMASK%5D+of+France.)" ] }, { "cell_type": "code", "execution_count": 38, "metadata": { "id": "bcNEJ6perOSs" }, "outputs": [], "source": [ "from transformers import DistilBertTokenizerFast\n", "tokenizer = DistilBertTokenizerFast.from_pretrained('distilbert-base-uncased')" ] }, { "cell_type": "markdown", "metadata": { "id": "3RdR0eZaDyyi" }, "source": [ "Luego aplicamos el tokenizador que acabamos de definir sobre nuestro conjunto de mensajes de entrenamiento y prueba. Como los SMS no tienen la misma longitud (cantidad de tokens) debemos definir los parámetros truncation y padding como True para que se obtener oraciones del mismo tamaño; uno se encarga de rellenar de ceros (padding) y el otro de truncar las oraciones más largas. Esto para obtener un conjunto y luego tensores rectangulares. " ] }, { "cell_type": "code", "execution_count": 39, "metadata": { "id": "-OL3fgLvrXvH" }, "outputs": [], "source": [ "train_encodings = tokenizer(X_train, truncation=True, padding=True)\n", "test_encodings = tokenizer(X_test, truncation=True, padding=True)\n" ] }, { "cell_type": "markdown", "metadata": { "id": "7JTdRQNVD4AK" }, "source": [ "Ahora se procede a importar Tensorflow para efecto de convertir en tensores los encodings generados en el paso anterior. Acá se junta cada uno a su correspondiente etiqueta. " ] }, { "cell_type": "code", "execution_count": 40, "metadata": { "id": "9B42CTCnrrEx" }, "outputs": [], "source": [ "import tensorflow as tf\n", "\n", "train_dataset = tf.data.Dataset.from_tensor_slices((\n", " dict(train_encodings),\n", " y_train\n", "))\n", "\n", "test_dataset = tf.data.Dataset.from_tensor_slices((\n", " dict(test_encodings),\n", " y_test\n", "))" ] }, { "cell_type": "markdown", "metadata": { "id": "G3Wj2cqXD5hx" }, "source": [ "### Entrenamiento\n", "\n", "A continuación se importan los módulos de TFDistilBertForSequenceClassification que es usado para la tarea de clasificación de sentimientos. También se importan los módulos y *TFTrainingArguments* y *TFTrainer*que son los encargados de definir los argumentos y posteriormente parametrizar el **trainer** del modelo y hacer las nuevas predicciones. \n" ] }, { "cell_type": "code", "execution_count": 41, "metadata": { "id": "NH1dupK0rzfn" }, "outputs": [], "source": [ "from transformers import TFDistilBertForSequenceClassification, TFTrainer, TFTrainingArguments\n", "\n", "training_args = TFTrainingArguments(\n", " eval_steps = 10, \n", " output_dir='./results', # output directory\n", " num_train_epochs=2, # total number of training epochs\n", " per_device_train_batch_size=8, # batch size per device during training\n", " per_device_eval_batch_size=8, # batch size for evaluation\n", " warmup_steps=500, # number of warmup steps for learning rate scheduler\n", " weight_decay=0.01, # strength of weight decay\n", " logging_dir='./logs', # directory for storing logs\n", " logging_steps=10,\n", ")" ] }, { "cell_type": "markdown", "metadata": { "id": "cqmXOhkSEAgC" }, "source": [ "Hemos determinado el conjunto de argumentos que serán utilizados en el reentrenamiento del modelo,estos quedan alojados en el objeto *training_args* y ahora definiremos el modelo refiriendo el modelo preentrenado que vamos a utilizar, que en este caso es _\"distilbert-base-uncased\"_. se creará el **trainer** al cual se le pasarán los argumentos antes definidos y los dos tensores de entrenamiento y prueba; para luego entrenar el modelo que hemos definido." ] }, { "cell_type": "code", "execution_count": 42, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "PZvTrEcfr7k-", "outputId": "f8fe7e3c-c9c8-4b92-c3fc-ea4821386beb" }, "outputs": [ { "output_type": "stream", "name": "stderr", "text": [ "Some layers from the model checkpoint at distilbert-base-uncased were not used when initializing TFDistilBertForSequenceClassification: ['vocab_projector', 'activation_13', 'vocab_layer_norm', 'vocab_transform']\n", "- This IS expected if you are initializing TFDistilBertForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", "- This IS NOT expected if you are initializing TFDistilBertForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", "Some layers of TFDistilBertForSequenceClassification were not initialized from the model checkpoint at distilbert-base-uncased and are newly initialized: ['dropout_39', 'pre_classifier', 'classifier']\n", "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n", "/usr/local/lib/python3.8/dist-packages/transformers/trainer_tf.py:115: FutureWarning: The class `TFTrainer` is deprecated and will be removed in version 5 of Transformers. We recommend using native Keras instead, by calling methods like `fit()` and `predict()` directly on the model object. Detailed examples of the Keras style can be found in our examples at https://github.com/huggingface/transformers/tree/main/examples/tensorflow\n", " warnings.warn(\n" ] } ], "source": [ "with training_args.strategy.scope():\n", " model = TFDistilBertForSequenceClassification.from_pretrained(\"distilbert-base-uncased\")\n", "\n", "trainer = TFTrainer(\n", " model=model, # the instantiated 🤗 Transformers model to be trained\n", " args=training_args, # training arguments, defined above\n", " train_dataset=train_dataset, # training dataset\n", " eval_dataset=test_dataset # evaluation dataset\n", ")\n" ] }, { "cell_type": "markdown", "metadata": { "id": "tnAE3agZ21dq" }, "source": [ "una vez instanciado el modelo que será reentrenado, parametrizados los argumentos para ello, se toma la data y se realiza el reentrenamiento del modelo. " ] }, { "cell_type": "code", "execution_count": 43, "metadata": { "id": "bIba4vQg7Ecp" }, "outputs": [], "source": [ "trainer.train()" ] }, { "cell_type": "markdown", "metadata": { "id": "Zerz-bv8EENp" }, "source": [ "Ahora solo queda por aplicar modelo que reentrenamos con el dataset de **entrenamiento**, hacer la predicción, y la evaluación de las predicciones. Este procedimiento se encuentra definido en el [manual de fine-tuning](https://huggingface.co/transformers/v3.5.1/training.html) que tiene Hugging Face disponible. " ] }, { "cell_type": "code", "execution_count": 44, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "R534aDi3xD0s", "outputId": "65c5ac93-eb67-4413-e048-f7b4d9fd8931" }, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "{'eval_loss': 0.02398163080215454}" ] }, "metadata": {}, "execution_count": 44 } ], "source": [ "trainer.evaluate(test_dataset)" ] }, { "cell_type": "markdown", "metadata": { "id": "4rLF3nApUndt" }, "source": [ "A continuación aplicamos el modelo reentrenado al set de prueba hacer la respectiva clasificación de cada una de las muestras. " ] }, { "cell_type": "markdown", "metadata": { "id": "jpGNNvWEWU9u" }, "source": [ "### Predicción del modelo\n", "\n", "Se aplica el modelo reentrenado al dataset de prueba *test_dataset* y se evalúa la precisión del mismo por medio del accuracy, es decir, acá le pasamos mensajes sin etiquetas y le pedimos que prediga si son o no spam. El modelo para la tarea que fue entrenado presenta un accuracy de 1, es decir clasifica perfectamente el set de prueba. " ] }, { "cell_type": "code", "execution_count": 45, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "UyBmI1WcxKjG", "outputId": "53067a82-55bf-4500-a38e-d890be6f7bf5" }, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "PredictionOutput(predictions=array([[ 3.4155877, -3.1767924],\n", " [-3.2374823, 3.135958 ],\n", " [ 3.348417 , -3.1216612],\n", " ...,\n", " [ 3.04905 , -2.8354154],\n", " [-3.1865208, 3.0687277],\n", " [ 3.212608 , -3.0316095]], dtype=float32), label_ids=array([0, 1, 0, ..., 0, 1, 0], dtype=int32), metrics={'eval_loss': 0.023984665530068533})" ] }, "metadata": {}, "execution_count": 45 } ], "source": [ "trainer.predict(test_dataset)" ] }, { "cell_type": "code", "execution_count": 46, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "9Qc5FtM8xn9A", "outputId": "0d517424-b5d1-4324-be3c-6f90335aa4fd" }, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "(1115,)" ] }, "metadata": {}, "execution_count": 46 } ], "source": [ "trainer.predict(test_dataset)[1].shape" ] }, { "cell_type": "markdown", "metadata": { "id": "LUHX_tCTWFuu" }, "source": [ "#### Salidas" ] }, { "cell_type": "code", "execution_count": 47, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "fUVX_IhWxkxg", "outputId": "a2e94ee6-54a2-414f-c2e7-98950deb7732" }, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "array([0, 1, 0, ..., 0, 1, 0], dtype=int32)" ] }, "metadata": {}, "execution_count": 47 } ], "source": [ "output=trainer.predict(test_dataset)[1]\n", "output" ] }, { "cell_type": "markdown", "metadata": { "id": "lUxvb6JcYB7_" }, "source": [ "#### Matriz de confusión, Accuracy" ] }, { "cell_type": "code", "execution_count": 48, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "cfCE06jQu5cI", "outputId": "a1d10897-a36f-47a8-e038-0f68ec5e7ded" }, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "array([[955, 0],\n", " [ 0, 160]])" ] }, "metadata": {}, "execution_count": 48 } ], "source": [ "from sklearn.metrics import confusion_matrix, accuracy_score\n", "\n", "confusion_matrix=confusion_matrix(y_test,output)\n", "confusion_matrix\n" ] }, { "cell_type": "code", "execution_count": 49, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "mv83DD8sl8JO", "outputId": "97612c62-b15f-453f-d51e-5cd12e554421" }, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "1.0" ] }, "metadata": {}, "execution_count": 49 } ], "source": [ "acc=accuracy_score(y_test,output)\n", "acc" ] }, { "cell_type": "markdown", "metadata": { "id": "Zm3mF58zYYze" }, "source": [ "#### Descarga del modelo" ] }, { "cell_type": "code", "execution_count": 59, "metadata": { "id": "okD5we1NwhQW" }, "outputs": [], "source": [ "trainer.save_model('ft_model')\n", "trainer.save_model('/content/drive/MyDrive/MLDS-2/MODULO III/Talleres/Modelo Entrenado')\n" ] }, { "cell_type": "code", "source": [ "!pip install transformers" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "iuipIt7zN8Ct", "outputId": "f078f829-8827-4e20-daed-9baf1e007394" }, "execution_count": 60, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n", "Requirement already satisfied: transformers in /usr/local/lib/python3.8/dist-packages (4.25.1)\n", "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.8/dist-packages (from transformers) (6.0)\n", "Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.8/dist-packages (from transformers) (1.21.6)\n", "Requirement already satisfied: huggingface-hub<1.0,>=0.10.0 in /usr/local/lib/python3.8/dist-packages (from transformers) (0.11.1)\n", "Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.8/dist-packages (from transformers) (2022.6.2)\n", "Requirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.8/dist-packages (from transformers) (4.64.1)\n", "Requirement already satisfied: tokenizers!=0.11.3,<0.14,>=0.11.1 in /usr/local/lib/python3.8/dist-packages (from transformers) (0.13.2)\n", "Requirement already satisfied: filelock in /usr/local/lib/python3.8/dist-packages (from transformers) (3.8.2)\n", "Requirement already satisfied: requests in /usr/local/lib/python3.8/dist-packages (from transformers) (2.23.0)\n", "Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.8/dist-packages (from transformers) (21.3)\n", "Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.8/dist-packages (from huggingface-hub<1.0,>=0.10.0->transformers) (4.4.0)\n", "Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.8/dist-packages (from packaging>=20.0->transformers) (3.0.9)\n", "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.8/dist-packages (from requests->transformers) (1.24.3)\n", "Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.8/dist-packages (from requests->transformers) (2.10)\n", "Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.8/dist-packages (from requests->transformers) (3.0.4)\n", "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.8/dist-packages (from requests->transformers) (2022.12.7)\n" ] } ] }, { "cell_type": "code", "source": [ "!pip install huggingface_hub" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "Oo5x6eVZN7_9", "outputId": "21c914ea-8540-4192-e11d-f77d3774fef1" }, "execution_count": 61, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n", "Requirement already satisfied: huggingface_hub in /usr/local/lib/python3.8/dist-packages (0.11.1)\n", "Requirement already satisfied: packaging>=20.9 in /usr/local/lib/python3.8/dist-packages (from huggingface_hub) (21.3)\n", "Requirement already satisfied: filelock in /usr/local/lib/python3.8/dist-packages (from huggingface_hub) (3.8.2)\n", "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.8/dist-packages (from huggingface_hub) (6.0)\n", "Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.8/dist-packages (from huggingface_hub) (4.4.0)\n", "Requirement already satisfied: tqdm in /usr/local/lib/python3.8/dist-packages (from huggingface_hub) (4.64.1)\n", "Requirement already satisfied: requests in /usr/local/lib/python3.8/dist-packages (from huggingface_hub) (2.23.0)\n", "Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.8/dist-packages (from packaging>=20.9->huggingface_hub) (3.0.9)\n", "Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.8/dist-packages (from requests->huggingface_hub) (2.10)\n", "Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.8/dist-packages (from requests->huggingface_hub) (3.0.4)\n", "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.8/dist-packages (from requests->huggingface_hub) (1.24.3)\n", "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.8/dist-packages (from requests->huggingface_hub) (2022.12.7)\n" ] } ] }, { "cell_type": "code", "source": [ "import torch\n", "from transformers import BertTokenizer, BertForSequenceClassification, TFDistilBertForSequenceClassification" ], "metadata": { "id": "XXn00BW7N79l" }, "execution_count": 62, "outputs": [] }, { "cell_type": "code", "source": [ "model2 = TFDistilBertForSequenceClassification.from_pretrained('/content/drive/MyDrive/MLDS-2/MODULO III/Talleres/Modelo Entrenado/')\n" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "nnajL6gxN7zN", "outputId": "a5477437-8a60-44f3-fa99-c83de5010cc6" }, "execution_count": 63, "outputs": [ { "output_type": "stream", "name": "stderr", "text": [ "Some layers from the model checkpoint at /content/drive/MyDrive/MLDS-2/MODULO III/Talleres/Modelo Entrenado/ were not used when initializing TFDistilBertForSequenceClassification: ['dropout_39']\n", "- This IS expected if you are initializing TFDistilBertForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", "- This IS NOT expected if you are initializing TFDistilBertForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", "Some layers of TFDistilBertForSequenceClassification were not initialized from the model checkpoint at /content/drive/MyDrive/MLDS-2/MODULO III/Talleres/Modelo Entrenado/ and are newly initialized: ['dropout_59']\n", "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n" ] } ] }, { "cell_type": "code", "source": [ "from huggingface_hub import notebook_login\n", "\n", "notebook_login()\n" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 359, "referenced_widgets": [ "eae418e551f44d90b53b67ab19f2681a", "d339a1c63e24406faf2c231028ab0d7f", "e4665b549c0f48698529d99f475e07ca", "92b55d980482446da8d2aed8b58593ff", "2dfe4e10862342918c0c151f3f719fef", "5e73bccd9598457b856dd44741261ed3", "cdc2b5d89c814b6ca8c2e7a30f17b1e2", "d76091dbf8334024b4edf2ec9e5bf32d", "f43c0faafc754458bce32f3eee55a153", "ff5b269f82684bc9ab81c3caae7299ca", "902f6a0765a0469ebd1994f789ad80e2", "c14909df983c43aeb3f62b670c376b60", "05e9de72aac143918ec0e31263075982", "b92abd39e834458d95ed024468835ff4", "46133dced4ec4122bbca398b70f4aadb", "a026cfb4f04a4e5f95fc6d6dcf53b9e1", "40af31601c0b4396bdf2da2d81ba1f1b" ] }, "id": "smWmyyktyYKr", "outputId": "7ae0dee8-eafb-4134-962e-f3ed0187f2c9" }, "execution_count": 65, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "Token is valid.\n", "Your token has been saved in your configured git credential helpers (store).\n", "Your token has been saved to /root/.huggingface/token\n", "Login successful\n" ] } ] }, { "cell_type": "code", "source": [ "model2.push_to_hub(\"Dfbenavidesr/distilbert-base-uncased-finetuned_clf-spam\")" ], "metadata": { "id": "1HTnPWjUTDCt" }, "execution_count": 66, "outputs": [] }, { "cell_type": "code", "source": [ "model2 = TFDistilBertForSequenceClassification.from_pretrained(\"Dfbenavidesr/distilbert-base-uncased-finetuned_clf-spam\")" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "YNboOPrSTwe1", "outputId": "7382c934-1760-41d8-dd80-3513fd37168c" }, "execution_count": 70, "outputs": [ { "output_type": "stream", "name": "stderr", "text": [ "Some layers from the model checkpoint at Dfbenavidesr/distilbert-base-uncased-finetuned_clf-spam were not used when initializing TFDistilBertForSequenceClassification: ['dropout_59']\n", "- This IS expected if you are initializing TFDistilBertForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n", "- This IS NOT expected if you are initializing TFDistilBertForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n", "Some layers of TFDistilBertForSequenceClassification were not initialized from the model checkpoint at Dfbenavidesr/distilbert-base-uncased-finetuned_clf-spam and are newly initialized: ['dropout_99']\n", "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n" ] } ] }, { "cell_type": "code", "source": [ "\n" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "5THCF3RsgirM", "outputId": "ddbc0b75-c437-4d97-c623-a62d421d865f" }, "execution_count": 71, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "" ] }, "metadata": {}, "execution_count": 71 } ] } ], "metadata": { "accelerator": "GPU", "colab": { "machine_shape": "hm", "provenance": [] }, "kernelspec": { "display_name": "Python 3", "name": "python3" }, "language_info": { "name": "python" }, "widgets": { "application/vnd.jupyter.widget-state+json": { "eae418e551f44d90b53b67ab19f2681a": { "model_module": "@jupyter-widgets/controls", "model_name": "VBoxModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "VBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "VBoxView", "box_style": "", "children": [ "IPY_MODEL_d339a1c63e24406faf2c231028ab0d7f", "IPY_MODEL_e4665b549c0f48698529d99f475e07ca", "IPY_MODEL_92b55d980482446da8d2aed8b58593ff", "IPY_MODEL_2dfe4e10862342918c0c151f3f719fef", "IPY_MODEL_5e73bccd9598457b856dd44741261ed3" ], "layout": "IPY_MODEL_cdc2b5d89c814b6ca8c2e7a30f17b1e2" } }, "d339a1c63e24406faf2c231028ab0d7f": { "model_module": "@jupyter-widgets/controls", "model_name": "HTMLModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_d76091dbf8334024b4edf2ec9e5bf32d", "placeholder": "​", "style": "IPY_MODEL_f43c0faafc754458bce32f3eee55a153", "value": "

Copy a token from your Hugging Face\ntokens page and paste it below.
Immediately click login after copying\nyour token or it might be stored in plain text in this notebook file.
" } }, "e4665b549c0f48698529d99f475e07ca": { "model_module": "@jupyter-widgets/controls", "model_name": "PasswordModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "PasswordModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "PasswordView", "continuous_update": true, "description": "Token:", "description_tooltip": null, "disabled": false, "layout": "IPY_MODEL_ff5b269f82684bc9ab81c3caae7299ca", "placeholder": "​", "style": "IPY_MODEL_902f6a0765a0469ebd1994f789ad80e2", "value": "" } }, "92b55d980482446da8d2aed8b58593ff": { "model_module": "@jupyter-widgets/controls", "model_name": "CheckboxModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "CheckboxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "CheckboxView", "description": "Add token as git credential?", "description_tooltip": null, "disabled": false, "indent": true, "layout": "IPY_MODEL_c14909df983c43aeb3f62b670c376b60", "style": "IPY_MODEL_05e9de72aac143918ec0e31263075982", "value": true } }, "2dfe4e10862342918c0c151f3f719fef": { "model_module": "@jupyter-widgets/controls", "model_name": "ButtonModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ButtonModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ButtonView", "button_style": "", "description": "Login", "disabled": false, "icon": "", "layout": "IPY_MODEL_b92abd39e834458d95ed024468835ff4", "style": "IPY_MODEL_46133dced4ec4122bbca398b70f4aadb", "tooltip": "" } }, "5e73bccd9598457b856dd44741261ed3": { "model_module": "@jupyter-widgets/controls", "model_name": "HTMLModel", "model_module_version": "1.5.0", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_a026cfb4f04a4e5f95fc6d6dcf53b9e1", "placeholder": "​", "style": "IPY_MODEL_40af31601c0b4396bdf2da2d81ba1f1b", "value": "\nPro Tip: If you don't already have one, you can create a dedicated\n'notebooks' token with 'write' access, that you can then easily reuse for all\nnotebooks. " } }, "cdc2b5d89c814b6ca8c2e7a30f17b1e2": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": "center", "align_self": null, "border": null, "bottom": null, "display": "flex", "flex": null, "flex_flow": "column", "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": "50%" } }, "d76091dbf8334024b4edf2ec9e5bf32d": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "f43c0faafc754458bce32f3eee55a153": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "ff5b269f82684bc9ab81c3caae7299ca": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "902f6a0765a0469ebd1994f789ad80e2": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "c14909df983c43aeb3f62b670c376b60": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "05e9de72aac143918ec0e31263075982": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "b92abd39e834458d95ed024468835ff4": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "46133dced4ec4122bbca398b70f4aadb": { "model_module": "@jupyter-widgets/controls", "model_name": "ButtonStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ButtonStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "button_color": null, "font_weight": "" } }, "a026cfb4f04a4e5f95fc6d6dcf53b9e1": { "model_module": "@jupyter-widgets/base", "model_name": "LayoutModel", "model_module_version": "1.2.0", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "40af31601c0b4396bdf2da2d81ba1f1b": { "model_module": "@jupyter-widgets/controls", "model_name": "DescriptionStyleModel", "model_module_version": "1.5.0", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } } } } }, "nbformat": 4, "nbformat_minor": 0 }