take_at_home_jr5349.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Hpi-lEQsxEyJ"
      },
      "source": [
        "# Midterm - Spring 2023\n",
        "\n",
        "## Problem 1: Take-at-home  (45 points total)\n",
        "\n",
        "You are applying for a position at the data science team of USDA and you are given data associated with determining appropriate parasite treatment of canines. The suggested treatment options are determined based on a **logistic regression** model that predicts if the canine is infected with a parasite. \n",
        "\n",
        "The data is given in the site: https://data.world/ehales/grls-parasite-study/workspace/file?filename=CBC_data.csv  and more specifically in the CBC_data.csv file. Login using you University Google account to access the data and the description that includes a paper on the study (**you dont need to read the paper to solve this problem**). Your target variable $y$ column is titled `parasite_status`. \n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vZYY_LWRxEyN"
      },
      "source": [
        "### Question 1 - Feature Engineering (5 points)\n",
        "\n",
        "Write the posterior probability expressions for logistic regression for the problem you are given to solve."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_Pzq9mohxEyO"
      },
      "source": [
        "$$p(y=1| \\mathbf{x}, \\mathbf w)=\\frac{1}{1+e^{-w^{T}x}}=\\frac{e^{w^{T}x}}{1+e^{w^{T}x}}$$ \n",
        "\n",
        "$$p(y=0| \\mathbf{x}, \\mathbf w)=1-\\frac{1}{1+e^{-w^{T}x}}=1-\\frac{e^{w^{T}x}}{1+e^{w^{T}x}}$$ "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nwdYXqk9xEyP"
      },
      "source": [
        "\n",
        "\n",
        "### Question 2 - Decision Boundary (5 points)\n",
        "\n",
        "Write the expression for the decision boundary assuming that $p(y=1)=p(y=0)$. The decision boundary is the line that separates the two classes.\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VuOrNLFTxEyQ"
      },
      "source": [
        "$$w_{0}+w_{1}x_{1}+w_{2}x_{2}+...+w_{d}x_{d}=0$$ Where $d$ is the number of features."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QSL1VoIkxEyR"
      },
      "source": [
        "\n",
        "\n",
        "### Question 3 - Loss function (5 points)\n",
        "\n",
        "Write the expression of the loss as a function of $\\mathbf w$ that makes sense for you to use in this problem. \n",
        "\n",
        "NOTE: The loss will be a function that will include this function: \n",
        "\n",
        "$$\\sigma(a) = \\frac{1}{1+e^{-a}}$$\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Zd-TyoPGxEyR"
      },
      "source": [
        "$$L_{CE} = \\ln(\\prod_{i=1}^{N}\\sigma(w^{T}x)^{y_i}(1-\\sigma(w^{T}x))^{1-y_i})=\\sum_{i=1}^{N}\\ln(\\sigma(w^{T}x)^{y_i}(1-\\sigma(w^{T}x))^{1-y_i})$$ I'm using the log function for mathmatical convenience when taking the gradiet in the following step. Since the log function is always increasing, a function and the logarithm of that function will have the same optimal values for $w$."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BIvuYZfgxEyS"
      },
      "source": [
        "\n",
        "### Question 4 - Gradient (5 points)\n",
        "\n",
        "Write the expression of the gradient of the loss with respect to the parameters - show all your work.\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0ORR1odKxEyT"
      },
      "source": [
        "$$ \\nabla_\\mathbf w L_{CE} = \\nabla_{w} \\sum_{i=1}^{N}\\ln(\\sigma(w^{T}x)^{y_i}(1-\\sigma(w^{T}x))^{1-y_i})=\\sum_{i=1}^{N}y_i\\ln(\\sigma(w^{T}x)+\\sum_{i=1}^{N}(1-y_i)\\ln(1-\\sigma(w^{T}x))$$\n",
        "Since the derivative of the sigmoid function is as follows:\n",
        "$$\\frac{d\\sigma(z)}{dz}=\\frac{1}{1+e^{-z}}(1-\\frac{1}{1+e^{-z}})= \\sigma(z)(1-\\sigma(z))$$\n",
        "Then the gradient of the loss with respect to $w_j$ are as follows:\n",
        "$$\\sum_{i=1}^{N}(y^{(i)}\\cdot1-y^{(i)}\\cdot\\sigma(w^{T}x^{(i)})-1\\cdot\\sigma(w^{T}x^{(i)})+y^{(i)}\\cdot\\sigma(w^{T}x^{(i)}))x_{j}^{(i)}=\\sum_{i=1}^{N}(y^{(i)}-\\sigma(w^{T}x^{(i)}))x_{j}^{(i)}$$\n",
        "The gradient of the loss with respect to $w_j$ is:\n",
        "$$\\sum_{i=1}^{N}(y^{(i)}-\\sigma(w^{T}x^{(i)}))x_{j}^{(i)}$$"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KQ4QJXrTxEyT"
      },
      "source": [
        "### Question 5 - Imbalanced dataset (10 points)\n",
        "\n",
        "You are now told that in the dataset  \n",
        "\n",
        "$$p(y=0) >> p(y=1)$$\n",
        "\n",
        "Can you comment if the accuracy of Logistic Regression will be affected by such imbalance?\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sZQzEx5txEyU"
      },
      "source": [
        "If a dataset used to train a logistic regression model is imbalanced, it can cause the model to pickup and expect patterns that are true in the training data, but may not be accurate of the true relationship between features. For example, if a test set contains samples that are 98% negative in the 'parasite_status' column, then a model that classifies every sample as negative reagrdless of the other features will perform with 98% accuracy, which is misleadingly good accuracy for such a poor model. It can also lead to the model classifying more future examples as false negatives, given more of the training data is saw was negative. Depending on the use case in the real world, Type I or Type II error may be favored, but in any case, a model should not have to favor one over the other simply because the data is was trained on was imbalanced. The true effects of this would be evident when the model is tested on validation sets and new test data after it is trained."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zRtEkkx5xEyU"
      },
      "source": [
        "\n",
        "### Question 6 - SGD (15 points)\n",
        "\n",
        "The interviewer was impressed with your answers and wants to test your programming skills. \n",
        "\n",
        "1. Use the dataset to train a logistic regressor that will predict the target variable $y$. \n",
        "\n",
        " 2. Report the harmonic mean of precision (p) and recall (r) i.e the  [metric called $F_1$ score](https://en.wikipedia.org/wiki/F-score) that is calculated as shown below using a test dataset that is 20% of each group. Plot the $F_1$ score vs the iteration number  $t$. \n",
        "\n",
        "$$F_1 = \\frac{2}{r^{-1} + p^{-1}}$$\n",
        "\n",
        "Your code includes hyperparameter optimization of the learning rate and mini batch size. Please learn about cross validation which is a splitting strategy for tuning models [here](https://scikit-learn.org/stable/modules/cross_validation.html).\n",
        "\n",
        "You are allowed to use any library you want to code this problem.\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 575
        },
        "id": "kPur3Q4CxEyV",
        "outputId": "1b9c4dc9-179b-4d13-83c5-79a38b47ef58"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Based on ID, SEX, TYPEAREA, REPRO.STATE, & AGE, PARASITE_STATUS was correctly predicted 92.3841% of the time, with an F1 score of 0.0\n",
            "Based on ID, SEX, TYPEAREA, REPRO.STATE, & AGE, PARASITE_STATUS was correctly predicted 92.7152% of the time, with an F1 score of 0.0\n",
            "Based on ID, SEX, TYPEAREA, REPRO.STATE, & AGE, PARASITE_STATUS was correctly predicted 93.2119% of the time, with an F1 score of 0.0\n",
            "Based on ID, SEX, TYPEAREA, REPRO.STATE, & AGE, PARASITE_STATUS was correctly predicted 94.0397% of the time, with an F1 score of 0.0\n",
            "Based on ID, SEX, TYPEAREA, REPRO.STATE, & AGE, PARASITE_STATUS was correctly predicted 91.8874% of the time, with an F1 score of 0.0\n",
            "Based on ID, SEX, TYPEAREA, REPRO.STATE, & AGE, PARASITE_STATUS was correctly predicted 93.0464% of the time, with an F1 score of 0.0\n",
            "Based on ID, SEX, TYPEAREA, REPRO.STATE, & AGE, PARASITE_STATUS was correctly predicted 92.053% of the time, with an F1 score of 0.0\n",
            "Based on ID, SEX, TYPEAREA, REPRO.STATE, & AGE, PARASITE_STATUS was correctly predicted 92.053% of the time, with an F1 score of 0.0\n",
            "Based on ID, SEX, TYPEAREA, REPRO.STATE, & AGE, PARASITE_STATUS was correctly predicted 94.8675% of the time, with an F1 score of 0.0\n",
            "Based on ID, SEX, TYPEAREA, REPRO.STATE, & AGE, PARASITE_STATUS was correctly predicted 93.0464% of the time, with an F1 score of 0.0\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.9/dist-packages/sklearn/linear_model/_logistic.py:458: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
            "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
            "\n",
            "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
            "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
            "Please also refer to the documentation for alternative solver options:\n",
            "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
            "  n_iter_i = _check_optimize_result(\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZAAAAEGCAYAAABLgMOSAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAAsTAAALEwEAmpwYAAASFElEQVR4nO3dfZBddX3H8fenRKmPPEaKBAzWVBq1PswVfGrLKCLWh9CWGaEPZiw1tRW1WkdjnRGlTgdaq9VqnaaiUmVEB6um2oqIUq1aZBMRiZQmRZEgSJQURUcR/PaPeyI362az/LJ3z97u+zVz557f7/zuOd89M7ufPefcc06qCkmS7q6f67sASdJkMkAkSU0MEElSEwNEktTEAJEkNVnWdwEL6dBDD62VK1f2XYYkTZRNmzZ9u6qWT+9fUgGycuVKpqam+i5DkiZKkutm6vcQliSpiQEiSWpigEiSmhggkqQmBogkqYkBIklqYoBIkpoYIJKkJgaIJKmJASJJamKASJKaGCCSpCYGiCSpiQEiSWpigEiSmhggkqQmBogkqYkBIklqYoBIkpoYIJKkJgaIJKmJASJJamKASJKaGCCSpCYGiCSpSa8BkuSkJNck2ZZk/Qzz90/y/m7+ZUlWTpt/VJLbkrx8wYqWJAE9BkiS/YC3AU8HVgOnJVk9bdjpwM6qegjwJuCcafPfCPzbuGuVJP2sPvdAjgW2VdW1VXU7cAGwZtqYNcB53fSFwFOSBCDJycDXgC0LU64kaVSfAXIEcP1Ie3vXN+OYqroDuBU4JMl9gVcCr9vbSpKsSzKVZGrHjh3zUrgkaXJPor8WeFNV3ba3gVW1oaoGVTVYvnz5+CuTpCViWY/rvgE4cqS9ouubacz2JMuAA4DvAMcBpyT5K+BA4CdJflhVbx171ZIkoN8AuRxYleRohkFxKvA708ZsBNYCXwBOAT5VVQX86q4BSV4L3GZ4SNLC6i1AquqOJGcAFwH7Ae+sqi1JzgKmqmojcC7wniTbgFsYhowkaRHI8B/6pWEwGNTU1FTfZUjSREmyqaoG0/sn9SS6JKlnBogkqYkBIklqYoBIkpoYIJKkJgaIJKmJASJJamKASJKaGCCSpCYGiCSpiQEiSWpigEiSmhggkqQmBogkqYkBIklqYoBIkpoYIJKkJgaIJKmJASJJamKASJKaGCCSpCYGiCSpiQEiSWpigEiSmhggkqQmBogkqYkBIklqYoBIkpoYIJKkJgaIJKlJrwGS5KQk1yTZlmT9DPP3T/L+bv5lSVZ2/U9NsinJV7r3Jy948ZK0xPUWIEn2A94GPB1YDZyWZPW0YacDO6vqIcCbgHO6/m8Dz6qqRwBrgfcsTNWSpF363AM5FthWVddW1e3ABcCaaWPWAOd10xcCT0mSqvpSVX2z698C3CvJ/gtStSQJ6DdAjgCuH2lv7/pmHFNVdwC3AodMG/PbwOaq+tGY6pQkzWBZ3wXsiyQPY3hY68RZxqwD1gEcddRRC1SZJP3/1+ceyA3AkSPtFV3fjGOSLAMOAL7TtVcAHwKeW1X/s6eVVNWGqhpU1WD58uXzWL4kLW19BsjlwKokRye5J3AqsHHamI0MT5IDnAJ8qqoqyYHAx4D1VfW5hSpYknSX3gKkO6dxBnARcDXwgarakuSsJM/uhp0LHJJkG/AyYNdXfc8AHgK8JskV3esBC/wjSNKSlqrqu4YFMxgMampqqu8yJGmiJNlUVYPp/V6JLklqYoBIkpoYIJKkJgaIJKmJASJJamKASJKaGCCSpCYGiCSpiQEiSWpigEiSmhggkqQmBogkqYkBIklqYoBIkprMKUCSPCnJ87rp5UmOHm9ZkqTFbq8BkuRM4JXAq7quewDvHWdRkqTFby57IL8JPBv4PkBVfRO43ziLkiQtfnMJkNtr+NjCAkhyn/GWJEmaBHMJkA8k+QfgwCTPBz4J/ON4y5IkLXbLZpuZJMD7gWOA7wIPBV5TVRcvQG2SpEVs1gCpqkryr1X1CMDQkCT91FwOYW1O8tixVyJJmiiz7oF0jgN+N8l1DL+JFYY7J78y1sokSYvaXALkaWOvQpI0cfZ6CKuqrgMOBJ7VvQ7s+iRJS9hcrkR/CXA+8IDu9d4kLxp3YZKkxW0uh7BOB46rqu8DJDkH+ALwd+MsTJK0uM3lW1gB7hxp39n1SZKWsLnsgbwLuCzJh7r2ycC5Y6tIkjQR9hogVfXGJJcCT+q6nldVXxprVZKkRW+vAZLkccCWqtrcte+f5Liqumzs1UmSFq25nAN5O3DbSPu2rm+fJTkpyTVJtiVZP8P8/ZO8v5t/WZKVI/Ne1fVfk8RrVSRpgc3pJHp3O3cAquonzO3cyewLTfYD3gY8HVgNnJZk9bRhpwM7q+ohwJuAc7rPrgZOBR4GnAT8fbc8SdICmUsQXJvkxdy11/EnwLXzsO5jgW1VdS1AkguANcBXR8asAV7bTV8IvLW7Q/Aa4IKq+hHwtSTbuuV9YR7q+hmv+5ctfPWb3x3HoiVp7FY/8P6c+ayHzfty57IH8gLgCcAN3es4YN08rPsI4PqR9vaub8YxVXUHcCtwyBw/C0CSdUmmkkzt2LFjHsqWJMHcvoV1M8PDRROpqjYAGwAGg0HtZfiMxpHckjTp9rgHkuT5SVZ100nyziS3JrkyyWPmYd03AEeOtFd0fTOOSbIMOAD4zhw/K0kao9kOYb0E+Ho3fRrwSODBwMuAN8/Dui8HViU5Osk9Ge7lbJw2ZiOwtps+BfhUd0J/I3Bq9y2to4FVwBfnoSZJ0hzNFiB3VNWPu+lnAv9UVd+pqk8C99nXFXfnNM4ALgKuBj5QVVuSnJXk2d2wc4FDupPkLwPWd5/dAnyA4Qn3jwMvrKo7p69DkjQ+GfmG7u4zks3AM4CdwHXAk7s/3CS5uqp+ecGqnCeDwaCmpqb6LkOSJkqSTVU1mN4/20n01wBTwH7AxpHw+HXm52u8kqQJtscAqaqPJnkQcL+q2jkyawp4ztgrkyQtarN+jbc7T7FzWt/3x1qRJGkizOVCQkmSfoYBIklq0hQgSY6Z70IkSZOldQ/kE/NahSRp4uzxJHqSt+xpFnDgWKqRJE2M2b6F9Tzgz4AfzTDvtPGUI0maFLMFyOXAVVX1+ekzkrx2bBVJkibCbAFyCvDDmWZU1dHjKUeSNClmO4l+36r6wYJVIkmaKLMFyId3TST54PhLkSRNktkCJCPTDx53IZKkyTJbgNQepiVJmvUk+iOTfJfhnsi9umm6dlXV/cdenSRp0Zrtdu77LWQhkqTJ4s0UJUlNDBBJUhMDRJLUxACRJDUxQCRJTQwQSVITA0SS1MQAkSQ1MUAkSU0MEElSEwNEktTEAJEkNTFAJElNegmQJAcnuTjJ1u79oD2MW9uN2Zpkbdd37yQfS/JfSbYkOXthq5ckQX97IOuBS6pqFXBJ195NkoOBM4HjgGOBM0eC5g1VdQzwaOCJSZ6+MGVLknbpK0DWAOd10+cBJ88w5mnAxVV1S1XtBC4GTqqqH1TVpwGq6nZgM7Bi/CVLkkb1FSCHVdWN3fRNwGEzjDkCuH6kvb3r+6kkBwLPYrgXI0laQLM90nafJPkk8AszzHr1aKOqKsndfuZ6kmXA+4C3VNW1s4xbB6wDOOqoo+7uaiRJezC2AKmqE/Y0L8m3khxeVTcmORy4eYZhNwDHj7RXAJeOtDcAW6vqb/dSx4ZuLIPB4G4HlSRpZn0dwtoIrO2m1wIfmWHMRcCJSQ7qTp6f2PWR5PXAAcCfjr9USdJM+gqQs4GnJtkKnNC1STJI8g6AqroF+Avg8u51VlXdkmQFw8Ngq4HNSa5I8od9/BCStJSlaukc1RkMBjU1NdV3GZI0UZJsqqrB9H6vRJckNTFAJElNDBBJUhMDRJLUxACRJDUxQCRJTQwQSVITA0SS1MQAkSQ1MUAkSU0MEElSEwNEktTEAJEkNTFAJElNDBBJUhMDRJLUxACRJDUxQCRJTQwQSVITA0SS1MQAkSQ1MUAkSU0MEElSEwNEktTEAJEkNTFAJElNDBBJUhMDRJLUxACRJDUxQCRJTQwQSVKTXgIkycFJLk6ytXs/aA/j1nZjtiZZO8P8jUmuGn/FkqTp+toDWQ9cUlWrgEu69m6SHAycCRwHHAucORo0SX4LuG1hypUkTddXgKwBzuumzwNOnmHM04CLq+qWqtoJXAycBJDkvsDLgNePv1RJ0kz6CpDDqurGbvom4LAZxhwBXD/S3t71AfwF8DfAD/a2oiTrkkwlmdqxY8c+lCxJGrVsXAtO8kngF2aY9erRRlVVkroby30U8ItV9dIkK/c2vqo2ABsABoPBnNcjSZrd2AKkqk7Y07wk30pyeFXdmORw4OYZht0AHD/SXgFcCjweGCT5OsP6H5Dk0qo6HknSgunrENZGYNe3qtYCH5lhzEXAiUkO6k6enwhcVFVvr6oHVtVK4EnAfxsekrTw+gqQs4GnJtkKnNC1STJI8g6AqrqF4bmOy7vXWV2fJGkRSNXSOS0wGAxqamqq7zIkaaIk2VRVg+n9XokuSWpigEiSmhggkqQmBogkqYkBIklqYoBIkpoYIJKkJgaIJKmJASJJamKASJKaGCCSpCYGiCSpiQEiSWpigEiSmhggkqQmBogkqYkBIklqYoBIkpoYIJKkJgaIJKmJASJJamKASJKaGCCSpCYGiCSpSaqq7xoWTJIdwHV917GPDgW+3XcRi4TbYnduj925Pe6yr9viQVW1fHrnkgqQ/w+STFXVoO86FgO3xe7cHrtze9xlXNvCQ1iSpCYGiCSpiQEyeTb0XcAi4rbYndtjd26Pu4xlW3gORJLUxD0QSVITA0SS1MQAmQBJjkzy6SRfTbIlyUv6rmkxSLJfki8l+WjftfQtyYFJLkzyX0muTvL4vmvqS5KXdr8nVyV5X5Kf77umhZTknUluTnLVSN/BSS5OsrV7P2g+1mWATIY7gD+rqtXA44AXJlndc02LwUuAq/suYpF4M/DxqjoGeCRLdLskOQJ4MTCoqocD+wGn9lvVgns3cNK0vvXAJVW1Crika+8zA2QCVNWNVbW5m/4ewz8OR/RbVb+SrACeAbyj71r6luQA4NeAcwGq6vaq+t9ei+rXMuBeSZYB9wa+2XM9C6qqPgPcMq17DXBeN30ecPJ8rMsAmTBJVgKPBi7ruZS+/S3wCuAnPdexGBwN7ADe1R3Se0eS+/RdVB+q6gbgDcA3gBuBW6vqE/1WtSgcVlU3dtM3AYfNx0INkAmS5L7AB4E/rarv9l1PX5I8E7i5qjb1XcsisQx4DPD2qno08H3m6RDFpOmO7a9hGKoPBO6T5Pf6rWpxqeG1G/Ny/YYBMiGS3INheJxfVf/cdz09eyLw7CRfBy4Anpzkvf2W1KvtwPaq2rVXeiHDQFmKTgC+VlU7qurHwD8DT+i5psXgW0kOB+jeb56PhRogEyBJGB7fvrqq3th3PX2rqldV1YqqWsnwBOmnqmrJ/pdZVTcB1yd5aNf1FOCrPZbUp28Aj0ty7+735iks0S8UTLMRWNtNrwU+Mh8LNUAmwxOB32f4n/YV3es3+i5Ki8qLgPOTXAk8CvjLfsvpR7cXdiGwGfgKw79xS+qWJkneB3wBeGiS7UlOB84GnppkK8O9tLPnZV3eykSS1MI9EElSEwNEktTEAJEkNTFAJElNDBBJUhMDRGqQ5LbufWWS35nnZf/5tPbn53P50nwxQKR9sxK4WwHS3eRvNrsFSFV5JbUWJQNE2jdnA7/aXdz50u4ZJX+d5PIkVyb5I4Akxyf5bJKNdFeJJ/lwkk3dsyvWdX1nM7yT7BVJzu/6du3tpFv2VUm+kuQ5I8u+dOR5IOd3V2FLY7W3/4QkzW498PKqeiZAFwS3VtVjk+wPfC7JrrvBPgZ4eFV9rWv/QVXdkuRewOVJPlhV65OcUVWPmmFdv8XwKvNHAod2n/lMN+/RwMMY3rr8cwzvXvAf8/3DSqPcA5Hm14nAc5NcwfCW+4cAq7p5XxwJD4AXJ/ky8J/AkSPj9uRJwPuq6s6q+hbw78BjR5a9vap+AlzB8NCaNFbugUjzK8CLquqi3TqT4xneZn20fQLw+Kr6QZJLgX159OqPRqbvxN9tLQD3QKR98z3gfiPti4A/7m6/T5Jf2sPDnQ4AdnbhcQzDRxXv8uNdn5/ms8BzuvMsyxk+hfCL8/JTSA38L0XaN1cCd3aHot7N8NnkK4HN3YnsHcz8+NCPAy9IcjVwDcPDWLtsAK5Msrmqfnek/0PA44EvM3wg0Cuq6qYugKQF5914JUlNPIQlSWpigEiSmhggkqQmBogkqYkBIklqYoBIkpoYIJKkJv8H7O99GVpZGGUAAAAASUVORK5CYII=\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ],
      "source": [
        "import numpy as np\n",
        "from sklearn import preprocessing\n",
        "import pandas as pd\n",
        "from sklearn.model_selection import train_test_split\n",
        "from sklearn.linear_model import LogisticRegression\n",
        "from sklearn.metrics import f1_score\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "df = pd.read_csv('CBC_data.csv')\n",
        "\n",
        "# preprocessing via one hot encoding\n",
        "df_obj = df.select_dtypes(include=[object])\n",
        "pre = preprocessing.LabelEncoder()\n",
        "df_obj = df_obj.apply(pre.fit_transform)\n",
        "df['ID'] = df_obj['ID']\n",
        "df['SEX']= df_obj['SEX']\n",
        "df['TYPEAREA']= df_obj['TYPEAREA']\n",
        "df['SEX.REPRO']= df_obj['SEX.REPRO']\n",
        "df['REPRO.STATUS']= df_obj['REPRO.STATUS']\n",
        "df['PARASITE_STATUS']= df_obj['PARASITE_STATUS']\n",
        "\n",
        "\n",
        "# features used: ID, SEX, TYPEAREA, REPRO.STATE, AGE\n",
        "X = df.iloc[:, [0,1,2,4,5]]\n",
        "# target variable\n",
        "y = df['PARASITE_STATUS']\n",
        "\n",
        "# ITERATIONS \n",
        "t = 10\n",
        "x = [1,2,3,4,5,6,7,8,9,10]\n",
        "f1 = []\n",
        "\n",
        "for i in range(t):\n",
        "  # trains on 80% of data, tests of remaining 20%\n",
        "  x_train, x_test, y_train, y_test = train_test_split(X, y, test_size=0.2)\n",
        "  lr = LogisticRegression()\n",
        "  lr.fit(x_train, y_train)\n",
        "  predictions = lr.predict(x_test)\n",
        "  score = lr.score(x_test, y_test)\n",
        "  f1.append(f1_score(y_test.tolist(), predictions))\n",
        "  print('Based on ID, SEX, TYPEAREA, REPRO.STATE, & AGE, PARASITE_STATUS was correctly predicted {}% of the time, with an F1 score of {}'.format(round(score*100, 4), f1_score(y_test.tolist(), predictions)))\n",
        "\n",
        "\n",
        "plt.xlabel('Iteration')\n",
        "plt.ylabel('F1 Score')\n",
        "plt.plot(x,f1)\n",
        "plt.show()\n",
        "\n"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.9.6"
    },
    "orig_nbformat": 4,
    "vscode": {
      "interpreter": {
        "hash": "31f2aee4e71d21fbe5cf8b01ff0e069b9275f58929596ceb00d14d90e3e16cd6"
      }
    },
    "colab": {
      "provenance": []
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}