MidtermTakehome/index.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2IdTjyqZHKK5"
      },
      "source": [
        "# Midterm - Spring 2023\n",
        "\n",
        "## Problem 1: Take-at-home  (45 points total)\n",
        "\n",
        "You are applying for a position at the data science team of USDA and you are given data associated with determining appropriate parasite treatment of canines. The suggested treatment options are determined based on a **logistic regression** model that predicts if the canine is infected with a parasite. \n",
        "\n",
        "The data is given in the site: https://data.world/ehales/grls-parasite-study/workspace/file?filename=CBC_data.csv  and more specifically in the CBC_data.csv file. Login using you University Google account to access the data and the description that includes a paper on the study (**you dont need to read the paper to solve this problem**). Your target variable $y$ column is titled `parasite_status`. \n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fp6F7zIuHKK-"
      },
      "source": [
        "### Question 1 - Feature Engineering (5 points)\n",
        "\n",
        "In this step you outline the following as potential features (this is a limited example - we can have many features as in your programming exercise below). \n",
        "\n",
        "Write the posterior probability expressions for logistic regression for the problem you are given to solve."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZDRhz6bDHKLA"
      },
      "source": [
        "$$p(y=1| \\mathbf{x}, \\mathbf w)=\\frac{p(x | y=1)p(y=1)}{p(x | y=1)p(y=1)+p(x|y=0)p(y=0)} = \\frac{1}{1+e^{-\\alpha}}=\\sigma(\\alpha)$$ \n",
        "\n",
        "$$p(y=0| \\mathbf{x}, \\mathbf w)=1-\\frac{p(x | y=1)p(y=1)}{p(x | y=1)p(y=1)+p(x|y=0)p(y=0)} = 1-\\sigma(\\alpha)$$\n",
        "\n",
        "the equeation for alpha is shown below. $$\\alpha=ln\\frac{p(x|y=1)p(y=1)}{p(x|y=0)p(y=0)}$$"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zXYe2qtSHKLB"
      },
      "source": [
        "\n",
        "\n",
        "### Question 2 - Decision Boundary (5 points)\n",
        "\n",
        "Write the expression for the decision boundary assuming that $p(y=1)=p(y=0)$. The decision boundary is the line that separates the two classes.\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5sM9vvj3HKLC"
      },
      "source": [
        "The decision boundary should be the intersection of the probability mass functions, so as to minimize false positives and false negatives. this can be written via the following equality:\n",
        "\n",
        "$$p(y=1|x,w)=p(y=0|x,w)$$\n",
        "\n",
        "which can also be written as the following:\n",
        "\n",
        "$$\\sigma(\\alpha)=1-\\sigma(\\alpha)$$\n",
        "\n",
        "visually, the decision boundary should sit at the intersection of the sigmoid line and the inverse simga line, representing the conditions where there is an equal probability that y=1 and y=0."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uY7lH1geHKLD"
      },
      "source": [
        "\n",
        "\n",
        "### Question 3 - Loss function (5 points)\n",
        "\n",
        "Write the expression of the loss as a function of $\\mathbf w$ that makes sense for you to use in this problem. \n",
        "\n",
        "NOTE: The loss will be a function that will include this function: \n",
        "\n",
        "$$\\sigma(a) = \\frac{1}{1+e^{-a}}$$\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "or-qkG4MHKLE"
      },
      "source": [
        "$$L_{CE} = -\\Sigma^m_{i=1}[y_i\\ln(σ(w^Τx_i))+(1-y_i)\\ln(1-\\sigma(w^Τx_i)] $$"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qq1aXGcMHKLF"
      },
      "source": [
        "\n",
        "### Question 4 - Gradient (5 points)\n",
        "\n",
        "Write the expression of the gradient of the loss with respect to the parameters - show all your work.\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fZklUA8PHKLH"
      },
      "source": [
        "$$ \\nabla_\\mathbf w L_{CE} =\\Sigma^m_{i=1}[(σ(w^Τx_i)-y_i)x_i]$$"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WHj0C8T7HKLI"
      },
      "source": [
        "### Question 5 - Imbalanced dataset (10 points)\n",
        "\n",
        "You are now told that in the dataset  \n",
        "\n",
        "$$p(y=0) >> p(y=1)$$\n",
        "\n",
        "Can you comment if the accuracy of Logistic Regression will be affected by such imbalance?\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ft7xpMVIHKLK"
      },
      "source": [
        "If the p(y=0) is much larger than p(y=1), this will lead to the regression being much more likely to return ŷ=0. This will lead to a higher rate of false negatives, thereby effecting the accuracy of the Logisitic Regression. The model will have a low rate of false positives, however, as it will be much less likely to predict a positive in the first place. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iE0WczaxHKLL"
      },
      "source": [
        "\n",
        "### Question 6 - SGD (15 points)\n",
        "\n",
        "The interviewer was impressed with your answers and wants to test your programming skills. \n",
        "\n",
        "1. Use the dataset to train a logistic regressor that will predict the target variable $y$. \n",
        "\n",
        " 2. Report the harmonic mean of precision (p) and recall (r) i.e the  [metric called $F_1$ score](https://en.wikipedia.org/wiki/F-score) that is calculated as shown below using a test dataset that is 20% of each group. Plot the $F_1$ score vs the iteration number  $t$. \n",
        "\n",
        "$$F_1 = \\frac{2}{r^{-1} + p^{-1}}$$\n",
        "\n",
        "Your code includes hyperparameter optimization of the learning rate and mini batch size. Please learn about cross validation which is a splitting strategy for tuning models [here](https://scikit-learn.org/stable/modules/cross_validation.html).\n",
        "\n",
        "You are allowed to use any library you want to code this problem.\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 99,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 321
        },
        "id": "keQIm2V6HKLN",
        "outputId": "0a4ae008-a656-4b91-c40f-b80653b25364"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "w vector:  [[-1.84834009e-03 -4.54038579e-03  7.88831514e-06  1.83512257e-03\n",
            "   3.41803018e-03 -2.00747551e-03 -3.22278484e-03  1.61564091e-04]]\n"
          ]
        },
        {
          "data": {
            "text/plain": [
              "<matplotlib.collections.PathCollection at 0x7ff0072dc520>"
            ]
          },
          "execution_count": 99,
          "metadata": {},
          "output_type": "execute_result"
        },
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAAsTAAALEwEAmpwYAAAUkUlEQVR4nO3df2xd533f8fdnlOxwyWYlDltUlFOpsKtBgTNruVVSJMsyB46U/ohU123sdKizGXCLzViHNuqkFVgTD4WTeYhbYP5jRp3NddfagetowtJO8+oAHYLUNWWlUWVXjeK6sei0VmwrXVYttpTv/uCRQzEkdUle6lIP3y+A4D3Pee59vs89h59zcc7lvakqJEnt+lvDLkCStLwMeklqnEEvSY0z6CWpcQa9JDVuzbALmOmNb3xjbdy4cdhlSNJF5eDBg1+rqrHZ1q24oN+4cSMTExPDLkOSLipJ/mKudZ66kaTGGfSS1DiDXpIaZ9BLUuMMeklq3Ip7140kXez2HZrkzgNHee7kKdavG2X39s3s2jo+tHoMekkaoH2HJtn78GFOvXIGgMmTp9j78GGAoYW9QT8AK+3ore80zG3k/rG63Hng6Kshf9apV85w54GjBv3FaiUevXWuYW4j94/V57mTpxbUfiF4MXaJ5jt6a2UY5jZy/1h91q8bXVD7hWDQL9FKPHrrXMPcRu4fq8/u7ZsZXTtyTtvo2hF2b988pIoM+iVbiUdvnWuY28j9Y/XZtXWcO66/mvF1owQYXzfKHddfPdRTdQb9Eq3Eo7fONcxt5P6xOu3aOs7n9lzLn3/sh/ncnmuHfj3Gi7FLdHYD+q6KlWuY28j9QytBqmrYNZyj1+uVH1MsSQuT5GBV9WZb56kbSWqcQS9JjTPoJalxBr0kNa6voE+yI8nRJMeS7Jll/buSPJHkdJIbprVfk+TzSY4k+WKSDwyyeEnS+Z337ZVJRoC7geuA48DjSfZX1ZPTun0F+BDw4Rl3/xvgp6vqS0nWAweTHKiqk4MoXpLm4ofJfVs/76PfBhyrqqcBkjwA7AReDfqqeqZb963pd6yqP5t2+7kkzwNjwMmlFi5Jc/HD5M7Vz6mbceDZacvHu7YFSbINuAT48izrbk0ykWTixIkTC31oSTqHHyZ3rgtyMTbJ9wD3A/+0qr41c31V3VNVvarqjY2NXYiSJDXMD5M7Vz9BPwlcMW15Q9fWlyR/F/gM8EtV9YcLK0+SFs4PkztXP0H/OHBVkk1JLgFuBPb38+Bd/08Dv1FVDy2+TEnqnx8md67zBn1VnQZuAw4ATwGfqqojSW5P8n6AJD+Q5DjwE8B/SnKku/tPAu8CPpTkC93PNcsxEUk6ayV+VPAw+aFmktQAP9RMklYxg16SGmfQS1LjDHpJapxBL0mNM+glqXEGvSQ1zqCXpMYZ9JLUOINekhpn0EtS4wx6SWqcQS9JjTPoJalxBr0kNc6gl6TGGfSS1DiDXpIaZ9BLUuMMeklqnEEvSY0z6CWpcX0FfZIdSY4mOZZkzyzr35XkiSSnk9wwY93NSb7U/dw8qMKl5bbv0CTv+NijbNrzGd7xsUfZd2hy2CVJi7LmfB2SjAB3A9cBx4HHk+yvqiendfsK8CHgwzPu+wbgl4EeUMDB7r4vDaZ8aXnsOzTJ3ocPc+qVMwBMnjzF3ocPA7Br6/gwS5MWrJ9X9NuAY1X1dFW9DDwA7JzeoaqeqaovAt+acd/twCNV9WIX7o8AOwZQt7Ss7jxw9NWQP+vUK2e488DRIVUkLV4/QT8OPDtt+XjX1o++7pvk1iQTSSZOnDjR50NLy+e5k6cW1C6tZCviYmxV3VNVvarqjY2NDbscifXrRhfULq1k/QT9JHDFtOUNXVs/lnJfaWh2b9/M6NqRc9pG146we/vmIVUkLV4/Qf84cFWSTUkuAW4E9vf5+AeA9yZ5fZLXA+/t2qQVbdfWce64/mrG140SYHzdKHdcf7UXYnVROu+7bqrqdJLbmAroEeCTVXUkye3ARFXtT/IDwKeB1wM/muSjVfXmqnoxyb9j6mABcHtVvbhMc5EGatfWcYNdTUhVDbuGc/R6vZqYmBh2GZJ0UUlysKp6s61bERdjJUnLx6CXpMYZ9JLUOINekhpn0EtS4wx6SWqcQS9JjTPoJalxBr0kNc6gl6TGGfSS1DiDXpIaZ9BLUuMMeklqnEEvSY0z6CWpcQa9JDXOoJekxhn0ktQ4g16SGmfQS1Lj+gr6JDuSHE1yLMmeWdZfmuTBbv1jSTZ27WuT3JfkcJKnkuwdcP2SpPM4b9AnGQHuBt4HbAFuSrJlRrdbgJeq6krgLuDjXftPAJdW1dXAW4GfOXsQkCRdGP28ot8GHKuqp6vqZeABYOeMPjuB+7rbDwHvSRKggNcmWQOMAi8Dfz2QyiVJfekn6MeBZ6ctH+/aZu1TVaeBrwOXMxX6/xf4KvAV4D9U1YszB0hya5KJJBMnTpxY8CQkSXNb7oux24AzwHpgE/ALSb5vZqequqeqelXVGxsbW+aSJGl16SfoJ4Erpi1v6Npm7dOdprkMeAH4IPA/quqVqnoe+BzQW2rRkqT+9RP0jwNXJdmU5BLgRmD/jD77gZu72zcAj1ZVMXW65lqAJK8F3g786SAKlyT157xB351zvw04ADwFfKqqjiS5Pcn7u273ApcnOQb8PHD2LZh3A69LcoSpA8Z/rqovDnoSkqS5ZeqF98rR6/VqYmJi2GVI0kUlycGqmvXUuP8ZK0mNM+glqXEGvSQ1zqCXpMYZ9JLUOINekhpn0EtS4wx6SWqcQS9JjTPoJalxBr0kNc6gl6TGGfSS1DiDXpIat2bYBUjShbTv0CR3HjjKcydPsX7dKLu3b2bX1plfg90Wg17SqrHv0CR7Hz7MqVfOADB58hR7Hz4M0HTYe+pG0qpx54Gjr4b8WadeOcOdB44OqaILw6CXtGo8d/LUgtpbYdBLWjXWrxtdUHsrDHpJq8bu7ZsZXTtyTtvo2hF2b988pIouDC/GSlo1zl5w9V03ktSwXVvHmw/2mfo6dZNkR5KjSY4l2TPL+kuTPNitfyzJxmnr3pLk80mOJDmc5DUDrF+SdB7nDfokI8DdwPuALcBNSbbM6HYL8FJVXQncBXy8u+8a4DeBn62qNwPvBl4ZWPWSpPPq5xX9NuBYVT1dVS8DDwA7Z/TZCdzX3X4IeE+SAO8FvlhVfwxQVS9U1RkkSRdMP0E/Djw7bfl41zZrn6o6DXwduBz4fqCSHEjyRJJfnG2AJLcmmUgyceLEiYXOQZI0j+V+e+Ua4J3AT3W/fyzJe2Z2qqp7qqpXVb2xsbFlLkmSVpd+gn4SuGLa8oaubdY+3Xn5y4AXmHr1/wdV9bWq+hvgd4F/sNSiJUn96yfoHweuSrIpySXAjcD+GX32Azd3t28AHq2qAg4AVyf5290B4B8BTw6mdElSP877PvqqOp3kNqZCewT4ZFUdSXI7MFFV+4F7gfuTHANeZOpgQFW9lOQTTB0sCvjdqvrMMs1FkjSLTL3wXjl6vV5NTEwMuwxJuqgkOVhVvdnW+Vk3ktQ4g16SGmfQS1LjDHpJapxBL0mNM+glqXEGvSQ1zqCXpMYZ9JLUOINekhpn0EtS4wx6SWqcQS9JjTPoJalxBr0kNc6gl6TGGfSS1DiDXpIaZ9BLUuMMeklqnEEvSY3rK+iT7EhyNMmxJHtmWX9pkge79Y8l2Thj/ZuSfCPJhwdUtySpT+cN+iQjwN3A+4AtwE1JtszodgvwUlVdCdwFfHzG+k8Av7f0ciVJC9XPK/ptwLGqerqqXgYeAHbO6LMTuK+7/RDwniQBSLIL+HPgyEAqliQtSD9BPw48O235eNc2a5+qOg18Hbg8yeuAfw18dOmlSpIWY7kvxn4EuKuqvjFfpyS3JplIMnHixIllLkmSVpc1ffSZBK6Ytryha5utz/Eka4DLgBeAtwE3JPn3wDrgW0n+X1X9x+l3rqp7gHsAer1eLWIekqQ59BP0jwNXJdnEVKDfCHxwRp/9wM3A54EbgEerqoB/eLZDko8A35gZ8pKk5XXeoK+q00luAw4AI8Anq+pIktuBiaraD9wL3J/kGPAiUwcDSdIKkKkX3itHr9eriYmJYZchSReVJAerqjfbOv8zVpIaZ9BLUuMMeklqnEEvSY0z6CWpcQa9JDXOoJekxhn0ktQ4g16SGmfQS1LjDHpJapxBL0mNM+glqXEGvSQ1zqCXpMYZ9JLUOINekhpn0EtS4wx6SWqcQS9JjTPoJalxBr0kNa6voE+yI8nRJMeS7Jll/aVJHuzWP5ZkY9d+XZKDSQ53v68dcP2SpPM4b9AnGQHuBt4HbAFuSrJlRrdbgJeq6krgLuDjXfvXgB+tqquBm4H7B1W4JKk//byi3wYcq6qnq+pl4AFg54w+O4H7utsPAe9Jkqo6VFXPde1HgNEklw6icElSf/oJ+nHg2WnLx7u2WftU1Wng68DlM/r8OPBEVX1z5gBJbk0ykWTixIkT/dYuSerDBbkYm+TNTJ3O+ZnZ1lfVPVXVq6re2NjYhShJklaNfoJ+Erhi2vKGrm3WPknWAJcBL3TLG4BPAz9dVV9easGSpIXpJ+gfB65KsinJJcCNwP4ZffYzdbEV4Abg0aqqJOuAzwB7qupzA6pZkrQA5w367pz7bcAB4CngU1V1JMntSd7fdbsXuDzJMeDngbNvwbwNuBL4t0m+0P1818BnIUmaU6pq2DWco9fr1cTExLDLkKSLSpKDVdWbbZ3/GStJjTPoJalxBr0kNc6gl6TGGfSS1DiDXpIaZ9BLUuMMeklqnEEvSY0z6CWpcQa9JDXOoJekxhn0ktQ4g16SGmfQS1LjDHpJapxBL0mNM+glqXEGvSQ1zqCXpMYZ9JLUOINekhq3pp9OSXYAvwaMAL9eVR+bsf5S4DeAtwIvAB+oqme6dXuBW4AzwL+sqgMDq36afYcmufPAUZ47eYr160bZvX0zu7aOD6x9vjGWu6YLMb/Wx17MvjOo7TrM/bb17bqY53y5t8UgxxiUVNX8HZIR4M+A64DjwOPATVX15LQ+/xx4S1X9bJIbgR+rqg8k2QL8NrANWA/8L+D7q+rMXOP1er2amJhY0CT2HZpk78OHOfXKtx92dO0IP/7WcX7n4OSS2++4/mqAWce44/qrZ90gg6ppvrEHNb/Wx55rG8Hc22lQ23UxY19Mz+1KHHu+53wug/x7HdT2Xug8khysqt6s6/oI+h8EPlJV27vlvQBVdce0Pge6Pp9Psgb4S2AM2DO97/R+c423mKB/x8ceZfLkqe9oH0k4M8v8Fto+vm4UYNYxxteN8rk91y5bTfONPaj5tT72XNsI5t5Og9quixn7YnpuV+LY8z3ncxnk3+ugtvdC5zFf0Pdz6mYceHba8nHgbXP1qarTSb4OXN61/+GM+37HISrJrcCtAG9605v6KOlcz83y5AGzPnmLaZ/r8edbN6ia5ht7UPNrfexBbr+FbtfFjH0xPbcrcez5xpjLhfh7HeS+s1Ar4mJsVd1TVb2q6o2NjS34/uu7VwkzjSQDaV+/bnTOMRbaPsixBzW/1see63HmWzeo7bqYsS+m53Yljj3fcz6XYe5rg5zHXPoJ+kngimnLG7q2Wft0p24uY+qibD/3XbLd2zczunbknLbRtSPc9LYrBtK+e/vmOcfYvX3zstY039iDml/rY8+1jWDu7TSo7bqYsS+m53Yljj3fcz6XYe5rg5zHXPo5dfM4cFWSTUyF9I3AB2f02Q/cDHweuAF4tKoqyX7gt5J8gqmLsVcBfzSo4s86e8FitqvWve99w0Daz+r3yvgga5pv7EHNr/WxF7PvDGq7DnO/bX27LvQ5vxDbYpBjDMp5L8YCJPkh4FeZenvlJ6vqV5LcDkxU1f4krwHuB7YCLwI3VtXT3X1/CfhnwGngX1XV78031mIuxkrSarekd91caAa9JC3cfEG/Ii7GSpKWj0EvSY0z6CWpcQa9JDVuxV2MTXIC+IslPMQbga8NqJyLifNeXZz36tLPvL+3qmb9j9MVF/RLlWRirivPLXPeq4vzXl2WOm9P3UhS4wx6SWpci0F/z7ALGBLnvbo479VlSfNu7hy9JOlcLb6ilyRNY9BLUuOaCfokO5IcTXIsyZ5h17OcknwyyfNJ/mRa2xuSPJLkS93v1w+zxkFLckWSzyZ5MsmRJD/Xtbc+79ck+aMkf9zN+6Nd+6Ykj3X7+4NJLhl2rcshyUiSQ0n+e7e8Wub9TJLDSb6QZKJrW/S+3kTQd19gfjfwPmALcFP3xeSt+i/Ajhlte4Dfr6qrgN/vlltyGviFqtoCvB34F902bn3e3wSuraq/D1wD7EjyduDjwF1VdSXwEnDL8EpcVj8HPDVtebXMG+AfV9U1094/v+h9vYmgB7YBx6rq6ap6GXgA2DnkmpZNVf0BU5/7P91O4L7u9n3ArgtZ03Krqq9W1RPd7f/D1B//OO3Pu6rqG93i2u6ngGuBh7r25uYNkGQD8MPAr3fLYRXMex6L3tdbCfrZvsB8cF/PcnH47qr6anf7L4HvHmYxyynJRqa+5OYxVsG8u9MXXwCeBx4BvgycrKrTXZdW9/dfBX4R+Fa3fDmrY94wdTD/n0kOJrm1a1v0vt7PVwnqItN9jWOT75tN8jrgd5j6trK/zrQvVm513lV1BrgmyTrg08DfG25Fyy/JjwDPV9XBJO8ecjnD8M6qmkzyXcAjSf50+sqF7uutvKK/IF9CvsL9VZLvAeh+Pz/kegYuyVqmQv6/VtXDXXPz8z6rqk4CnwV+EFiX5OwLtRb393cA70/yDFOnYq8Ffo325w1AVU12v59n6uC+jSXs660E/atfYN5dhb+RqS8sX03OfkE73e//NsRaBq47P3sv8FRVfWLaqtbnPda9kifJKHAdU9cnPgvc0HVrbt5VtbeqNlTVRqb+nh+tqp+i8XkDJHltkr9z9jbwXuBPWMK+3sx/xs72BebDrWj5JPlt4N1MfXTpXwG/DOwDPgW8iamPef7Jqpp5wfaileSdwP8GDvPtc7b/hqnz9C3P+y1MXXgbYeqF2aeq6vYk38fUK903AIeAf1JV3xxepcunO3Xz4ar6kdUw726On+4W1wC/VVW/kuRyFrmvNxP0kqTZtXLqRpI0B4Nekhpn0EtS4wx6SWqcQS9JjTPoJalxBr0kNe7/A9ZCdgVOdEY7AAAAAElFTkSuQmCC",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light"
          },
          "output_type": "display_data"
        }
      ],
      "source": [
        "from numpy.core.fromnumeric import size\n",
        "# write your code here\n",
        "\n",
        "import pandas as pd\n",
        "import numpy as np\n",
        "import matplotlib.pyplot as plt\n",
        "from sklearn.linear_model import LogisticRegression\n",
        "from sklearn.model_selection import train_test_split\n",
        "from sklearn import datasets\n",
        "from sklearn import svm\n",
        "\n",
        "data=pd.read_csv(\"./CBC_data.csv\").dropna()\n",
        "y=data[[\"PARASITE_STATUS\"]].values\n",
        "X=data[[\"RBC\",\"HGB\",\"WBC\",\"EOS.CNT\",\"MONO.CNT\",\"NUT.CNT\",\"PL.CNT\",\"LYMP.CNT\"]]\n",
        "\n",
        "n_epochs=50\n",
        "eta=3*10**-7\n",
        "ones=[1,1,1,1,1,1,1,1]\n",
        "w=np.matrix(np.zeros(X.shape[1]))\n",
        "\n",
        "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=0)\n",
        "\n",
        "m=y_train.size\n",
        "\n",
        "def sig_fun(w, X):\n",
        "  output = 1.0/(1 + np.exp(-np.dot(w,X.T)))\n",
        "  #print(np.exp(-np.dot(w,X.T)),\"  -  \",w,X.T)\n",
        "  return   output\n",
        "\n",
        "\n",
        "def grad_fun(w,X,y):\n",
        "  #print(np.dot((sig_fun(w,X)-y),X))\n",
        "  return np.dot((sig_fun(w,X)-y),X)\n",
        "\n",
        "def pred_values(w, X):\n",
        "    probabilities = sig_fun(w, X)\n",
        "    pred_value = np.where(probabilities >= .5, 1, 0) \n",
        "    return np.squeeze(pred_value)\n",
        "\n",
        "def f1_score(X_test,y_test,w):\n",
        "  predictions = pred_values(w,X_test)\n",
        "  fp=0;tp=0;fn=0\n",
        "  for i in range(y_test.size):\n",
        "    yi=1 if (y_train[i])==(\"Positive\") else 0\n",
        "    #print(yi,predictions[i])\n",
        "    if(yi==1):\n",
        "      if(predictions[i]==1):\n",
        "        tp+=1\n",
        "      else:\n",
        "        fn+=1\n",
        "    else:\n",
        "      if(predictions[i]==1):\n",
        "        fp+=1   \n",
        "  #print(fp,fn,tp)\n",
        "  if(tp==0):\n",
        "    return 0\n",
        "  recall=tp/(tp+fn)\n",
        "  percision=tp/(tp+fp)\n",
        "  return 2/(percision**-1+recall**-1)\n",
        "\n",
        "\n",
        "iteration_number=[]\n",
        "F_score_v_iteration=[]\n",
        "\n",
        "\n",
        "for epoch in range(n_epochs):\n",
        "  for iteration in range(m):\n",
        "    random_index=np.random.randint(m)\n",
        "    xi=X_train[random_index:random_index+1].values\n",
        "    #if(type(np.dot(ones,xi.T)[0])!=type(1)):\n",
        "    #  continue\n",
        "    #print(xi)\n",
        "    yi=1 if (y_train[random_index:random_index+1][0])==(\"Positive\") else 0\n",
        "    gradients=grad_fun(w,xi,yi)\n",
        "    w=w-eta*gradients\n",
        "  iteration_number.append(epoch)\n",
        "  F_score_v_iteration.append(f1_score(X_test,y_test,w))\n",
        "    #print(w,gradients)\n",
        "    #print(yi,(y_train[random_index:random_index+1][0]))\n",
        "\n",
        "print(\"w vector: \",w)\n",
        "plt.scatter(iteration_number,F_score_v_iteration)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aOh-_F2u-Fvt"
      },
      "source": [
        "Most of the items in the data set are cases where the animal did not have a parasite. This means that the combined probability of an animal having a parasite, which was our positive case, is very low. Our model does not handle this very well, and tends to always predict no parasite. In this case, when the number of true positives is 0, we set the f1 score equal to zero to avoid a division by zero error during the runtime. This is shown on the graph above."
      ]
    }
  ],
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "display_name": "ai-course",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "name": "python",
      "version": "3.10.8"
    },
    "orig_nbformat": 4,
    "vscode": {
      "interpreter": {
        "hash": "62556f7a043365a66e0918c892755cfafede529a87e97207556f006a109bade4"
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}