아래의 영어를 한글로 번역해주세요.

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 0 - Introduction\n",
    "\n",
    "Welcome to the <span style=\"color:yellowgreen\">Foundations of Machine Learning</span> (ECE5984_41) course!\n",
    "\n",
    "This is the <span style=\"color:red\">first</span> lab practice for this class. You will implement linear regression algorithm and apply it to the provided dataset.\n",
    "\n",
    "### Outline\n",
    "- [ 1 - Packages ](#1-packages)\n",
    "- [ 2 - Problem Statement and Dataset ](#2-problem-statement-and-dataset)\n",
    "- [ 3 - Linear Regression ](#3-linear-regression)\n",
    "  - [ 3.1 Normal Equation ](#31-normal-equation) - Exercise 1\n",
    "  - [ 3.2 Gradient Descent ](#32-gradient-descent) - Exercise 2, Exercise 3, Exercise 4"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Packages \n",
    "\n",
    "You have to install and use below packages for your HW#1.\n",
    "- [numpy](https://www.numpy.org): Fundamental package for matrix computation with python.\n",
    "- [matplotlib](https://matplotlib.org): Package for visualization of graph with python.\n",
    "- [pandas](https://pandas.pydata.org): Open source package for data analysis and manipulation.\n",
    "\n",
    "**Do not use other machine learning packages in this homework, e.g., *tensorflow*, *pytorch*, *jax*, etc.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "import math\n",
    "import copy\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import pandas as pd\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Problem Statement and Dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Problem Statement\n",
    "\n",
    "[Student Marks Dataset](https://www.kaggle.com/datasets/nikhil7280/student-performance-multiple-linear-regression)\n",
    "\n",
    "The dataset consists 100 samples of Marks(Grades) of students including their \"study time\" & \"number of courses\".\n",
    "\n",
    "In this homework, we would like to fit the linear regression model on this dataset. For practice, we will only use \"study time\" attribute."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Dataset\n",
    "\n",
    "You will start by loading the dataset. Make sure the attached 'Student_Marks.csv' file is in your current directory path (can be downloaded from icampus).\n",
    "\n",
    "Each column describes the below attributes:\n",
    "- \\[ time_study \\]: Total hour the student has studied.\n",
    "- \\[ Marks \\]: The final score the student has achieved.\n",
    "\n",
    "Run below cells and take a look at the dataset!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load the dataset\n",
    "dataset = pd.read_csv(\"./Student_Marks.csv\")\n",
    "\n",
    "# Display top 5 elements\n",
    "dataset.head(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# check data type of each attributes\n",
    "dataset.dtypes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# check dataset shape: (num data, num attributes)\n",
    "dataset.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Extract \"Hours Studied\" and \"Performance Index\" attributes. \n",
    "# Each attributes will be the data and label, respectively.\n",
    "data = dataset[\"time_study\"].to_numpy()\n",
    "label = dataset[\"Marks\"].to_numpy()\n",
    "\n",
    "# Check data statistics\n",
    "print(\"The shape of data is\", data.shape)\n",
    "print(\"The shape of label is\", label.shape)\n",
    "print()\n",
    "\n",
    "print(\"The first Five elements of data are\", data[:5])\n",
    "print(\"The first Five elements of label are\", label[:5])\n",
    "print()\n",
    "\n",
    "# Plot data\n",
    "plt.scatter(data, label, marker='x', c='r')\n",
    "plt.title(\"Study Time - Grade\")\n",
    "plt.xlabel(\"Study Time\")\n",
    "plt.ylabel(\"Grade\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Linear Regression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.1 Normal Equation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <span style=\"color:#ffd33d\"> Exercise 1 </span>: Complete the code for the normal equation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Complete the code for the normal equation\n",
    "def normal_equation(data, label):\n",
    "    \"\"\"\n",
    "    Directly compute optimal w* = (w, b) of the linear regression model\n",
    "    \n",
    "    Args:\n",
    "        data    (np.ndarray): Shape (N, ) Input ot the model\n",
    "        label   (np.ndarray): Shape (N, ) Label of the data\n",
    "    \n",
    "    Return:\n",
    "        w       (np.ndarray): weight(slope) of the linear regression model\n",
    "        b       (np.ndarray): bias of the linear regression model \n",
    "    \"\"\"\n",
    "    #[NOTE] write your from code here! \n",
    "\n",
    "    return w, b\n",
    "\n",
    "w, b = normal_equation(data, label)\n",
    "print(f\"w: {w:.6f}, b: {b:.6f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Plot your linear regression model and check if it works well"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Obtain weight and bias of your linear regression model\n",
    "w, b = normal_equation(data, label)\n",
    "\n",
    "# Plot data\n",
    "plt.scatter(data, label, marker='x', c='r')\n",
    "plt.title(\"Study Time - Grade\")\n",
    "plt.xlabel(\"Study Time\")\n",
    "plt.ylabel(\"Grade\")\n",
    "\n",
    "x_min, x_max = 0, math.ceil(max(data))\n",
    "y_min, y_max = 0, math.ceil(max(label))\n",
    "\n",
    "x = np.array([x_min, x_max])\n",
    "y = w * x + b\n",
    "plt.plot(x, y)\n",
    "\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.2 Gradient Descent"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <span style=\"color:#ffd33d\"> Exercise 2 </span>: Complete the code for computing the loss(cost)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\\> Here, the <span style=\"color:skyblue\">compute_loss()</span> method computes the loss of the linear regression model. \n",
    "\n",
    "Feel free to change or add variables, but do not modify the method name and its arguements."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "def compute_loss(data, label, w, b):\n",
    "    \"\"\"\n",
    "    Compute the loss(cost) of the linear regression model, given data and label.\n",
    "\n",
    "    Args:\n",
    "        data    (np.ndarray): Shape (N, ) Input to the model\n",
    "        label   (np.ndarray): Shape (N, ) Label of the data\n",
    "        w       (float): Weight of the linear regression model\n",
    "        b       (float): Bias of the linear regression model\n",
    "    \n",
    "    Return:\n",
    "        total_loss (float): Total loss of the linear regression model, given data and label\n",
    "    \"\"\"\n",
    "    total_loss = 0\n",
    "\n",
    "    #[NOTE] write your code here!\n",
    "\n",
    "    return total_loss"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <span style=\"color:#ffd33d\"> Exercise 4 </span>: Complete the code for computing the gradient of the loss w.r.t weights and bias"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\\> Here, the <span style=\"color:skyblue\">compute_gradient()</span> method computes the gradient of the loss with respect to the parameters of the linear regression model.\n",
    "\n",
    "Feel free to change or add variables, but do not modify the method name and its arguments."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "def compute_gradient(data, label, w, b):\n",
    "    \"\"\"\n",
    "    Compute the loss(cost) of the linear regression model, given data and label.\n",
    "\n",
    "    Args:\n",
    "        data    (np.ndarray): Shape (N, ) Input to the model\n",
    "        label   (np.ndarray): Shape (N, ) Label of the data\n",
    "        w       (float): Weight of the linear regression model\n",
    "        b       (float): Bias of the linear regression model\n",
    "    \n",
    "    Return:\n",
    "       grad_w   (float): The gradient of the loss w.r.t weight w\n",
    "       grad_b   (float): The gradient of the loss w.r.t bias b\n",
    "    \"\"\"\n",
    "    grad_w, grad_b = 0, 0\n",
    "\n",
    "    #[NOTE] write your code here!\n",
    "\n",
    "    return grad_w, grad_b"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <span style=\"color:#ffd33d\"> Exercise 4 </span>: Complete the code of gradient descent algorithm"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\\> <span style=\"color:skyblue\">gradient_descent()</span> method applies gradient descent on the given dataset. You should make 'loop' inside this method to iteratively update w and b. You __**must**__ use <span style=\"color:skyblue\">compute_loss()</span> and <span style=\"color:skyblue\">compute_gradient()</span> method to calculate loss and gradient, respectively.\n",
    "\n",
    "Also, for the purpose of visualization of the training curve, this method returns some 'history' lists. You can just simply append updated values to the corresponding 'history' list.\n",
    "\n",
    "Feel free to change or add variables, but do not modify the method name, its arguments, and return values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "def gradient_descent(data, label, w_init, b_init, iters=1500, lr=0.0001):\n",
    "    \"\"\"\n",
    "    Performs batch gradient descent to obatain weight and bias of the linear regression model.\n",
    "    \n",
    "    Args:\n",
    "      data    (np.ndarray): Shape (N,)\n",
    "      label   (np.ndarray): Shape (N,)\n",
    "      w_init  (float): Initial value of weight of the model\n",
    "      b_init  (float): Initial values of bias of the model\n",
    "      lr      (float): Learning rate\n",
    "      iters   (int): Number of iterations to run gradient descent\n",
    "    Returns\n",
    "      w             (float): Weight of the 1D linear regression model obtained with BGD\n",
    "      b             (float): Bias of the 1D linear regression model obtained with BGD\n",
    "      loss_history  (list): loss values of every iteration steps\n",
    "      w_history     (list): w values of every iteration steps\n",
    "      b_history     (list): b values of every iteration steps\n",
    "    \"\"\"\n",
    "    loss_history = []\n",
    "    w_history = []\n",
    "    b_history = []\n",
    "    w = w_init\n",
    "    b = b_init\n",
    "\n",
    "    loss = 0\n",
    "    for i in range(iters):\n",
    "        #[NOTE] write your code here!\n",
    "\n",
    "        # print loss for every 100 iterations\n",
    "        if i % 100 == 99:\n",
    "            print(f\"[ {i + 1:4}/{iters} ] Loss: {loss:.4f} | w: {w:.4f} | b: {b:.4f}\")\n",
    "    \n",
    "    return w, b, loss_history, w_history, b_history"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now apply your gradient descent algorithm. (Also try other hyperparameter values!)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# initial values of w and b\n",
    "w_init, b_init = .0, .0\n",
    "\n",
    "# hyperparameters for gradient descent algorithm\n",
    "iters = 1500\n",
    "learning_rate = 0.0001\n",
    "\n",
    "# obtain w and b with gradient descent\n",
    "w_gd, b_gd, loss_history, w_history, b_history = gradient_descent(\n",
    "                                                data, label, w_init, b_init,\n",
    "                                                iters=iters,\n",
    "                                                lr=learning_rate,\n",
    "                                            )\n",
    "\n",
    "print(\"w,b found by gradient descent:\", w_gd, b_gd)\n",
    "\n",
    "# obtain w and b with normal equation\n",
    "w_ne, b_ne = normal_equation(data, label)\n",
    "print(\"w,b found by normal equation:\", w_ne, b_ne)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "loss_history = loss_history[100:] # it is okay to remove this line\n",
    "\n",
    "# plot training curve\n",
    "plt.plot(range(len(loss_history)), loss_history)\n",
    "plt.title(\"Training Curve\")\n",
    "plt.xlabel(\"Iter\")\n",
    "plt.ylabel(\"Loss\")\n",
    "plt.show()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "fid",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}