updating prose

nbs/gradio.ipynb

@@ -409,6 +409,25 @@
     "![](hf_space_create.png)\n"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Set up your git repo\n",
+    "\n",
+    "After creating your space, you'll be offered up some instructions to configure the git repo managed by HF.  Here's the approach I found easiest:\n",
+    "\n",
+    "1. Locally go a `git clone https://huggingface.co/spaces/{your_username}/{your_space_name}`\n",
+    "\n",
+    "2. `cd` into your `{your_space_name}` directory and copy/move all your example(s), models, etc... into it\n",
+    "\n",
+    "3. Install `git lfs` on your system (on my Ubuntu 16.04 box it was as simple as `sudo apt-get install git-lfs`). See [the docs](https://git-lfs.github.com/) for more details.\n",
+    "\n",
+    "4. Configure `git lfs` for your repo by running `git lfs install` and then `git lfs track \"*.pkl\"` to ensure it is handling the BIG files (in this case our `export.pkl`)\n",
+    "\n",
+    "5. You may want to update your `.gitignore` to remove training data, etc... that *should not* be included in the repo.\n"
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},
@@ -486,13 +505,28 @@
    ]
   },
   {
-   "cell_type": "code",
-   "execution_count": null,
+   "cell_type": "markdown",
    "metadata": {},
-   "outputs": [],
    "source": [
-    "\n"
+    "### Commit and Push\n",
+    "\n",
+    "Simply ...\n",
+    "\n",
+    "```\n",
+    "git add .\n",
+    "git commit -am 'initial commit'\n",
+    "git push\n",
+    "```\n",
+    "\n",
+    "... all your code will be properly pushed to your HF Space's repo and your application will be spun up for you.  And voila, you now have a ***free*** web application you can use to show off your machine learning prowess just like this one!\n",
+    "\n",
+    "![](hf_space_app.png)"
    ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": []
   }
  ],
  "metadata": {

nbs/train.ipynb

@@ -4,24 +4,58 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "# Marvel Character"
+    "# Marvel Character Recognizer\n",
+    "\n",
+    "> Thwarting DC fan adverserial attacks one hero at a time."
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "https://www.kaggle.com/code/jhoward/is-it-a-bird-creating-a-model-from-your-own-data\n",
+    "The inspiration for this work dervies from the [Is it a bird? Creating a model from your own data](https://www.kaggle.com/code/jhoward/is-it-a-bird-creating-a-model-from-your-own-data) notebook presented by Jeremy Howard in session 1 of the 2022 [fast.ai](https://www.fast.ai/) course.  I've only made a few minor tweeks to demonstrate how folks can turn a classification task (Jeremy's notebook) into a regression task (this notebook)\n",
+    "\n",
+    "This notebook is structured to run seamlessly in kaggle, colab, and local environments.  The only install you'll need to make is `pip install fastai==2.6.0` to get things operational\n",
     "\n"
    ]
   },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# ! pip install fastai=2.6.0"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Why are we doing this?\n",
+    "\n",
+    "Good question!\n",
+    "\n",
+    "We already got a good Marvel character classifier folks can use [here](https://notebookse.jarvislabs.ai/jY5fsv-S9jKoQQrgd1dsoJuCDt6pTg6ZjBpNK9afxLIGInQv4OlHVuTMHqOPh2LU/).  It works great for classifying 10 different Marvel heroes!\n",
+    "\n",
+    "***BUT*** what does it do with non-Marvel characters?  Let's see:\n",
+    "\n",
+    "![batman.jpg]\n",
+    "\n",
+    "Yikes! That is defintely NOT Black Panther! And as it is every DC fan's dream to see their heroes mistaken as characters in the vastly superior Marvel universe, it's right for us to suspect they may use this model to convince those unaware that they indeed are.  This model, as it was trained to predict 1 of 10 classes, will always predict a class (even when it really shouldn't)\n",
+    "\n",
+    "**The solution**\n",
+    "\n",
+    "Turn this classification task into a regression task and train a model that will return the probability the uploaded character image is from Marvel!  And guess what, there are only like three changes we need to make to Jeremy's example to make this happen."
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": 1,
    "metadata": {},
    "outputs": [],
    "source": [
-    "import time\n",
+    "import os, time\n",
     "\n",
     "from fastai.vision.all import *\n",
     "from fastcore.all import *\n",
@@ -41,8 +75,18 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "data_path = Path(\"../data\")\n",
-    "model_path = Path(\"../models\")\n"
+    "is_kaggle = os.environ.get('KAGGLE_KERNEL_RUN_TYPE', '')\n",
+    "\n",
+    "if is_kaggle:\n",
+    "    data_path = Path(\"data\")\n",
+    "    model_path = Path(\"models\")\n",
+    "else:\n",
+    "    data_path = Path(\"../data\")\n",
+    "    model_path = Path(\"../models\")\n",
+    "\n",
+    "\n",
+    "data_path.mkdir(exist_ok=True, parents=True)\n",
+    "model_path"
    ]
   },
   {
@@ -220,12 +264,14 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Step 2: Build your `DataLoaders`"
+    "## Step 2: Build your `DataLoaders`\n",
+    "\n",
+    "**CHANGE #1**: Create a labeling function that labels Marvel images as 1.0 and DC images as 0.0"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 9,
+   "execution_count": 1,
    "metadata": {},
    "outputs": [],
    "source": [
@@ -233,6 +279,13 @@
     "    return 1.0 if img.parent.name.lower().startswith(\"marvel\") else 0.0\n"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "**CHANGE #2**: Swap out the `CategoryBlock` with a `RegressionBlock` and set `get_y` to use our custom labeling function above."
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": 10,
@@ -269,14 +322,34 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "TODO: Explain `DataBlock` above"
+    "### What is this `DataBlock` thing?\n",
+    "\n",
+    "A `DataBlock` is a blueprint for building `DataLoaders` (the things which will provide mini-batches-of-examples at a time to your model). It defines the entire process from getting your raw data to turning it into a numerical representation your model can utilize.  \n",
+    "\n",
+    "Breaking it down as Jeremy did we can understand it as such:\n",
+    "\n",
+    "`blocks` says that our inputs are images and that our targets are going to be a continuous number (a float between 0 and 1)\n",
+    "\n",
+    "`get_items` says how we are going to get our data, in this case images from our local filesystem\n",
+    "\n",
+    "`get_y` says how we want to label our targets, in this case we'll use our labeling function to assign them a float (1.0 or 0.0)\n",
+    "\n",
+    "`splitter` defines how we are going to create our validation set, in this case we'll use 20% of our dataset\n",
+    "\n",
+    "`item_tfms` define operations we want to apply on our data *when we fetch an item (e.g., and image)*, in this case we want to resize them to 192x192 pixels by \"squishing\" them.\n",
+    "\n",
+    "After defining our `DataBlock`, we can call the `dataloaders()` method passing in the path to our images to kick things off.  This will in turn pass that path to the `get_items` function of the `DataBlock` and when all is said and done, we'll have `DataLoaders` ready for training on."
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Step 3: Train with a `Learner`"
+    "## Step 3: Train with a `Learner`\n",
+    "\n",
+    "**CHANGE #3**: Swap out the `error_rate` metric for one better suited for regression tasks like `rmse`, and also assign `y_range` to constrain our model to predicting values in the expected range (which for us is 0 and 1).\n",
+    "\n",
+    "One of the cool things with `vision_learner` is that it will automatically change the loss function to be one suited for our task. In this case, it knows we're doing regression from the `DataLoaders` and assigns `MSELoss` as our loss function without us have to worry about a thing."
    ]
   },
   {
@@ -302,7 +375,9 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "TODO: Explain metrics, y_range and loss_func above"
+    "And now we train!\n",
+    "\n",
+    "We're not going to get the best results (at least not yet) because this task is a bit more complex that just training a bird or not bird.  We'll improve this over future iterations, but for now, I want to show we can get something pretty decent up and running quickly with stuff you learn in the first 30 minutes of the first session of fastai!"
    ]
   },
   {
@@ -443,7 +518,22 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "TODO: Explain `Learner.export`"
+    "**What is fine-tuning?**\n",
+    "\n",
+    "From Jeremy's notebook:\n",
+    "\n",
+    "> \"Fine-tuning\" a model means that we're starting with a model someone else has trained using some other dataset (called the *pretrained model*), and adjusting the weights a little bit so that the model learns to recognise your particular dataset. In this case, the pretrained model was trained to recognise photos in *imagenet*, and widely-used computer vision dataset with images covering 1000 categories) For details on fine-tuning and why it's important, check out the [free fast.ai course](https://course.fast.ai/)."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "**What can we do with a trained model?**\n",
+    "\n",
+    "We'll, we can export it and use it in a web application or elsewhere of course!  \n",
+    "\n",
+    "And with fast.ai, everything required to save your model alongside the information required to build future `DataLoaders` or do item level predictions, can be achieved simply by calling `Learner.export()`"
    ]
   },
   {
@@ -466,7 +556,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "TODO: Explain `load_learner`"
+    "To use an exported learner for inference, we can call fastai's `load_learner` and pass in the location of our exported file above like this:"
    ]
   },
   {
@@ -482,7 +572,9 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "TODO: Explain what predict returns and why we write a custom predict function"
+    "`Learner.predict` returns three things: prediction, predicted index, and probabilities.\n",
+    "\n",
+    "For a regression task, each item shows the same one thing we care about: The predicted number.  But for classification task you would see the predicted class label, the index of that class in the list of classes, and the probability of all the classes."
    ]
   },
   {
@@ -689,6 +781,17 @@
     "test_img.to_thumb(256, 256)\n"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Thanks for reading!\n",
+    "\n",
+    "If you made it this far and  you liked this notebook, I would most definitely appreciate an upvote! And if you have questions/suggestions/whatever, feel free to leave those in the comments section below. I hope this notebooks makes the world even just a tiny bit better by helping everyone ensure they are rooting on the right heroes!\n",
+    "\n",
+    "You can find me at twitter [@waydegilliam](https://twitter.com/waydegilliam) and the fast.ai forums [@wgpubs](https://forums.fast.ai/u/wgpubs/summary)"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": null,