Using a neural network to classify images of intertidal invertebrates
By: Noah Jaffe
Tags: Image Classification, Neural Networks, Data Science, Yolov3

The first step was to gather my training data, the images I would show my model. I needed a few hundred images for each of my classes, which were: anemones, barnacles, bivalves, crabs, nudibranchs, and sea stars. I chose these classes as they are frequent denizens of the tidepools of the West Coast of North America, where I live. To get my training images, I took advantage of a handy Google Chrome plugin called Download All Images which, as the name suggests, allows the user to download all images from a particular page — in my case a Google search of the animal group. After I had my images downloaded to my computer, I needed to label them. These labels are fed into the model along with the images so that the model knows what it is looking at during training. To label my images, I used LabelImg, a tool available on Github. I now had my images files and their corresponding labels on my computer, so the next stage was to hop on to Google Colab and connect to their GPU so I could start my training.

To train my model, I uploaded my files to Google drive and mounted my drive to my Colab notebook so I could reference it. I utilized the Darknet NN framework (see this repo), which is written in C and allows us to work with our model in a notebook. After constructing my necessary config, object and training text files, and moving them to all the right places (the tutorial lays this out very clearly), I was ready to begin training. As is common in convolutional neural network training, I chose a batch size of 64 and a maximum number of batches (iterations) of 12,000, for a total of approximately 700 epochs (here is a reference if you’re confused as to the difference). Then it was just a matter of training! Even with my trusty GPU in hand (or, I guess, in cloud), this process took awhile — approximately three days. One challenge to be aware of when utilizing Colab for a long period of time is that Google will sometimes kick you off and your processes will be interrupted. Fortunately, the tutorial has a couple of workarounds for this, most importantly code to save your model output every 100 iterations.

After my model was trained, I needed to test it! As with many tasks in Python, there are multiple ways to go about this, and the tutorial has a testing section. However, my final deliverable for this project was going to be a Streamlit app, and the code in the tutorial, which used bash commands, was harder for me to work with in my app. So instead, I tested my model using another useful module called OpenCV, roughly following this tutorial (Google “openCV image detection for more useful tutorials!) As described above, the output to my neural network model was a document of the model weights: connections from one discriminatory layer to the next. (These weights documents can be massive — mine was approximately 250 MB.) After defining some testing functions to test my model (see the “Testing_notebook.ipynb” notebook in the repo), I was ready to see if my model could classify images of my classes!

Leptasterias spp. Label above image = classification, number = confidence. Picture by author.

Success! I had made a working classifier. I am including only a single image here for copyright reasons (this is a picture I took during grad school), but check out some other examples of classifications in the “Project_5_final_pres.pdf” file in the repo.

Of course, my classifier did not work perfectly, and some classes were more difficult than others; crabs were especially difficult because they can be in many different positions, as opposed to my other classes which are much more sessile. All in all, though, this classifier represents a successful first step in constructing a tool to identify intertidal invertebrates, which was my goal. More training would no doubt improve the model, and I would also consider tinkering with the hyperparameters of my NN, a process I avoided for this project.