Identifying Objects Using Your Browser using TensorFlowJS
Tags: machinelearning, tensorflowjs, pwa, javascript
Published: 2018-06-12 08:35:59

---
title: Identifying Objects Using Your Browser using TensorFlowJS
published: true
description: The Google Brain team has released TensorFlowJS - an accessible JavaScript Library that now brings machine learning to everyone. So as a developer, what does this mean? For us, it means we get to play around with some new technology.
tags: Machine Learning, TensorFlowJS, PWA, JavaScript
---
You might be familiar with the TV show [Silicon Valley](https://en.wikipedia.org/wiki/Silicon_Valley_(TV_series)) and the [“Hot Dog” episode](https://www.youtube.com/watch?v=ACmydtFDTGs) where the cast created an app to simply (and yet hilariously) determine the object as a hot dog or not a hot dog. And it’s not science fiction anymore with applications, like Google Lens, rolled out to most modern smartphones. These days anyone can simply point their camera and get the information they need, quickly.

![Silicon Valley: Hot Dog episode](https://thepracticaldev.s3.amazonaws.com/i/g474ilocbdst7mofjyv0.gif)

There are services like [Google Cloud Vision API](https://cloud.google.com/vision/), [AWS Rekognition](https://aws.amazon.com/rekognition/) and [Clarifai](https://clarifai.com/) - to name a few - that are available in the market for anyone to implement and use. Though these services let you do to more with less code, it does come with a pay-as-you-go price tag. Also, it's a generic image identifier and may have a different use case.

Enter: **_TensorFlowJS_**.

![TensorFlow JS](https://thepracticaldev.s3.amazonaws.com/i/eq6fpdrg0jqthcufidur.jpg)

It's a JavaScript library released by the Google Brain Team that brings machine learning for everyone. It was originally written in Python, C++ and CUDA. Thanks to the team, they have ported it to JavaScript, where it is commonly used in the browser. Though TensorFlowJS is not exactly the same as his big brother Python version, the library is already equipped with the necessary APIs to build and train models from scratch, run TensorFlow models and retrain pre-existing models, all the convenience of your browser.

TensorFlow has been circulating around my reading feeds and with Google’s recent IO event, it inspired and pushed me to get my hands dirty in machine learning.

## Road To Discovery

![Road to discovery](https://thepracticaldev.s3.amazonaws.com/i/3ycvbc8x6j8w9c0h237l.jpg)

Warnings from different sources suggest that, TensorFlow wouldn’t be helpful if you didn’t have any machine learning background. Python keeps popping up as the language of choice for developing in machine learning and it seemed that I needed to learn the basics in order to create to proceed. This was one of many hurdles that I encountered. But I was still determined to create my own image identifier.

### Warning: If you don’t have any Machine Learning background, TensorFlow is not for you

The first step is always going back to basics and read the documentations at [TensorFlowJS’s website](https://js.tensorflow.org/). It seemed to be pretty straightforward at first, but I was wrong. More questions surfaced and I was beginning to believe the warning signs earlier. Maybe I do need to learn about machine learning before I dive into TensorFlowJS. Even scouring YouTube for tutorials and references didn’t help much. I did manage to “create” an image classifier locally in my machine but it was running in Python. I needed it to be client-side just like the [Emoji Scavenger Hunt](https://emojiscavengerhunt.withgoogle.com/).

Having found the [repository](https://github.com/google/emoji-scavenger-hunt) to the Emoji Scavenger Hunt and hours of reverse engineering the codes to suit to my needs, I was able to finally create my own image classifier that works smoothly in the client-side.

### Teach it as if it was like a 2-year-old

I thought that my biggest hurdle would be developing in Python. I was developing in a Windows environment initially and it was confusing and a pain to set up. But the moment I switched to a Mac environment, everything was smooth sailing. The biggest lesson that I learned was providing the system with good data. My colleagues told me in order to have results with high accuracy, you must provide good initial data.

![Teach it as if it was like a 2-year-old](https://thepracticaldev.s3.amazonaws.com/i/gtg68zr81gchr1912dvw.jpg)

An analogy to simply understand how machine learning works is, teaching it by showing images as if it was like a 2-year-old, where data is the set of images and the 2-year-old is the machine learning system. For example, if you want the child to learn what an apple is, you would show the child different pictures of apples only. Nothing else should be in the picture. No other fruits, no other elements. After a certain number of pictures that the child has seen, they will be able to know when they see an apple in real-life. On the other hand, if you give the child pictures with apples and oranges, apples and bananas, apples and grapes. The child will get confused when they see those fruits together.

The moral of the analogy is that the images that will be initially fed to the system for machine learning should be easy to comprehend for someone or something who doesn’t know what the subject is.

## Riddle Me This PWA

![Riddle me this](https://thepracticaldev.s3.amazonaws.com/i/mzs2802qaetkpnsmtbkk.jpg)

> _Riddle me this riddle me that<br/>Random riddles taken out from the hat<br/>When you know the answer take a snap<br/>If you are correct I will give you a clap_

The goal was to come up with my own image identifier and put it into some good use. The Riddle Me This is a PWA that will show you random riddles of common items that can be found around your home. Your challenge is to find the answer and take a picture of it. If you are correct, you proceed with the other riddles. If you are wrong, well...keep guessing.

Have a go at the link below! Happy hunting!

[![Nat's Eye](https://thepracticaldev.s3.amazonaws.com/i/ihzedef1w5h7cxxangnc.png) https://goo.gl/oaVLDu](https://goo.gl/oaVLDu)