A Beginner's Guide to Healthcare AI and TensorFlow.js
Tags: beginners, machinelearning, javascript, codenewbie
Published: 2020-05-05 00:31:40

#Intro

I am a semi-beginner in all things tech and coming up on actually finally completing my coding bootcamp course with Flatiron on full stack web development! Woo 🎉 The course covers standard software engineering, and not AI/machine learning/data science, but I've always been interested in it, and wanted to learn general programming before one day moving into something more directly AI related. But I could definitely be like this meme about wannabe AI enthusiasts - so this is a post for - cannot emphasize enough - beginners.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/zf6iwhde0l78ogb2c3ea.png)<figcaption>Image credit: [@spicy_coding_memes](https://www.instagram.com/p/B7a_r7TCwP0/?igshid=qcq4gbvg9jzq)</figcaption>


#COVID-19
Yes, this is a post about COVID-19. AI and coronavirus. Who needs another one of those? Not me! Send tweet. 

I promise I'll try to be original here. I've always been very interested in healthcare tech and AI - maybe because I love to watch *Grey's Anatomy* and I've always wanted the glory of medicine without having to go to medical school! But for real - and at the risk of sounding like *Silicon Valley*:
![Alt text](https://d2v7i6t2.map2.ssl.hwcdn.net/content/quotes/2371_500.jpg)
I really do want to use technology to make... the world... at least slightly less terrible than it is now. Whether or not that's possible is a question for another day!

Most coding bootcamps have a 'final project' that combines everything we've learned in the past 15 weeks, and mine is due at the end of this week, so I've been spending a lot of time looking at trending projects/hackathon submissions to gather inspiration and try to think of something that hadn't been totally done a million times before (this might also be impossible). As I said, I've been generally interested in something healthcare-related for this final project since before I even started, and now with COVID, I couldn't help it.

For inspiration, this [project](https://devpost.com/software/faco-fight-against-corona-jfcza9) in particular caught my attention, because of the use of audio recognition and coughs. There doesn't seem to be a lot of specific research on the use of cough recognition and coronavirus, but the research on uses in other medical conditions/cases seems very promising. 

There are many other very cool uses for AI/COVID - using [imaging](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7187882/), fever detection [glasses](https://techcrunch.com/2020/04/16/chinese-startup-rokid-pitches-covid-19-detection-glasses-in-u-s/), [cleaning robots](https://www.bbc.com/news/technology-52340651), and - potentially more controversial - 'smart' [doors](https://www.scmp.com/tech/article/3077964/ring-smart-doorbell-monitors-people-covid-19-isolation) that monitor people's activity and movement in quarantine. If all of these seem a little too sci-fi and impractical, there is also the more familiar example of AI chatbot technology, such as IBM [Watson](https://www.ibm.com/watson/covid-response), in helping to automate COVID diagnosis, questioning, and spread of information. While this is a beginner's post, it's no secret that AI in general has a ton of varying uses, some controversial and some, frankly, mundane, so I won't go into that too much more here. 

#TensorFlow

All of this got me very interested in a basic implementation of [TensorFlow](https://www.tensorflow.org/), Google's open-source machine-learning platform. My final bootcamp project is a COVID-19 self-report tracker and symptom checker, with a Rails backend and React.js frontend, so I looked into using TensorFlow.js, specifically for [audio](https://www.tensorflow.org/js/tutorials/transfer/audio_recognizer) recognition. TensorFlow and AI in general seem more common in Python, so I found this tutorial on implementing a simple audio recognizer in JavaScript especially exciting!! 

As I said, I was *inspired* by examples of using AI to diagnose COVID, but I am nowhere near being able to do that in any meaningful way, so I really just wanted to implement the most basic possible use of AI in my app to show that I could. I was able to "train" the TensorFlow model by coughing with my microphone on for a few seconds, and there you go, a sliding bar would move to the right if I was trying to record a cough, and to the left if I was trying to record anything other than a cough (i.e. regular talking). 

I really did nothing impressive here - all the cool stuff is handled by TensorFlow and this excellent tutorial! The simple audio recognizer uses `Transfer learning`, which "is a technique that shortcuts much of this by taking a piece of a model that has already been trained on a related task and reusing it in a new model" (straight from the old copy and paste here) but it's nice to know that so many of these seemingly complex AI technologies are actually just building on a ton of past work. So something relatively simple, like an image or audio recognizer, doesn't require as much re-training. This is why all I had to do to make my simple audio function the way I wanted was to "teach" it the sound of a cough. It already "knew" basic human speech pretty well, so then the distinction between that library and a new sound wasn't far off.

```javascript
const INPUT_SHAPE = [NUM_FRAMES, 232, 1];
let model;

async function train() {
 toggleButtons(false);
 const ys = tf.oneHot(examples.map(e => e.label), 3);
 const xsShape = [examples.length, ...INPUT_SHAPE];
 const xs = tf.tensor(flatten(examples.map(e => e.vals)), xsShape);

 await model.fit(xs, ys, {
   batchSize: 16,
   epochs: 10,
   callbacks: {
     onEpochEnd: (epoch, logs) => {
       document.querySelector('#console').textContent =
           `Accuracy: ${(logs.acc * 100).toFixed(1)}% Epoch: ${epoch + 1}`;
     }
   }
 });
 tf.dispose([xs, ys]);
 toggleButtons(true);
}
```

But I thought, this could even be used just for a simple validation on an online symptom submission so the user knows if they are coughing enough into the microphone, and/or prevents the user from submitting useless audio. While small, these are all things that can make the process of doctors remotely diagnosing and treating patients even just a little bit easier, which is very exciting to me!

#Conclusion

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/o5ewivr2gupw7pwxrlsw.png)<figcaption>Image credit: [@pure.python](https://www.instagram.com/p/B9D1r4HAZap/?igshid=1j7blknevdbaz)</figcaption>

This isn't a tutorial, and I don't have any unique insights on using TensorFlow (yet!) but there seem to be a ton of amazing tutorials and it really was easier than I expected to use TensorFlow in this simple way. If you, like me, are interested in AI/ML but haven't had time to do that Coursera course in full yet and are feeling overwhelmed, my advice is to just go for it and see what you can do!

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/s99x6k1aypkknr7ps4kx.png)<figcaption>Cover image credit: [@coderhumor](https://www.instagram.com/p/B86VG2BAIUU/?igshid=1wxurxx33v9u8)</figcaption>

#References
*https://devpost.com/software/faco-fight-against-corona-jfcza9
*https://devpost.com/software/corona-mask-detector
*https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7187882/
*https://venturebeat.com/2020/03/03/how-people-are-using-ai-to-detect-and-fight-the-coronavirus/
*https://www.scmp.com/tech/article/3077964/ring-smart-doorbell-monitors-people-covid-19-isolation
*https://www.bbc.com/news/technology-52340651
*https://techcrunch.com/2020/04/16/chinese-startup-rokid-pitches-covid-19-detection-glasses-in-u-s/