Flutter Web and Machine Learning
Tags: javascript, flutter, machinelearning, tensorflow
Published: 2020-04-25 10:37:19

*In case it helped :)*
<a href="https://www.buymeacoffee.com/aseemwangoo" target="_blank"><img src="https://www.buymeacoffee.com/assets/img/custom_images/orange_img.png" alt="Pass Me A Coffee!!" style="height: 41px !important;width: 174px !important;box-shadow: 0px 3px 2px 0px rgba(190, 190, 190, 0.5) !important;-webkit-box-shadow: 0px 3px 2px 0px rgba(190, 190, 190, 0.5) !important;" ></a>

<!-- wp:paragraph -->
<p>We will cover how to implement</p>
<!-- /wp:paragraph -->

<!-- wp:list {"ordered":true} -->
<ol><li><strong>Machine Learning using TensorFlow..</strong></li><li><strong>Feature Extraction from image…</strong></li></ol>
<!-- /wp:list -->

<!-- wp:heading {"level":4} -->
<h4>Pre-Requisite&nbsp;:</h4>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>This article uses the concept of calling JavaScript functions from Flutter Web, which is explained <a rel="noreferrer noopener" href="https://medium.com/flutterpub/flutter-web-and-location-2be96d3ad3e" target="_blank"><strong>in de</strong></a><strong><a href="https://flatteredwithflutter.com/detect-user-location-in-flutter-web/" target="_blank" rel="noreferrer noopener" aria-label="tail here (opens in a new tab)">tail here</a></strong><a rel="noreferrer noopener" href="https://medium.com/flutterpub/flutter-web-and-location-2be96d3ad3e" target="_blank"><strong>.</strong></a></p>
<!-- /wp:paragraph -->

<!-- wp:separator -->
<hr class="wp-block-separator"/>
<!-- /wp:separator -->

<!-- wp:heading {"level":4} -->
<h4>Machine Learning using TensorFlow in Flutter&nbsp;Web..</h4>
<!-- /wp:heading -->

####Article here: https://flatteredwithflutter.com/machine-learning-in-flutter-web/

<!-- wp:paragraph -->
<p>We will use <a rel="noreferrer noopener" href="https://www.tensorflow.org/js/tutorials" target="_blank"><strong>TensorFlow.js</strong></a>, which is a JavaScript Library for training and deploying machine learning models in the browser and in Node.js</p>
<!-- /wp:paragraph -->

{% youtube lYf5WwyzUHE %}

<!-- wp:paragraph -->
<p><strong>Setup&nbsp;:</strong></p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Using Script Tags</p>
<!-- /wp:paragraph -->

<!-- wp:code -->
<pre class="wp-block-code"><code>script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs@1.0.0/dist/tf.min.js">&lt;/script></code></pre>
<!-- /wp:code -->

<!-- wp:paragraph -->
<p>Add the above script tag inside the head section of your index.html file</p>
<!-- /wp:paragraph -->

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/2y2ya1u39yfq67rasoeb.png)

<!-- wp:paragraph -->
<p>That’s it…..</p>
<!-- /wp:paragraph -->

<!-- wp:separator -->
<hr class="wp-block-separator"/>
<!-- /wp:separator -->

<!-- wp:heading {"level":4} -->
<h4>Implementing a Model in Flutter&nbsp;Web…</h4>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>What will we do&nbsp;:</p>
<!-- /wp:paragraph -->

<!-- wp:list {"ordered":true} -->
<ol><li>Create a linear Model</li><li>Train the model</li><li>Enter a sample value to get the output…</li></ol>
<!-- /wp:list -->

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/4ahwtuwpsb276twdeslm.png)

<!-- wp:paragraph -->
<p><strong>Explanation&nbsp;:</strong></p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>We will create the above linear model. This model follows the formula</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p><strong>(2x — 1)</strong>. For instance,</p>
<!-- /wp:paragraph -->

<!-- wp:list {"ordered":true} -->
<ol><li>when x = -1, then y = -3</li><li>x = 0, y = -1 and so on…..</li></ol>
<!-- /wp:list -->

<!-- wp:paragraph -->
<p>We will give a sample input as 12, and predict the value from this model..</p>
<!-- /wp:paragraph -->

<!-- wp:heading {"level":4} -->
<h4>Create the&nbsp;model…</h4>
<!-- /wp:heading -->

<!-- wp:list {"ordered":true} -->
<ol><li>Create a js file (in our case ml.js)</li><li>Define a function (in our case learnLinear)</li></ol>
<!-- /wp:list -->

<!-- wp:preformatted -->
<pre class="wp-block-preformatted">async function learnLinear(input) {}</pre>
<!-- /wp:preformatted -->

<!-- wp:paragraph -->
<p>Initialize a sequential model, using <a href="https://js.tensorflow.org/api/0.6.1/#sequential" rel="noreferrer noopener" target="_blank"><strong>tf.sequential</strong></a><strong>.</strong></p>
<!-- /wp:paragraph -->

<!-- wp:preformatted -->
<pre class="wp-block-preformatted">const model = tf.sequential();</pre>
<!-- /wp:preformatted -->

<!-- wp:paragraph -->
<p>A sequential model is any model where the outputs of one layer are the inputs to the next layer.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Lets add our input layer to this model, using <a rel="noreferrer noopener" href="https://js.tensorflow.org/api/0.6.1/#layers.dense" target="_blank">tf.layers.dense</a>.</p>
<!-- /wp:paragraph -->

<!-- wp:separator -->
<hr class="wp-block-separator"/>
<!-- /wp:separator -->

<!-- wp:preformatted -->
<pre class="wp-block-preformatted">model.add(tf.layers.dense({ units: 1, inputShape: [1] }));</pre>
<!-- /wp:preformatted -->

<!-- wp:paragraph -->
<p>Parameters&nbsp;:</p>
<!-- /wp:paragraph -->

<!-- wp:list -->
<ul><li><strong>units (number)&nbsp;</strong>: Size of the output space. We will output just a single number</li><li><strong>inputShape</strong>&nbsp;: Defines the shape of input. We will provide the input as an array of length 1.</li></ul>
<!-- /wp:list -->

<!-- wp:paragraph -->
<p>Finally, we add this layer to our sequential model, using <a href="https://js.tensorflow.org/api/0.6.1/#tf.Sequential.add" rel="noreferrer noopener" target="_blank">model.add</a></p>
<!-- /wp:paragraph -->

<!-- wp:separator -->
<hr class="wp-block-separator"/>
<!-- /wp:separator -->

<!-- wp:paragraph -->
<p>Next, we need to compile the model,</p>
<!-- /wp:paragraph -->

<!-- wp:preformatted -->
<pre class="wp-block-preformatted">model.compile({<br>    loss: 'meanSquaredError',<br>    optimizer: 'sgd'<br>});</pre>
<!-- /wp:preformatted -->

<!-- wp:separator -->
<hr class="wp-block-separator"/>
<!-- /wp:separator -->

<!-- wp:paragraph -->
<p>We use <a href="https://js.tensorflow.org/api/0.6.1/#tf.Model.compile" rel="noreferrer noopener" target="_blank">model.compile</a> for compiling the model..</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Parameters&nbsp;:</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p><strong>loss</strong>&nbsp;: we seek to minimize the error. Cross-entropy and mean squared error are the two main <strong>types of loss</strong> functions to use when training neural network models.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>optimizer&nbsp;: string name for an Optimizer. In our case <strong>Stochastic Gradient Descent or sgd</strong></p>
<!-- /wp:paragraph -->

<!-- wp:separator -->
<hr class="wp-block-separator"/>
<!-- /wp:separator -->

<!-- wp:paragraph -->
<p>Next, we need to train the model,</p>
<!-- /wp:paragraph -->

<!-- wp:preformatted -->
<pre class="wp-block-preformatted">// INPUT -&gt; [6, 1] 6rows 1 columns<br>const xs = tf.tensor2d([-1, 0, 1, 2, 3, 4], [6, 1]);<br>const ys = tf.tensor2d([-3, -1, 1, 3, 5, 7], [6, 1]);</pre>
<!-- /wp:preformatted -->

<!-- wp:paragraph -->
<p>We define the input for x-axis using <a href="https://js.tensorflow.org/api/0.6.1/#tensor2d" rel="noreferrer noopener" target="_blank">tf.tensor2d,</a> <strong>called as xs</strong></p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Parameters&nbsp;:</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p><strong>values&nbsp;:</strong> The values of the tensor. Can be nested array of numbers, or a flat array. In our case [-1, 0, 1, 2, 3, 4]</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p><strong>shape&nbsp;: </strong>The shape of the tensor. If not provided, it is inferred from <code>values</code>. In our case, its an array of 6 rows and 1 column, hence [6, 1]</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Similarly, we define the output for y-axis using <a rel="noreferrer noopener" href="https://js.tensorflow.org/api/0.6.1/#tensor2d" target="_blank">tf.tensor2d,</a> <strong>called as ys</strong></p>
<!-- /wp:paragraph -->

<!-- wp:preformatted -->
<pre class="wp-block-preformatted">// TRAIN MODEL -&gt; EPOCHS (ITERATIONS)<br>await model.fit(xs, ys, { epochs: 250 });</pre>
<!-- /wp:preformatted -->

<!-- wp:separator -->
<hr class="wp-block-separator"/>
<!-- /wp:separator -->

<!-- wp:paragraph -->
<p>Now, we train the model using <a href="https://js.tensorflow.org/api/0.6.1/#tensor2d" rel="noreferrer noopener" target="_blank">model.fit</a></p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Parameters&nbsp;:</p>
<!-- /wp:paragraph -->

<!-- wp:list -->
<ul><li><strong>x</strong>&nbsp;: an input array of <a href="https://js.tensorflow.org/api/0.6.1/#class:Tensor" rel="noreferrer noopener" target="_blank">tf.Tensor</a>s, in our case xs</li><li><strong>y</strong>: an output array of <a href="https://js.tensorflow.org/api/0.6.1/#class:Tensor" rel="noreferrer noopener" target="_blank">tf.Tensor</a>s, in our case ys</li><li><strong>epochs</strong>: Times to iterate over the training data arrays.</li></ul>
<!-- /wp:list -->

<!-- wp:paragraph -->
<p>As we trained our model now, lets test it…..Time to predict values using <a rel="noreferrer noopener" href="https://js.tensorflow.org/api/0.6.1/#tf.Sequential.predict" target="_blank">model.predict</a></p>
<!-- /wp:paragraph -->

<!-- wp:preformatted -->
<pre class="wp-block-preformatted">// PREDICT THE VALUE NOW...
var predictions = model.predict(tf.tensor2d([input], [1, 1]));

let result = predictions.dataSync();
console.log('Res', result[0]); //number</pre>
<!-- /wp:preformatted -->

<!-- wp:separator -->
<hr class="wp-block-separator"/>
<!-- /wp:separator -->

<!-- wp:paragraph -->
<p>Parameters&nbsp;:</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p><strong>x: I</strong>nput data, as an <code>Array</code> of <a href="https://js.tensorflow.org/api/0.6.1/#class:Tensor" rel="noreferrer noopener" target="_blank">tf.Tensor</a>s, in our case this value is an array of 1 element, passed from dart.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>The result is stored in a <code>predictions</code> variable. In order to retrieve the data, we call</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p><a href="https://js.tensorflow.org/api/0.6.1/#tf.Tensor.dataSync" rel="noreferrer noopener" target="_blank">dataSync</a>&nbsp;: Synchronously downloads the values from the <a href="https://js.tensorflow.org/api/0.6.1/#class:Tensor" rel="noreferrer noopener" target="_blank">tf.Tensor</a> as an array..</p>
<!-- /wp:paragraph -->

<!-- wp:heading {"level":4} -->
<h4>Get Predicted Value in Flutter&nbsp;Web…</h4>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>In the above step, we created the TensorFlow model as a JS function which accepts a parameter..</p>
<!-- /wp:paragraph -->

<!-- wp:preformatted -->
<pre class="wp-block-preformatted">async function learnLinear(input) {}</pre>
<!-- /wp:preformatted -->

<!-- wp:list {"ordered":true} -->
<ol><li>Import the package</li></ol>
<!-- /wp:list -->

<!-- wp:preformatted -->
<pre class="wp-block-preformatted">import 'package:js/js_util.dart' as jsutil;</pre>
<!-- /wp:preformatted -->

<!-- wp:paragraph -->
<p>2. Create a dart file calling the JS function…</p>
<!-- /wp:paragraph -->

<!-- wp:preformatted -->
<pre class="wp-block-preformatted">@JS()
library main;

import 'package:js/js.dart';

@JS('learnLinear')
external num linearModel(int number);</pre>
<!-- /wp:preformatted -->

<!-- wp:quote -->
<blockquote class="wp-block-quote"><p>Note&nbsp;: <code><em>learnLinear</em></code> is the same JS function which we defined in the above section</p></blockquote>
<!-- /wp:quote -->

<!-- wp:paragraph -->
<p>3. As our function is an async function, we need to await the result from it..</p>
<!-- /wp:paragraph -->

<!-- wp:preformatted -->
<pre class="wp-block-preformatted">await jsutil.promiseToFuture&lt;num&gt;(linearModel(12))</pre>
<!-- /wp:preformatted -->

<!-- wp:paragraph -->
<p>We will make use of <a href="https://api.dart.dev/stable/2.7.1/dart-js_util/promiseToFuture.html" rel="noreferrer noopener" target="_blank"><strong>promiseToFuture</strong></a><strong>. </strong>What this does is</p>
<!-- /wp:paragraph -->

<!-- wp:quote -->
<blockquote class="wp-block-quote"><p>Converts a JavaScript Promise to a Dart <a href="https://api.dart.dev/stable/2.7.1/dart-async/Future-class.html" rel="noreferrer noopener" target="_blank">Future</a>.</p></blockquote>
<!-- /wp:quote -->

<!-- wp:separator -->
<hr class="wp-block-separator"/>
<!-- /wp:separator -->

<!-- wp:paragraph -->
<p>Lets call this function from a button now,</p>
<!-- /wp:paragraph -->

<!-- wp:preformatted -->
<pre class="wp-block-preformatted">OutlineButton(<br>  onPressed: () async {<br>      await jsutil.promiseToFuture&lt;num&gt;(linearModel(12));<br>  },<br>  child: const Text('Linear Model x=12'),<br>)</pre>
<!-- /wp:preformatted -->

<!-- wp:paragraph -->
<p>We have provided input value as <strong>12</strong>, and the output we get is :</p>
<!-- /wp:paragraph -->

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/r0qek1m52cxxsp65j5g5.png)

<!-- wp:heading {"level":4} -->
<h4>Feature Extraction From&nbsp;Image…</h4>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>For the feature extraction, we use an existing model <strong>called MobileNet.</strong></p>
<!-- /wp:paragraph -->

<!-- wp:quote -->
<blockquote class="wp-block-quote"><p>MobileNets are small, low-latency, low-power models parameterized to meet the resource constraints of a variety of use cases. They can be built upon for classification, detection similar to how other popular large scale models, are used.</p></blockquote>
<!-- /wp:quote -->

<!-- wp:paragraph -->
<p>It takes any browser-based image elements (<code>&lt;img&gt;</code>, <code>&lt;video&gt;</code>, <code>&lt;canvas&gt;</code>) as inputs, and returns an array of most likely predictions and their confidences.</p>
<!-- /wp:paragraph -->

<!-- wp:list {"ordered":true} -->
<ol><li><strong>Setup&nbsp;:</strong></li></ol>
<!-- /wp:list -->

<!-- wp:paragraph -->
<p>Using Script Tags</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>&lt;script src="https://cdn.jsdelivr.net/npm/@tensorflow-models/mobilenet@1.0.0"> &lt;/script></p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Add the above script tag inside the head section of your index.html file</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>2. <strong>Function in JS&nbsp;:&nbsp;</strong></p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>We will define an image tag inside our body html as</p>
<!-- /wp:paragraph -->

<!-- wp:preformatted -->
<pre class="wp-block-preformatted">&lt;img id="img" src="" hidden&gt;&lt;/img&gt;</pre>
<!-- /wp:preformatted -->

<!-- wp:paragraph -->
<p>Define a function in JS as&nbsp;:</p>
<!-- /wp:paragraph -->

<!-- wp:preformatted -->
<pre class="wp-block-preformatted">async function classifyImage() {}</pre>
<!-- /wp:preformatted -->

<!-- wp:paragraph -->
<p>Get the source of the image tag as</p>
<!-- /wp:paragraph -->

<!-- wp:preformatted -->
<pre class="wp-block-preformatted">const img = document.getElementById('img');</pre>
<!-- /wp:preformatted -->

<!-- wp:paragraph -->
<p>Load the mobilenet model and extract the features from the image selected as</p>
<!-- /wp:paragraph -->

<!-- wp:preformatted -->
<pre class="wp-block-preformatted">// LOAD MOBILENET MODEL
const model = await mobilenet.load();

// CLASSIFY THE IMAGE
let predictions = await model.classify(img);
console.log('Pred >>>', predictions);

return predictions</pre>
<!-- /wp:preformatted -->

<!-- wp:paragraph -->
<p>Predictions is an array which looks like this :</p>
<!-- /wp:paragraph -->

<!-- wp:preformatted -->
<pre class="wp-block-preformatted">[{<br>  className: "Egyptian cat",<br>  probability: 0.8380282521247864<br>}, {<br>  className: "tabby, tabby cat",<br>  probability: 0.04644153267145157<br>}, {<br>  className: "Siamese cat, Siamese",<br>  probability: 0.024488523602485657<br>}]</pre>
<!-- /wp:preformatted -->

<!-- wp:paragraph -->
<p>Finally, return these predictions.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>3. <strong>Function in dart :</strong></p>
<!-- /wp:paragraph -->

<!-- wp:preformatted -->
<pre class="wp-block-preformatted">@JS()
library main;

import 'package:js/js.dart';

@JS('learnLinear')
external num linearModel(int number);

@JS('classifyImage')
external List&lt;Object> imageClassifier();</pre>
<!-- /wp:preformatted -->

<!-- wp:quote -->
<blockquote class="wp-block-quote"><p>Note&nbsp;: This file was already created in the above section, we just added the last 2 lines…The name classifyImage is same as the JS function created in step 1</p></blockquote>
<!-- /wp:quote -->

<!-- wp:paragraph -->
<p><strong>4. Call the function from button</strong></p>
<!-- /wp:paragraph -->

<!-- wp:preformatted -->
<pre class="wp-block-preformatted">OutlineButton(<br>  onPressed: () async {<br>      await jsutil.promiseToFuture&lt;List&lt;Object&gt;&gt;(imageClassifier());<br>  },<br>  child: const Text('Feature Extraction'),<br>)</pre>
<!-- /wp:preformatted -->

<!-- wp:paragraph -->
<p>The return type of the <code>imageClassifier()</code> is a <code>List&lt;Object&gt;</code>&nbsp;. In order to extract the results, we need to convert this list into a custom Model class</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p><strong>5. Convert into Custom Model</strong></p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>We create a custom Class called <strong>ImageResults</strong> as</p>
<!-- /wp:paragraph -->

<!-- wp:preformatted -->
<pre class="wp-block-preformatted">@JS()
@anonymous
class ImageResults {
  external factory ImageResults({
    String className,
    num probability,
  });

external String get className;
external num get probability;

Map&lt;String, dynamic> toMap() {
    final _map = &lt;String, dynamic>{
      'className': className,
      'probability': probability,
    };
    return _map;
  }
}</pre>
<!-- /wp:preformatted -->

<!-- wp:paragraph -->
<p>First, we will convert each <strong>Object into a String</strong>, and then the string into <strong>ImageResults model…</strong></p>
<!-- /wp:paragraph -->

<!-- wp:preformatted -->
<pre class="wp-block-preformatted">List&lt;ImageResults> listOfImageResults(List&lt;Object> _val) {
  final _listOfMap = &lt;ImageResults>[];
  
  for (final item in _val) {
    final _jsString = stringify(item);
    _listOfMap.add(jsonObject(_jsString));
  }
  return _listOfMap;
}</pre>
<!-- /wp:preformatted -->

<!-- wp:paragraph -->
<p><strong>stringify</strong> is a function, defined as&nbsp;</p>
<!-- /wp:paragraph -->

<!-- wp:preformatted -->
<pre class="wp-block-preformatted">@JS('JSON.stringify')<br>external String stringify(Object obj);</pre>
<!-- /wp:preformatted -->

<!-- wp:paragraph -->
<p>this string is converted to <strong>ImageResults</strong> model using <strong>jsonObject..</strong></p>
<!-- /wp:paragraph -->

<!-- wp:preformatted -->
<pre class="wp-block-preformatted">@JS('JSON.parse')<br>external ImageResults jsonObject(String str);</pre>
<!-- /wp:preformatted -->

<!-- wp:separator -->
<hr class="wp-block-separator"/>
<!-- /wp:separator -->

<!-- wp:paragraph -->
<p>Now, you can easily access the values in dart as :</p>
<!-- /wp:paragraph -->

<!-- wp:preformatted -->
<pre class="wp-block-preformatted">for (final ImageResults _item in _listOfMap) ...[<br>   Text('ClassName : ${_item.className}'),<br>   Text('Probability : ${_item.probability}\n'),<br>]</pre>
<!-- /wp:preformatted -->

*In case it helped :)*
<a href="https://www.buymeacoffee.com/aseemwangoo" target="_blank"><img src="https://www.buymeacoffee.com/assets/img/custom_images/orange_img.png" alt="Pass Me A Coffee!!" style="height: 41px !important;width: 174px !important;box-shadow: 0px 3px 2px 0px rgba(190, 190, 190, 0.5) !important;-webkit-box-shadow: 0px 3px 2px 0px rgba(190, 190, 190, 0.5) !important;" ></a>

<!-- wp:quote -->
<blockquote class="wp-block-quote"><p>Hosted URL&nbsp;: <a href="https://fir-signin-4477d.firebaseapp.com/#/" rel="noreferrer noopener" target="_blank">https://fir-signin-4477d.firebaseapp.com/#/</a></p></blockquote>
<!-- /wp:quote -->

<!-- wp:quote -->
<blockquote class="wp-block-quote"><p><a href="https://github.com/AseemWangoo/experiments_with_web" rel="noreferrer noopener" target="_blank">Source code for Flutter Web App..</a></p></blockquote>
<!-- /wp:quote -->

{% youtube lYf5WwyzUHE %}