Building an Image Recognition Website with SvelteKit and TensorFlow.js
Tags: beginners, tutorial, machinelearning, svelte
Published: 2024-02-20 07:39:22

## Introduction

In this article, I'll show you how to build a simple website that allows users to upload an image and get predictions about its content using MobileNet model from TensorFlow.js. This project is perfect for beginners in TensorFlow.js who want to get started with machine learning in the browser.

## Prerequisites

Before you start, you should have some knowledge of web development and be familiar with JavaScript, TypeScript and Tailwind CSS. Make sure the following tools are also installed:

- npm or yarn
- VSCode (https://code.visualstudio.com/download)
- Svelte extension (https://marketplace.visualstudio.com/items?itemName=svelte.svelte-vscode)

## Step 1: Create a new SvelteKit project

Start by creating a new SvelteKit project using the following command in the terminal:

```bash
npm init svelte@next my-project
```

![Screenshot of SvelteKit installation](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/xp9zgw03o3thjdpfkf11.png)

![Screenshot of the end of SvelteKit installation](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/5h3c9quk5lmnkk6j2od5.png)

## Step 2: Install Tailwind CSS and Flowbite Svelte

- Tailwind CSS is a CSS framework. [https://tailwindcss.com/](https://tailwindcss.com/)
- Flowbite Svelte is an official Flowbite component library for Svelte. [https://flowbite-svelte.com/](https://flowbite-svelte.com/)

Install Tailwind CSS in a SvelteKit project in the terminal:

```bash
npm install -D tailwindcss postcss autoprefixer
npx tailwindcss init -p
```

Update your `tailwind.config.js` file:

```javascript
/** @type {import('tailwindcss').Config} */
export default {
  content: ['./src/**/*.{html,js,svelte,ts}'],
  theme: {
    extend: {}
  },
  plugins: []
};
```

Create `./src/app.css` file:

```css
@tailwind base;
@tailwind components;
@tailwind utilities;
```

Create a `./src/routes/+layout.svelte`fle and importe the `app.css` file:

```javascript
<script>
  import "../app.css";
</script>

<slot />
```

Install Flowbite Svelte in the terminal:

```bash
npm i -D flowbite-svelte flowbite
```

Update your `tailwind.config.js` file:

```javascript
/** @type {import('tailwindcss').Config} */
export default {
	content: [
		'./src/**/*.{html,js,svelte,ts}',
		'./node_modules/flowbite-svelte/**/*.{html,js,svelte,ts}'
	],
	theme: {
		extend: {
			colors: {
				// flowbite-svelte
				primary: {
					50: '#FFF5F2',
					100: '#FFF1EE',
					200: '#FFE4DE',
					300: '#FFD5CC',
					400: '#FFBCAD',
					500: '#FE795D',
					600: '#EF562F',
					700: '#EB4F27',
					800: '#CC4522',
					900: '#A5371B'
				}
			}
		}
	},
	plugins: [require('flowbite/plugin')]
};
```

## Step 3: Install TensorFlow.js
- TensorFlow.js documentation [https://www.tensorflow.org/js](https://www.tensorflow.org/js)
- Pre-trained TensorFlow.js models [https://github.com/tensorflow/tfjs-models#readme](https://github.com/tensorflow/tfjs-models#readme)

Install TensorFlow.js in the terminal:

```bash
npm i @tensorflow-models/mobilenet @tensorflow/tfjs @tensorflow/tfjs-vis
```

## Step 4: Import the MobileNet model
Go to `./src/routes/+page.svelte` and delete everything there.

Create a `<script lang="ts"></script>` tag and add these imports:

```javascript
<script lang="ts">
	import * as tf from '@tensorflow/tfjs';
	import * as mobilenet from '@tensorflow-models/mobilenet';
</script>
```

## Step 5: Create a function to load the model

Create a function to load the MobileNet model and return it. Add this code in `script` tag under imports:

```javascript
let model: mobilenet.MobileNet;

const loadModel = async () => {
		const version = 2;
		const alpha = 0.5;
		model = await mobilenet.load({ version, alpha });
		console.log('Model loaded successfully');
		return model;
	};
```

## Step 6: Create a function to preprocess an image before using it in a machine learning model.

Add this code in `script` tag below the previous function:

```javascript
const preprocessImage = (imageElement: HTMLImageElement) => {
		const imageTensor = tf.browser.fromPixels(imageElement);
		const resizedImageTensor = tf.image.resizeBilinear(
			imageTensor,
			[224, 224]
		);
		return resizedImageTensor;
	};
```

Explanation of the preprocessImage function
Feature: Preprocess an image before using it in a machine learning model.

**Inputs:**

`imageElement`: An `HTMLImageElement` representing an image in the HTML DOM.

**Steps:**

Convert the image to 3D tensor (height, width, color channels) with `tf.browser.fromPixels`.
Resize the image to 224x224 pixels using bilinear interpolation: `tf.image.resizeBilinear`.

`Output:`

A tensor representing the preprocessed image, ready for use in a machine learning model.

## Step 7: Create a function to make a prediction

Add this code in `script` tag below the previous function:

```javascript
const classifyImage = async (
		model: mobilenet.MobileNet,
		preprocessedImage: tf.Tensor3D
	) => {
		const predictions = await model.classify(preprocessedImage);
		return predictions;
	};
```

## Step 8: Put it all together

Now we need to combine the functions we created to load the model and make predictions. Add this code in `script` tag below the previous function:

```javascript
let predictionsResult: Prediction[] = [];

const predictionImage = async (imgDisplay: HTMLImageElement) => {
		const model = await loadModel();
		const preprocessedImage = preprocessImage(imgDisplay);
		predictionsResult = await classifyImage(model, preprocessedImage);
	};
```

## Step 9: Create an user interface with Flowbite Svelte and Tailwind CSS.
I put here the complete code of the `+page.svelte` file. I added some elements like a spinner or a TypeScript interface.

```javascript
<script lang="ts">
	import * as tf from '@tensorflow/tfjs';
	import * as mobilenet from '@tensorflow-models/mobilenet';
	import {
		Label,
		Fileupload,
		Heading,
		List,
		Li,
		Card,
		Spinner
	} from 'flowbite-svelte';

	let fileuploadprops = {
		id: 'inputImage'
	};

	let predictionsResult: Prediction[] = [];
	let imgDisplay: string;
	let files: FileList;
	let imgtoprocess: HTMLImageElement;
	let model: mobilenet.MobileNet;
	let predictionsLoading = false;

	$: if (files) {
		imgDisplay = URL.createObjectURL(files[0]);
	}

	interface Prediction {
		className: string;
		probability: number;
	}

	const loadModel = async () => {
		const version = 2;
		const alpha = 0.5;
		model = await mobilenet.load({ version, alpha });
		console.log('Model loaded successfully');
		return model;
	};

	const preprocessImage = (imageElement: HTMLImageElement) => {
		const imageTensor = tf.browser.fromPixels(imageElement);
		const resizedImageTensor = tf.image.resizeBilinear(
			imageTensor,
			[224, 224]
		);
		return resizedImageTensor;
	};

	const classifyImage = async (
		model: mobilenet.MobileNet,
		preprocessedImage: tf.Tensor3D
	) => {
		const predictions = await model.classify(preprocessedImage);
		return predictions;
	};

	const predictionImage = async (imgDisplay: HTMLImageElement) => {
		predictionsLoading = true;
		const model = await loadModel();
		const preprocessedImage = preprocessImage(imgDisplay);
		predictionsResult = await classifyImage(model, preprocessedImage);
		predictionsLoading = false;
	};

	const handleImageChange = () => {
		predictionImage(imgtoprocess);
		URL.revokeObjectURL(imgDisplay);
	};
</script>

<svelte:head>
	<title>ML Vision</title>
	<meta
		name="description"
		content="A simple ML Vision app"
	/>
</svelte:head>

<section class="flex flex-col gap-12">
	<Heading tag="h1" class="text-center">ML Vision with MobileNet images Classifier</Heading>
	<div class="ml-auto mr-auto block w-1/2">
		<Label class="pb-2 text-lg">Select an image to classify</Label>
		<Fileupload
			{...fileuploadprops}
			bind:files
			on:change={handleImageChange}
		/>
	</div>
	<Card
		size="md"
		padding="sm"
		class="ml-auto mr-auto"
	>
		<img
			hidden={!imgDisplay}
			src={imgDisplay}
			alt=""
			bind:this={imgtoprocess}
		/>
		<div class="py-4">
			<Heading tag="h3">Predictions:</Heading>
			<List
				position="outside"
				class="p-4"
			>
				{#if predictionsLoading}
					<span class="flex flex-col items-center"
						><Spinner color="blue" /></span
					>
				{:else}
					{#each predictionsResult as prediction}
						<Li
							>{prediction.className}: {Math.round(
								prediction.probability * 100
							)}%</Li
						>
					{/each}
				{/if}
			</List>
		</div>
	</Card>
</section>
```

## Step 10: Add Svelte components 
I create a Footer and Menu components and display them in `+layout.svelte` file.

- Create a file `./src/lib/components/footer/Footer.svelte` and add this code:

```javascript
<script>
	import {
		Footer,
		FooterCopyright,
	} from 'flowbite-svelte';
</script>

<Footer class="rounded-none bg-black p-4 shadow md:px-6 md:py-8">
	<hr class="my-6 border-white sm:mx-auto lg:my-8" />
	<FooterCopyright
		aClass="text-[#6C70ED] text-base"
		spanClass="text-white block text-sm text-center"
		href="/"
		by="Flowbite"
	/>
</Footer>
```

- Create a file `./src/lib/components/menu/Menu.svelte` and add this code:

```javascript
<script lang="ts">
	import {
		Navbar,
		NavBrand,
		NavLi,
		NavUl,
		NavHamburger
	} from 'flowbite-svelte';
</script>

<Navbar
	let:hidden
	let:toggle
    class="bg-black text-white"
>
	<NavBrand href="/">
	</NavBrand>
	<NavHamburger on:click={toggle} class="hover:bg-[#6C70ED]" menuClass="outline-[#6C70ED]" />
 	<NavUl {hidden} ulClass="bg-black flex flex-col p-4 mt-4 md:flex-row md:space-x-8 rtl:space-x-reverse md:mt-0 md:text-sm md:font-medium" divClass="w-full md:block md:w-auto">
		<NavLi
			href="/"
			active={true} class="md:text-white md:hover:text-[#6C70ED]" nonActiveClass="text-white">Home</NavLi
		>
	</NavUl>
</Navbar>
```

- Then edit the `./src/routes/+layout.svelte` file with this code:

```javascript
<script>
	import Menu from '$lib/components/menu/Menu.svelte';
	import Footer from '$lib/components/footer/Footer.svelte';
	import '../app.css';
</script>

<div class="flex flex-col min-h-screen">
	<Menu />

	<main class="flex flex-col flex-1 p-4 w-full box-border	max-w-5xl mt-0 mb-0 ml-auto mr-auto">
		<slot />
	</main>

	<Footer />
</div>
```

## Step 11: Try your project 
To run your project:

```bash
npm run dev -- --open
```

Choose an image with a dog or a vegetable. See the result!

## Conclusion

In this article, you learned how to create a simple image recognition website with SvelteKit and TensorFlow.js. This project is a great starting point for learning about machine learning in the browser.

## Project on Github

[https://github.com/Estelle-K/ml-vision](https://github.com/Estelle-K/ml-vision)