Image Classification for Natural Disaster Damage with Amazon SageMaker Jumpstart
Tags: aws, classification, ai, tutorial
Published: 2023-06-19 16:48:17

## Recovery in the aftermath of a natural disaster

Northern New South Wales experienced multiple floods in 2022, the town of Lismore did not have measures to predict or prevent natural disasters in the low-lying flood-prone region.


After the 2022 floods, many communities were displaced and homes were declared uninhabitable.

There are a few questions to consider in the aftermath:

* How do government to help citizens find temporary accommodation? How do they assess photos of flood damage?

* How can insurance companies assess photos of flood damage to homes and businesses for residential and commercial claims?

Machine learning can be used to explore patterns instead of relying on human intuition to classify images.

### Learning Objectives

In this lesson you will learn how to:

* Onboard Amazon SageMaker Domain
* Set up Amazon SageMaker Studio 
* Import a Jupyter Notebook
* Import your own dataset or use an existing dataset
* Train and deploy an image classification model using Amazon SageMaker Jumpstart
* Clean up resources 

### What is Amazon SageMaker Jumpstart?

You may train and deploy a machine learning model to solve your business problems with [pre-trained and open-source models](https://docs.aws.amazon.com/sagemaker/latest/dg/studio-jumpstart.html) with a few clicks. You may also use [SageMaker Python SDK] (https://sagemaker.readthedocs.io/en/stable/overview.html#use-sagemaker-jumpstart-algorithms-with-pretrained-models) for programmatic access.
SageMaker Jumpstart Industry notebooks can only be run on Amazon SageMaker Studio.

### What are the benefits?

You may build, experiment and deploy machine models with:

**[Foundation models](https://docs.aws.amazon.com/sagemaker/latest/dg/jumpstart-foundation-models.html)** 

You may access pre-trained foundation models for your business use case. This allows you to browse and choose from a large library of model providers, build, experiment, customize and deploy your Generative AI models.

**[Built-in algorithms and pre-trained models](https://docs.aws.amazon.com/sagemaker/latest/dg/jumpstart-models.html)**

You may access built-in algorithms and pre-trained models to help you achieve specific tasks such as vision, text, audio and documents. The models can be fine-tuned and also deployed quickly especially if you need to test and build from an idea or productionise your machine learning models.

**[Solution Templates](https://aws.amazon.com/sagemaker/jumpstart/?refid=d99f4c97-e210-42bd-9558-53ea535b290b&sagemaker-data-wrangler-whats-new.sort-by=item.additionalFields.postDateTime&sagemaker-data-wrangler-whats-new.sort-order=desc)**

You may access pre-built ML solutions, browse the solution templates, select a template that includes for your use case and customize by using your own data and deployment is a click of a button.

* Share machine learning artifacts

### What are common use cases for Amazon SageMaker Jumpstart?

Amazon SageMaker Jumpstart suggests the following [use cases](https://docs.aws.amazon.com/sagemaker/latest/dg/jumpstart-solutions.html):

* Demand forecasting

* Credit rating prediction

* Fraud detection

* Computer vision

* Extract and analyze data from documents

* Predictive maintenance

* Churn prediction

* Personalized recommendations

* Reinforcement learning

* Healthcare and life sciences

* Financial pricing

* Causal inference

### Solution Architecture

This is my proposed diagram of using Amazon SageMaker Jumpstart with pre-trained ML solution templates that are initiated with Amazon CloudFormation.


![Image class](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0ozznczctwcffykvueoz.jpg)

### Dataset

The Hurricane dataset was provided in the **[Financial Services Lab](https://catalog.us-east-1.prod.workshops.aws/workshops/f560a788-af64-4e5a-a02c-a6c88516ab02/en-US/industrylab/finserv/damage-detection)** for damage classification.

### Pre-requisites

* You will need to have an existing AWS account or you may create one [here](https://dev.to/aws-builders/getting-started-with-aws-a-sweet-journey-5cjj).
* You will need to login as an Administrative User and [Group](https://dev.to/aws-builders/creating-your-first-iam-admin-user-and-user-group-in-your-aws-account-machine-learning-part-1-3cne)

Ensure that your IAM permissions for Administrative User allows you access to the following AWS services:

```
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "sagemaker:*"
            ],
            "Resource": [
                "arn:aws:sagemaker:*:*:domain/*",
                "arn:aws:sagemaker:*:*:user-profile/*",
                "arn:aws:sagemaker:*:*:app/*",
                "arn:aws:sagemaker:*:*:flow-definition/*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "iam:GetRole",
                "servicecatalog:*"
            ],
            "Resource": [
                "*"
            ]
        }
    ]
}
```

* [Onboard Amazon SageMaker Domain](https://docs.aws.amazon.com/sagemaker/latest/dg/onboard-quick-start.html)

### Tutorial 1: [Onboard to Amazon SageMaker Domain](https://docs.aws.amazon.com/sagemaker/latest/dg/onboard-quick-start.html)

Before you start using Amazon SageMaker you will also have to onboard SageMaker Domain.

Step 1: Navigate to Amazon SageMaker and select the AWS region  that you prefer to use. I will select AWS region **Asia Pacific ap-southeast-2**.


![Sagemkaer](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/nvgbiq8t32d0q92hctj7.jpg)

Step 2: Choose **Domains** on the left-hand side menu.

![domain](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/jbsk9bcvb6hftdo81qzj.png)

Step 3: Select **Create domain**.


![create domain](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/86s5miz45xe6ypi5y2kp.jpg)

Step 4: Select **Quick Setup** on the left-hand side menu.


![quick](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/u2zvzvynu0jsfq8on24c.jpg)


Step 5: Create a unique domain name.

![domain name](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/yojj24enwrekj9t89bzi.jpg)


Step 6: You may use the default name under **User profile** or you may create a unique name.


![Iuser](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/a2iqpwncca5ve3yyy78j.jpg)

You may select from the drop-down menu, the execution role  **SagemakerFullAccessrole** and click **Submit**.

If you do not have this execution role, you may create one [here](https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-roles.html).

Also, ensure that the box is checked for enable 'SageMaker Canvas permissions' and finally click **Submit**.

Step 7: Under VPC, you may choose an **existing VPC** or the **default VPC**. Under the default VPC, two subnets were selected and click **Save and continue**.


![vpc](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/xajptr9bqjfgqunkrqdf.jpg)

In a few minutes, the domain will be onboarded with the status ready with the User profile name.


![user ready](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/nx3rjzy9tc7d42b61kpu.jpg)


### Tutorial 2: [Launch Amazon SageMaker Studio Using Amazon Management Console](https://docs.aws.amazon.com/sagemaker/latest/dg/studio-launch.html)

Step 1: In the search bar, type the word 'SageMaker' to open the Amazon SageMaker Console. On the left-hand side menu navigate to **Studio** and you will see the new domain name created for the user profile, click **Open Studio**.


![Iopenn](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/2afteoyi4vubs40yvphc.jpg)


Amazon SageMaker Studio will take a few moments to launch.

![launch](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7unel8ek8o93xm3uttdq.jpg)

You will be directed to the Amazon SageMaker Studio homepage to confirm that you have successfully launched a [SageMaker Studio Domain](https://catalog.us-east-1.prod.workshops.aws/workshops/f560a788-af64-4e5a-a02c-a6c88516ab02/en-US/introduction/setup-sagemaker).


![home studio](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/07vv19s6t44efw2737jl.jpg)


### Tutorial 3: [How to train and deploy an image classification model using Amazon SageMaker Jumpstart](https://catalog.us-east-1.prod.workshops.aws/workshops/f560a788-af64-4e5a-a02c-a6c88516ab02/en-US/industrylab/finserv/damage-detection)

In this tutorial, Amazon SageMaker Jumpstart will be used to train an image classification model on the Hurricane Harvey 2017 dataset.

Step 1:  The dataset and Jupyter notebooks will be imported into Amazon SageMaker Studio by:

 * Selecting **File -> New -> Terminal**


![import](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/eioivzb943rcpz2qixjv.jpg)

The terminal environment will launch as shown below:


![envi](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/f4zuqv9kth1lpl3w0ill.jpg)

Step 2: Download the Hurricane Harvey 2017 dataset to the Amazon SageMaker Notebook by copying this code into the Terminal

```
mkdir lcnc
cd lcnc
sudo yum install -y unzip
curl 'https://static.us-east-1.prod.workshops.aws/public/40de25f9-f9de-4fba-8871-0bf4761d175e/static/resources/finserv/vision.zip' --output vision.zip
unzip vision.zip

```
The image files will take a few seconds to be imported.


![data import](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/jj35c1sr2602n2zzqegq.jpg)

Step 3: Open the Data pane on the left-handside menu click to view the 'lcnc' folder.


![Ilcnc](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/006o1n9adg0aisr0yz5o.jpg)

Step 4: Explore the data and copy the data from the local SageMaker folder to S3 and navigate to the **lcnc/vision/** folder.


![double click](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/slujld638kdzpyk6a1ww.jpg)

Step 5: Double-click on the jupyter notebook 'explore-data.ipynb' and click **Select**.

![Ijup](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/gjfrtcdth0wz3yofhclw.jpg)

This will initiate the kernel.


![sart kernn](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/bbv45fmrrrg86hoxd0pt.jpg)

Step 6: Run and execute all the cells in the jupyter notebook by pressing on your keyboard **Ctrl + Enter**.


![execute](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/1zhrtec3p22susv91h7d.jpg)

The files in the local folder have been successfully uploaded into your Amazon S3 bucket.


![uploads3](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/kcje4ziulsfey2vzsvgt.jpg)


You may also check the that the image files have been uploaded from local folder and into your Amazon S3 bucket 'damage-clf' folder.


![file](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/24crax8sjmetcdonb3s5.jpg)


![damageon](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/vx8rk90fvgxvqoeltd51.jpg)


Step 7: Click on the 'home' icon to navigate to the homepage and scroll down to the bottom to **SageMaker Jumpstart** and click 'Models, notebooks, solutions' to view the 'Model zoo' which includes pre-trained models, example notebooks and pre-built solutions.


![model zoo](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/10a011jpnrmb5xcpclll.jpg)

Step 8: Scroll down until you reach the section 'Explore all image classification models (162)' and double-click this hyperlink.

![Image class](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/pw2vpjxaayw9dgxajdoi.jpg)


Step 9: Select the model **Resnet 50** and click **View model**.


![Ires](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8nz4k5e0j7lactaxuvt5.jpg)

Step 10: From this pre-trained 'Resnet 50' model, you may click **deploy** to deploy the pre-trained model (ImageNet) as a SageMaker real-time endpoint to get inference.


![deplpy](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ps7z4h796ww6990cj573.jpg)

The deployment process will take a few seconds to complete.


![process](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/v88h90iqwwz4d6hcsj8m.jpg)

The model endpoint is in service and ready to make inference.


![make inference](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/1am5t9wt1bk2d9sro9qt.jpg)


### Tutorial 4: Fine tune the model and bring in custom data

Step 1: To fine-tune the model on custom hurricane damage images, choose the S3 location of the images we just uploaded using the 'explore-data notebook' and specify the instance type we want to use for training as shown below.


![train](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/tpxtg7uw165cj7hyolap.jpg)


Step 2: Select the instance type 'ml.c5.2xlarge'.


![deplpy comfig](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/mkxsnngabbsoq90sw5w7.jpg)


Step 3: Provide a model name e.g. hurricane-damage-abc

Step 4: Custom output S3 bucket

Select **Default output S3 bucket**.

![custom](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/dzgfd876evjpau94ftbc.jpg)

Copy the S3 URI of the training dataset.


![copy uri](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/yejwjparxyn658ytabga.jpg)

Step 5: Update the hyper parameters with the configuration below and select **Train**.


![hypern](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/tn5imbhshegdiawicbqk.jpg)

Training the model will take a few moments.


![training](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/naxwrt7hmir4ojyxians.jpg)

The model was successfully trained using custom data uploaded into Amazon S3.


![trained model](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/8fytdc10z9lx5d06d8nk.jpg)

Note: You may also inspect the training model artifacts saved in the Amazon S3 bucket.


![training artifacts](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/d7g24ipbnh4xsivlo8au.jpg)

Step 5:  To deploy the fine-tuned model, configure the settings such as specifying the S3 URI path to save the model artifact.


![deplpy custom](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/9dz74prxzr2a6lopoy9n.jpg)

![test modeln](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/lzxt2uqxvx45vxa0obuw.jpg)

Step 6: Under **Security settings**, select 'Find VPC', choose three subnets and also the security group and click **Deploy**.

After a few minutes, the endpoint is ready with the status'in service'.


![endpointcreated](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hsxnzkamp9dytxggbeqb.jpg)


Step 7: On the left-handside pane, double-click on the jupyter notebook 'make-predictions.ipynb'.

And execute the cells in the notebook to make predictions.


![doublr](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/th6ejnyn0c7emsmjt5o0.jpg)

**Important note:** On cell 5, be sure to replace default endpoint name from screenshot below and enter the endpoint name for your deployed model from Step 6.


![enspoint](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/szj6d1r7xfmi9ckud5k4.jpg)

Step 8: Make predictions from the test data 'damaged'.

When making predictions on new data (i.e. photos), the probability is that property is 'damaged'.

![test data](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/jdktvojcyevmpzbbouzr.jpg)

Step 9: Make predictions from the test data 'not-damaged'.

When making predictions on new data (i.e. photos), the probability is that property is 'not damaged'.


![not](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/gr15w6bwfa4kdlde54qe.jpg)

### Clean Up Resources

Once you have successfully deployed the model you must delete the endpoint to avoid surprise end of month bills.

Step 1: Delete the endpoints by navigating to **Deployments-> endpoints**


![delete](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/tnusnb3snu2yg2oqe3pe.jpg)

Step 2: Click on the hyperlinks to delete the model endpoint.


![one by one](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/48fdxb9ufrtbd59y2wc8.jpg)

![delete endpoint](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/bol3vew82urf7ixw9tdi.jpg)


![endpoint deeleetd](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/prxoffyy81p17inozjz4.jpg)


### Conclusion

In this lesson, you have learnt how to set up Amazon SageMaker Studio for your machine learning project and also deploy an image classification model using pre-trained models in Amazon SageMaker Jumpstart. Keep on building and exploring in the 'Model Zoo' of Amazon SageMaker Jumpstart.

Until the next lesson, happy learning! 😀

### References

* [AWS Machine Learning Low-Code Immersion Day: Damage Detection](https://catalog.us-east-1.prod.workshops.aws/workshops/f560a788-af64-4e5a-a02c-a6c88516ab02/en-US/industrylab/finserv/damage-detection)

* [Setting up your machine learning project quickly with Amazon SageMaker Studio](https://aws.amazon.com/getting-started/hands-on/machine-learning-tutorial-get-started-with-your-project-quickly/)

* [Amazon SageMaker Developer Guide](https://docs.aws.amazon.com/sagemaker/latest/dg/studio-jumpstart.html)

### Resources

* [Amazon SageMaker Python SDK](https://docs.aws.amazon.com/sagemaker/latest/dg/studio-jumpstart-industry.html)

* [Amazon SageMaker Jumpstart Industry: Financial](https://docs.aws.amazon.com/sagemaker/latest/dg/studio-jumpstart-industry.html)

* [Amazon SageMaker Jumpstart Industry pack on Github](https://github.com/aws/sagemaker-jumpstart-industry-pack/)

* [Amazon SageMaker JumpStart Industry Python SDK](https://sagemaker-jumpstart-industry-pack.readthedocs.io/en/latest/notebooks/index.html#)

* [How to create an AWS account](https://docs.aws.amazon.com/sagemaker/latest/dg/gs-set-up.html)

* [AWS Command Line Interface](https://docs.aws.amazon.com/cli/v1/userguide/install-windows.html)

### Next Lesson

The next few lessons will delve into a mix of classic machine learning modelling techniques as well as AI.

### Last week: AWS re:Inforce 2023 on 13-14 June 

You may watch on Youtube the keynote from CJ Moses, Chief Information Security Officer (CISO), AWS last week. You may also watch the leadership sessions, keynotes and breakout sessions from AWS re:Inforce 2023 at this [link](https://www.youtube.com/channel/UCdoadna9HFHsxXWhafhNvKw).

{% embed https://youtu.be/_piUB5FrYVE %}

### Coming soon: AWS re:Invent 2023 conference 

You may [register](https://reinvent.awsevents.com/) now for AWS re:Invent 2023 conference on November 27 to December 1 2023 in Las Vegas.

You may watch the AWS re:Invent 2022 keynote from Amazon CEO Adam Selipsky on Youtube on-demand.

{% embed https://www.youtube.com/watch?v=Xus8C2s5K9A&t=2861s %}