Share your Dinosaurs
Tags: mulesofthackathon, salesforce, einstionvisionapi, imageclassification
Published: 2020-11-29 09:29:10

#Every day is an API day: Share your Dinosaurs#
   WELCOME to Jurassic World! Do your kids love dinosaurs or do you love dinosaurs? This idea is inspired by my teammate's kid who is very into dinosaurs. With the love of a father, we make an API that can make drawing dinosaurs more interesting.
   We are using the latest AI technology to recognize the features of the picture and match it with the dinosaurs in our database.

Our application has two major functions.
   1. Match the drawing with dinosaurs in our database and analyze the ability of the drawing
   2. See the drawings of others and find friends with similar interest
  
The application will show the dinosaur that is most alike to the drawing and analysis its ability and strengths like power, speed, aggressivity, defensive, and intelligence.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/2pf9nw0d4lenexfj9lon.jpeg)
  When the users access the page, we may request to access their location and mark it on Google map. The users can check others' drawing and know where the drawing is from.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/usewybxknzdlfp5uz11l.png)

{% youtube i_PX_9m0Eho%}

#So What we need for this project:#
1. Anypoint Platform
2. Salesforce Developer Org
3. Amazon S3 

#STEP 1: Set up Salesforce Developer Org#
First of all, you need to set up the Einstein Vision api for image classification. For more information about [Einstein Vision api](https://metamind.readme.io/docs/introduction-to-the-einstein-predictive-vision-service)

You need to set up an account for Einstein Vision Api and get the Access Token. Please follow the step [here](https://metamind.readme.io/docs/what-you-need-to-call-api)
After we finished the steps above, we need to create a dataset and train it.

## Creating a dataset
You will need to collect at least 10 photos for each category of dinosaurs and save it under the folder with name of dinosaur. 
 ![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/xrpopg3e7aohocobl5g6.JPG)
When your dataset is ready, zip the file and create dataset using the following CURL:
```
curl -X POST -H "Authorization: Bearer <TOKEN>" -H "Cache-Control: no-cache" -H "Content-Type: multipart/form-data" -F "data=@C:\Data\mountainvsbeach.zip"  https://api.einstein.ai/v1/vision/datasets/upload/sync

curl -X POST -H "Authorization: Bearer <TOKEN>" -H "Cache-Control: no-cache" -H "Content-Type: multipart/form-data" -F "path=http://metamind.io/images/mountainvsbeach.zip"  https://api.einstein.ai/v1/vision/datasets/upload/sync
```
You will get the result with Id and then used that Id to train the dataset using the following CURL:

```
curl -X POST -H "Authorization: Bearer <TOKEN>" -H "Cache-Control: no-cache" -H "Content-Type: multipart/form-data" -F "name=Beach Mountain Model" -F "datasetId=57" https://api.einstein.ai/v1/vision/train
```
Here you will get the result with modelId. Please save it and we will use this in our Api Later. 

#STEP 2: Create API#
Now it's time to create API. What our API do is 
1.Accept the image file Upload
2.Predict the image 
3.Store it in S3 and get the presigned URI
4.Get the presigned URI of the photos with the same prediction result  
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/7jdbnluym23gx5n6n19t.JPG)

So there is only one endpoint in our API which is /fileUpload and the RAML is as below.

```
#%RAML 1.0
title: Dinosaur Api

/fileUpload:
  post:
    queryParameters:
      name:
        description: children name
        type: string
        required: true
      age:
        description: children age
        type: string
        required: false
      gender:
        type: string
        required: false
      Long:
        description: Location
        type: string
        required: false
      Lad:
        description: Location
        type: string
        required: false
    body:
      multipart/form-data:
        description: The file(s) to be uploaded
    responses:
      200:
        body:
          application/json:
            example:
              output: !include /example.raml
```
The following is the example of responses which include the information of dinosaur, user and friends.
```
#%RAML 1.0 NamedExample
example:
  {
  "dinosaurs_info": {
    "uploadPhotoURL": "url1",
    "samplePhotoURL": "url2",
    "dinosaurName" : "Tyrannosaurus",
    "sampleDinosaurInfo": {
      	"power": 80,
      	"speed": 67,
      	"attack": 76,
      	"defence": 8,
      	"Intelligence": 4
      },
    "power": 36,
    "speed": 89,
    "attack": 8,
    "defence": 11,
    "Intelligence": 29
  },
  "user_info": {
    "name": "Yamada Taro",
    "age": "6",
    "location": "lad_long"
  },
  "friend_list": [
    {
    "name": "Taro1",
    "age": "6",
    "location": "lad_long",
    "url":"url1" 
    },
    {
    "name": "Taro2",
    "age": "6",
    "location": "lad_long",
    "url":"url2" 
    }
  ]
}
```
The implementation of the flow is like this:
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/ah3g60b4c232dokb8cdl.JPG)

In this flow, instead of using transform message component, we do the transformation directly inside the target component. For example, in the HTTP request component(Predict), we wrote the dataweave directly in the body like below:
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/a0fe1ws5blniu2h5evmr.JPG)
As you can see, reducing the component make the flow look more simple.

###Details of Implementation
1. Predict
 This is a HTTP request component to make a request to Einstein vision API(/Prediction with Image Base64 String). Because of mime type of the input is Multipart/form-data, I got the content of the input data with payload.parts[0].content and transform it to another Multipart/form-data format data for Einstien Vision API prediction as the capture shown above.For more information about [Multipart/form-data in Mulesoft](https://docs.mulesoft.com/mule-runtime/4.3/dataweave-formats-multipart)

2.Set myVar
I set all the information of user in file name concatenate with underscore and as an object user_info to used in the output.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/zanir09mum0k8hor7hhr.JPG)

3.Create object and Create object presigned uri
Uploading the photo to S3 and get the presigned URI, I use the bucket name, I used the property value.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/h6722bjlq9p57b6th9ll.JPG)

As the input data is base64 string, I need to decode it in order to upload it to S3. I also do this directly inside the target component instead of using transform message component.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/9362qe877m6zsqn25yqu.JPG)

Here I stored the result payload in variables by setting up the target variable on Advanced tab.
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/hvg4mcpmo2bgwkijjakp.JPG)

4. Init friend list and for loop
I create an array List(friendList) and adding value of it by using forEach component with the result of ListObject.  
![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/alp1goe8o78sf4pvsmy6.JPG)
When uploading photo I stored all the information of the user in the file name by concatenating with underscore and here I want to set the information as an object so I used SplitBy call.

```
%dw 2.0
import * from dw::core::Strings
output application/json
var friendInfo = substringBeforeLast(payload, ".") splitBy("_")
---
{
    "user_name": friendInfo[1],
    "age" : friendInfo[2],
    "gender" : friendInfo[3],
    "location_Long": friendInfo[4],
    "location_Lad": friendInfo[5],
    "url" : vars.friendPhotoURL
}
```

5. Set up the response value
Final Step is to set up the response value

```
%dw 2.0
output application/json
---
{
  "dinosaurs_info": { 
      "uploadPhotoURL" : vars.uploadPhotoURL,
      "samplePhotoURL" : vars.samplePhotoURL,
      "dinosaurName" : vars.response.probabilities[0].label,
      "sampleDinosaurInfo": {
      	"power": round(random() * 100),
      	"speed": round(random() * 100),
      	"attack": round(random() * 100),
      	"defence": round(random() * 100),
      	"Intelligence": round(random() * 100)
      },
      "power": round(vars.response.probabilities[0].probability * 100),
      "speed": round(random() * 100),
      "attack": round(random() * 100),
      "defence": round(random() * 100),
      "Intelligence": round(random() * 100)
      },
     "user_info": vars.myVar.user_info,
     "friend_list": vars.friendlist
  }

```

#Step 3: Set up UI on Salesforce Community Cloud（Experience Cloud）
We used Lightning Web Component for our pages and you can get the code from here: [salesforceCode](https://github.com/HninPwintP/MuleHackathon)

#About Our team
Keitaro K
Kazutaka U
Takamasa　H
Yun F
Holiday J

We are salesforce developers in Japan. We started learning Mulesoft this year. We decided to take part in this activity to learn more about this field. After discussing among the team, we decided the above topic because one of our member's children is currently crazy about dinosaurs and we think there are many kids with this interest around the world. By joining this competition, we not only feel accomplished for creating something fun for children, but also gain a bunch of knowledge during the implementation. 

Reference 
Dinosaur Api: https://github.com/HninPwintP/mulesoftHackathon2020/tree/master/dinosaur-api2
Salesforce code: https://github.com/HninPwintP/MuleHackathon