[Huawei Developers] Basavaraj.navi:
View attachment 5400719

Introduction

In this article, we will learn how to integrate Huawei Scene detection using Huawei HiAI. We will build the Pets cart where we can sell pets online and filter pets by scene detection using Huawei HiAI.

What is Scene Detection?

Scene detection can quickly classify images by identifying the type of scene to which the image content belongs, such as animals, green plants, food, buildings, and automobiles. Scene detection can also add smart classification labels to images, facilitating smart album generation and category-based image management.

Features

Fast: This algorithm is currently developed based on the deep neural network, to fully utilize the neural processing unit (NPU) of Huawei mobile phones to accelerate the neural network, achieving an acceleration of over 10 times.

Lightweight: This API greatly reduces the computing time and ROM space the algorithm model takes up, making your app more lightweight.

Abundant: Scene detection can identify 103 scenarios such as Cat, Dog, Snow, Cloudy sky, Beach, Greenery, Document, Stage, Fireworks, Food, Sunset, Blue sky, Flowers, Night, Bicycle, Historical buildings, Panda, Car, and Autumn leaves. The detection average accuracy is over 95% and the average recall rate is over 85% (lab data).

How to integrate Scene Detection

1. Configure the application on the AGC.

2. Apply for HiAI Engine Library

3. Client application development process.

Configure application on the AGC

Follow the steps

Step 1: We need to register as a developer account in AppGallery Connect. If you are already a developer ignore this step.

Step 2: Create an app by referring to Creating a Project and Creating an App in the Project

Step 3: Set the data storage location based on the current location.

Step 4: Generating a Signing Certificate Fingerprint.

Step 5: Configuring the Signing Certificate Fingerprint.

Step 6: Download your agconnect-services.json file, paste it into the app root directory.

Apply for HiAI Engine Library

What is Huawei HiAI?

HiAI is Huawei’s AI computing platform. HUAWEI HiAI is a mobile terminal–oriented artificial intelligence (AI) computing platform that constructs three layers of ecology: service capability openness, application capability openness, and chip capability openness. The three-layer open platform that integrates terminals, chips, and the cloud brings more extraordinary experience for users and developers.

How to apply for HiAI Engine?

Follow the steps

Step 1: Navigate to this URL, choose App Service > Development, and click HUAWEI HiAI.

Step 2: Click Apply for the HUAWEI HiAI kit.

Step 3: Enter required information like Product name and Package name, click the Next button.

Step 4: Verify the application details and click Submit button.

Step 5: Click the Download SDK button to open the SDK list.

Step 6: Unzip downloaded SDK and add to your android project under the libs folder.

Step 7: Add jar files dependencies into app build.gradle file.

implementation fileTree(include: ['*.aar', '*.jar'], dir: 'libs')
implementation 'com.google.code.gson:gson:2.8.6'
repositories {
flatDir {
dirs 'libs'
}

}Copy code

Client application development process

Follow the steps

Step 1: Create an Android application in the Android studio (Any IDE which is your favorite).

Step 2: Add the App level Gradle dependencies. Choose inside project Android > app > build.gradle.

apply plugin: 'com.android.application'
apply plugin: 'com.huawei.agconnect'

Root level gradle dependencies.

maven { url 'https://developer.huawei.com/repo/' }
classpath 'com.huawei.agconnect:agcp:1.4.1.300'

Step 3: Add permission in AndroidManifest.xml

<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
<!-- CAMERA -->
<uses-permission android:name="android.permission.CAMERA" />
<uses-feature android:name="android.hardware.camera" />
<uses-feature android:name="android.hardware.camera.autofocus" />

Step 4: Build application.

First request run time permission

private void requestPermissions() {
try {
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.M) {
int permission = ActivityCompat.checkSelfPermission(this,
Manifest.permission.WRITE_EXTERNAL_STORAGE);
if (permission != PackageManager.PERMISSION_GRANTED) {
ActivityCompat.requestPermissions(this, new String[]{Manifest.permission.WRITE_EXTERNAL_STORAGE,
Manifest.permission.READ_EXTERNAL_STORAGE, Manifest.permission.CAMERA}, 0x0010);
}
}
} catch (Exception e) {
e.printStackTrace();
}
}Copy code
Initialize vision base

private void initVisionBase() {
VisionBase.init(SceneDetectionActivity.this, new ConnectionCallback() {
[USER=439709]@override[/USER]
public void onServiceConnect() {
//This callback method is called when the connection to the service is successful.
//Here you can initialize the detector class, mark the service connection status, and more.
Log.i(LOG, "onServiceConnect ");
Toast.makeText(SceneDetectionActivity.this, "Service Connected", Toast.LENGTH_SHORT).show();
}

[USER=439709]@override[/USER]
public void onServiceDisconnect() {
//This callback method is called when disconnected from the service.
//You can choose to reconnect here or to handle exceptions.
Log.i(LOG, "onServiceDisconnect");
Toast.makeText(SceneDetectionActivity.this, "Service Disconnected", Toast.LENGTH_SHORT).show();
}
});
}

Build Async class for scene detection.

class SceneDetectionAsync extends AsyncTask<Bitmap, Void, JSONObject> {

[USER=439709]@override[/USER]
protected JSONObject doInBackground(Bitmap... bitmaps) {

//Bitmap bitmap = BitmapFactory.decodeFile(imgPath);//Obtain the Bitmap image. (Note that the Bitmap must be in the ARGB8888 format, that is, bitmap.getConfig() == Bitmap.Config.ARGB8888.)
Frame frame = new Frame();//Construct the Frame object
frame.setBitmap(bitmaps[0]);
SceneDetector sceneDetector = new SceneDetector(SceneDetectionActivity.this);//Construct Detector.
JSONObject jsonScene = sceneDetector.detect(frame, null);//Perform scene detection.
Scene sc = sceneDetector.convertResult(jsonScene);//Obtain the Java class result.
if (sc != null) {
int type = sc.getType();//Obtain the identified scene type.
Log.d(LOG, "Type:" + type);
}
Log.d(LOG, "Json data:" + jsonScene.toString());
return jsonScene;
}

[USER=439709]@override[/USER]
protected void onPostExecute(JSONObject data) {
super.onPostExecute(data);
if (dialog != null && dialog.isShowing()) {
dialog.dismiss();
}
adapter = new MyListAdapter(getPetsFilteredDataList(data));
recyclerView.setAdapter(adapter);
Toast.makeText(SceneDetectionActivity.this, "Data filtered successfully", Toast.LENGTH_SHORT).show();

}
}Copy code
Show select image dialog.

private void selectImage() {
try {
PackageManager pm = getPackageManager();
int hasPerm = pm.checkPermission(Manifest.permission.CAMERA, getPackageName());
if (hasPerm == PackageManager.PERMISSION_GRANTED) {
final CharSequence[] options = {"Take Photo", "Choose From Gallery", "Cancel"};
androidx.appcompat.app.AlertDialog.Builder builder = new androidx.appcompat.app.AlertDialog.Builder(this);
builder.setTitle("Select Option");
builder.setItems(options, new DialogInterface.OnClickListener() {
[USER=439709]@override[/USER]
public void onClick(DialogInterface dialog, int item) {
if (options[item].equals("Take Photo")) {
dialog.dismiss();
fileUri = getOutputMediaFileUri();
Log.d(LOG, "end get uri = " + fileUri);
Intent i = new Intent(MediaStore.ACTION_IMAGE_CAPTURE);
i.putExtra(MediaStore.EXTRA_OUTPUT, fileUri);
startActivityForResult(i, REQUEST_IMAGE_TAKE);
} else if (options[item].equals("Choose From Gallery")) {
dialog.dismiss();
Intent i = new Intent(Intent.ACTION_PICK, MediaStore.Images.Media.EXTERNAL_CONTENT_URI);
startActivityForResult(i, REQUEST_IMAGE_SELECT);
} else if (options[item].equals("Cancel")) {
dialog.dismiss();
}
}
});
builder.show();
} else
Toast.makeText(this, "Camera Permission error", Toast.LENGTH_SHORT).show();
} catch (Exception e) {
Toast.makeText(this, "Camera Permission error", Toast.LENGTH_SHORT).show();
e.printStackTrace();
}
}

/**
* Create a file Uri for saving an image or video
*/
private Uri getOutputMediaFileUri() {
//return Uri.fromFile(getOutputMediaFile(type));
Log.d(LOG, "authority = " + getPackageName() + ".provider");
Log.d(LOG, "getApplicationContext = " + getApplicationContext());
return FileProvider.getUriForFile(this, getPackageName() + ".fileprovider", getOutputMediaFile());
}

/**
* Create a File for saving an image
*/
private static File getOutputMediaFile() {

File mediaStorageDir = new File(Environment.getExternalStoragePublicDirectory(Environment.DIRECTORY_PICTURES), "LabelDetect");

// Create the storage directory if it does not exist
if (!mediaStorageDir.exists()) {
if (!mediaStorageDir.mkdirs()) {
Log.d(LOG, "failed to create directory");
return null;
}
}
// Create a media file name
String timeStamp = new SimpleDateFormat("yyyyMMdd_HHmmss").format(new Date());
File mediaFile = new File(mediaStorageDir.getPath() + File.separator +
"IMG_" + timeStamp + ".jpg");
Log.d(LOG, "mediaFile " + mediaFile);
return mediaFile;
}

When user select image start detecting.

@override
protected void onActivityResult(int requestCode, int resultCode, Intent data) {
if ((requestCode == REQUEST_IMAGE_TAKE || requestCode == REQUEST_IMAGE_SELECT) && resultCode == RESULT_OK) {
String imgPath;

if (requestCode == REQUEST_IMAGE_TAKE) {
imgPath = Environment.getExternalStorageDirectory() + fileUri.getPath();
} else {
Uri selectedImage = data.getData();
String[] filePathColumn = {MediaStore.Images.Media.DATA};
Cursor cursor = SceneDetectionActivity.this.getContentResolver().query(selectedImage,
filePathColumn, null, null, null);
cursor.moveToFirst();
int columnIndex = cursor.getColumnIndex(filePathColumn[0]);
imgPath = cursor.getString(columnIndex);
cursor.close();
}
Log.d(LOG, "imgPath = " + imgPath);
bmp = BitmapFactory.decodeFile(imgPath);
if (bmp != null) {
//Toast.makeText(this, "Bit map is not null", Toast.LENGTH_SHORT).show();
dialog = ProgressDialog.show(SceneDetectionActivity.this,
"Predicting...", "Wait for one sec...", true);
SceneDetectionAsync async = new SceneDetectionAsync();
async.execute(bmp);

} else {
Toast.makeText(this, "Bit map is null", Toast.LENGTH_SHORT).show();
}

}
super.onActivityResult(requestCode, resultCode, data);
}

Data set

private MyListData[] getPetsList() {
MyListData[] listData = new MyListData[]{
new MyListData("Labrador Retriever", "20000INR", "Age: 1yr", R.drawable.labrador_retriever),
new MyListData("Bengal Cat", "8000INR", "Age: 1 month", R.drawable.bengal_cat),
new MyListData("Parrot", "2500INR", "Age: 3months", R.drawable.parrot),
new MyListData("Rabbit", "1500INR", "Age: 1 month", R.drawable.rabbit_image),
new MyListData("Beagle", "20500INR", "Age:6months", R.drawable.beagle),
new MyListData("Bulldog", "19000INR", "1yr", R.drawable.bulldog),
new MyListData("German Shepherd", "18000INR", "Age: 2yr", R.drawable.german_shepherd_dog),
new MyListData("German Shorthaired Pointer", "20000INR", "Age: 8 months", R.drawable.german_shorthaired_pointer),
new MyListData("Golder retriever", "12000INR", "Age: 7months", R.drawable.golden_retriever),
new MyListData("Pembroke Welsh corgi", "9000INR", "Age: 10months", R.drawable.pembroke_welsh_corgi),
new MyListData("Pomeranian", "25000INR", "Age: 10months", R.drawable.pomeranian),
new MyListData("Poodle", "15000INR", "Age: 3months", R.drawable.poodle),
new MyListData("Rottweiler", "1700INR", "Age:2yr", R.drawable.rottweiler),
new MyListData("Shihtzu", "18000INR", "Age: 5months", R.drawable.shih_tzu),

};

return listData;
}

private MyListData[] getPetsFilteredDataList(JSONObject jsonObject) {
MyListData[] listData = null;
try {
//{"resultCode":0,"scene":"{\"type\":13}"}
String scene = jsonObject.getString("scene");
JSONObject object = new JSONObject(scene);
int type = object.getInt("type");
switch (type) {
case 1:
break;
case 12:
//Get Cats filtered data here
break;
case 13:
listData = getDogsData();
break;
}

} catch (JSONException e) {
e.printStackTrace();
} catch (Exception e) {
e.printStackTrace();
}
return listData;
}

private MyListData[] getDogsData() {
MyListData[] dogsList = new MyListData[]{
new MyListData("Labrador Retriever", "20000INR", "Age: 1yr", R.drawable.labrador_retriever),
new MyListData("Beagle", "20500INR", "Age:6months", R.drawable.beagle),
new MyListData("Bulldog", "19000INR", "1yr", R.drawable.bulldog),
new MyListData("German Shepherd", "18000INR", "Age: 2yr", R.drawable.german_shepherd_dog),
new MyListData("German Shorthaired Pointer", "20000INR", "Age: 8 months", R.drawable.german_shorthaired_pointer),
new MyListData("Golder retriever", "12000INR", "Age: 7months", R.drawable.golden_retriever),
new MyListData("Pembroke Welsh corgi", "9000INR", "Age: 10months", R.drawable.pembroke_welsh_corgi),
new MyListData("Pomeranian", "25000INR", "Age: 10months", R.drawable.pomeranian),
new MyListData("Poodle", "15000INR", "Age: 3months", R.drawable.poodle),
new MyListData("Rottweiler", "1700INR", "Age:2yr", R.drawable.rottweiler),
new MyListData("Shihtzu", "18000INR", "Age: 5months", R.drawable.shih_tzu),

};
return dogsList;
}

Result

Before Filter.

After filter

Tips and Tricks

Check dependencies downloaded properly.

Latest HMS Core APK is required.

Min SDK is 21. Otherwise we get Manifest merge issue.

Run detect() background thread otherwise app will crash with error.

If you are taking image from a camera or gallery make sure your app has camera and storage permission.

Add the downloaded huawei-hiai-vision-ove-10.0.4.307.aar, huawei-hiai-pdk-1.0.0.aar file to libs folder.

If device does not supports you will get 601 code in the result code

Maximum 20MB image

Conclusion

In this article, we have learnt the following concepts.

What is Scene detection?

Features of scene detection

How to integrate scene detection using Huawei HiAI

How to Apply Huawei HiAI

How to build the application

How to filter data by scene

Reference

Scene detection

Apply for Huawei HiAI