[Huawei Developers] HMS Community:
Overview

In this article, I will create a Doctor Consult Demo App along with the integration of ML Product Visual Search Api. Which provides an easy interface to consult with doctor. Users can scan their prescriptions using Product Visual Search Api.

Previous Articles Link:

https://forums.developer.huawei.com/forumPortal/en/topic/0201829733289720014?fid=0101187876626530001

https://forums.developer.huawei.com/forumPortal/en/topic/0201817617825540005?fid=0101187876626530001

https://forums.developer.huawei.com/forumPortal/en/topic/0201811543541800017?fid=0101187876626530001

HMS Core Map Service Introduction

ML Kit allows your apps to easily leverage Huawei's long-term proven expertise in machine learning to support diverse artificial intelligence (AI) applications throughout a wide range of industries. Thanks to Huawei's technology accumulation, ML Kit provides diversified leading machine learning capabilities that are easy to use, helping you develop various AI apps.

Product Visual Search: This service searches for the same or similar products in the pre-established product image library based on a product photo taken by a user, and returns the IDs of those products and related information. In addition, to better manage products in real time, this service supports offline product import, online product adding, deletion, modification, and query, and product distribution.

Prerequisite

Huawei Phone EMUI 3.0 or later.

Non-Huawei phones Android 4.4 or later (API level 19 or higher).

Android Studio.

AppGallery Account.

App Gallery Integration process

Sign In and Create or Choose a project on AppGallery Connect portal.

Navigate to Project settings and download the configuration file.

Navigate to General Information, and then provide Data Storage location.

App Development

Create A New Project.

Configure Project Gradle.

Configure App Gradle.

Configure AndroidManifest.xml.

Create Activity class with XML UI.

package com.hms.doctorconsultdemo.ml;

import android.Manifest;
import android.content.Intent;
import android.content.pm.PackageManager;
import android.graphics.Bitmap;
import android.os.Bundle;
import android.provider.MediaStore;
import android.util.Log;

import androidx.annotation.Nullable;
import androidx.core.app.ActivityCompat;

import com.huawei.hmf.tasks.Task;
import com.huawei.hms.mlsdk.MLAnalyzerFactory;
import com.huawei.hms.mlsdk.common.MLException;
import com.huawei.hms.mlsdk.common.MLFrame;
import com.huawei.hms.mlsdk.productvisionsearch.MLProductVisionSearch;
import com.huawei.hms.mlsdk.productvisionsearch.MLVisionSearchProduct;
import com.huawei.hms.mlsdk.productvisionsearch.MLVisionSearchProductImage;
import com.huawei.hms.mlsdk.productvisionsearch.cloud.MLRemoteProductVisionSearchAnalyzer;
import com.huawei.hms.mlsdk.productvisionsearch.cloud.MLRemoteProductVisionSearchAnalyzerSetting;

import java.util.ArrayList;
import java.util.List;

public class ScanMainActivity extends BaseActivity {

private static final String TAG = ScanMainActivity.class.getName();
private static final int CAMERA_PERMISSION_CODE = 100;

MLRemoteProductVisionSearchAnalyzer analyzer;

@Override
protected void onCreate(@Nullable Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
init();
initializeProductVisionSearch();
}

private void init() {
if (!(ActivityCompat.checkSelfPermission(this, Manifest.permission.CAMERA)
== PackageManager.PERMISSION_GRANTED)) {
this.requestCameraPermission();
}
initializeProductVisionSearch();
Intent intent = new Intent(MediaStore.ACTION_IMAGE_CAPTURE);
startActivityForResult(intent, 101);
}

private void requestCameraPermission() {
final String[] permissions = new String[]{Manifest.permission.CAMERA};

if (!ActivityCompat.shouldShowRequestPermissionRationale(this, Manifest.permission.CAMERA)) {
ActivityCompat.requestPermissions(this, permissions, this.CAMERA_PERMISSION_CODE);
return;
}
}

@Override
protected void onActivityResult(int requestCode, int resultCode, @Nullable Intent data) {
super.onActivityResult(requestCode, resultCode, data);
if (requestCode == 101) {
if (resultCode == RESULT_OK) {
Bitmap bitmap = (Bitmap) data.getExtras().get("data");
if (bitmap != null) {
MLFrame mlFrame = new MLFrame.Creator().setBitmap(bitmap).create();
mlImageDetection(mlFrame);
}

}

}
}

private void mlImageDetection(MLFrame mlFrame) {

Task<List<MLProductVisionSearch>> task = analyzer.asyncAnalyseFrame(mlFrame);
task.addOnSuccessListener(products -> {
Log.d(TAG, "success");
displaySuccess(products);
})
.addOnFailureListener(e -> {
try {
MLException mlException = (MLException) e;
int errorCode = mlException.getErrCode();
String errorMessage = mlException.getMessage();
} catch (Exception error) {
// Handle the conversion error.
}
});
}

private void initializeProductVisionSearch() {
MLRemoteProductVisionSearchAnalyzerSetting settings = new MLRemoteProductVisionSearchAnalyzerSetting.Factory()
.setLargestNumOfReturns(16)
.setRegion(MLRemoteProductVisionSearchAnalyzerSetting.REGION_DR_CHINA)
.create();
analyzer
= MLAnalyzerFactory.getInstance().getRemoteProductVisionSearchAnalyzer(settings);
}

private void displaySuccess(List<MLProductVisionSearch> productVisionSearchList) {
List<MLVisionSearchProductImage> productImageList = new ArrayList<>();
String prodcutType = "";
for (MLProductVisionSearch productVisionSearch : productVisionSearchList) {
Log.d(TAG, "type: " + productVisionSearch.getType());
prodcutType = productVisionSearch.getType();
for (MLVisionSearchProduct product : productVisionSearch.getProductList()) {
productImageList.addAll(product.getImageList());
Log.d(TAG, "custom content: " + product.getCustomContent());
}
}
StringBuffer buffer = new StringBuffer();
for (MLVisionSearchProductImage productImage : productImageList) {
String str = "ProductID: " + productImage.getProductId() + "\nImageID: " + productImage.getImageId() + "\nPossibility: " + productImage.getPossibility();
buffer.append(str);
buffer.append("\n");
}
Log.d(TAG, "display success: " + buffer.toString());
ScanDataActivity.start(this, productImageList);
}

}

App Build Result

View attachment 5574911

Tips and Tricks

Images in PNG, JPG, JPEG, and BMP formats are supported. GIF images are not supported.

ML Kit complies with GDPR requirements for data processing.

Face detection requires Android phones with the Arm architecture.

Conclusion

In this article, we have learned how to integrate HMS ML Kit using Product Visual Search Api in Android application. After completely read this article user can easily implement HMS ML Kit using Product Visual Search Api. So that Users can scan their prescriptions using Product Visual Search Api.

Thanks for reading this article. Be sure to like and comment to this article, if you found it helpful. It means a lot to me.

References

HMS ML Docs:

https://developer.huawei.com/consum...-Guides/service-introduction-0000001050040017

HMS Training Videos -

https://developer.huawei.com/consumer/en/training/