[Huawei Developers] John Wo:
Introduction

Black Friday is coming. Have you already piled the products you want to buy into your online shopping cart? What if you're feeling a little more frugal and want to save your hard-earned cash instead? Fortunately, there is Crazy Shopping Cart, a fast-paced and addictive virtual shopping game that integrates HUAWEI ML Kit's hand keypoint detection service to allow you to control a virtual shopping cart using hand gestures. In this post, I'll show you how the game's hand gesture control feature was developed using ML Kit.

How to Play

Players perform hand gestures to control the left-right movement of a shopping cart and collect as many items as possible. The cart speeds up every 15 seconds.

Looks fun right? Let's take a look at how the game's hand gesture control system is developed.

Development Process

1. Configure the Maven repository address.

Open the build.gradle file in the root directory of your Android Studio project.

buildscript {
repositories {
google()
jcenter()
maven {url 'https://developer.huawei.com/repo/'}
}
dependencies {
...
classpath 'com.huawei.agconnect:agcp:1.4.1.300'
}
}

allprojects {
repositories {
google()
jcenter()
maven {url 'https://developer.huawei.com/repo/'}
}
}

2. Perform integration in full SDK mode.

dependencies{
// Import the base SDK.
implementation 'com.huawei.hms:ml-computer-vision-handkeypoint:2.0.4.300'
// Import the hand keypoint detection model package.
implementation 'com.huawei.hms:ml-computer-vision-handkeypoint-model:2.0.4.300'
}

After integrating the SDK, add the following configuration to the file header:

Add apply plugin: 'com.huawei.agconnect' after apply plugin: 'com.android.application'.

3. Create a hand keypoint analyzer.

MLHandKeypointAnalyzer analyzer =MLHandKeypointAnalyzerFactory.getInstance().getHandKeypointAnalyzer();

4. Create the detection result processing class HandKeypointTransactor.

public class HandKeypointTransactor implements MLAnalyzer.MLTransactor<List<MLHandKeypoints>> {
@Override
public void transactResult(MLAnalyzer.Result<List<MLHandKeypoints>> results) {
SparseArray<List<MLHandKeypoints>> analyseList = results.getAnalyseList();
// Process the detection result as required.
// Other detection-related APIs provided by ML Kit cannot be called.
}
@Override
public void destroy() {
// Callback method used to release resources when detection completes.
}
}

5. Set the detection result processor, and bind it to the analyzer.

analyzer.setTransactor(new HandKeypointTransactor());

6. Create a LensEngine instance.

LensEngine lensEngine = new LensEngine.Creator(getApplicationContext(), analyzer)
.setLensType(LensEngine.BACK_LENS)
.applyDisplayDimension(1280, 720)
.applyFps(20.0f)
.enableAutomaticFocus(true)
.create();

7. Call the run method to start the camera and read video streams for detection.

// Implement other logic of the SurfaceView control by yourself.
SurfaceView mSurfaceView = findViewById(R.id.surface_view);
try {
lensEngine.run(mSurfaceView.getHolder());
} catch (IOException e) {
// Handle exceptions.
}

8. After the detection is complete, stop the analyzer, and release detection resources.

if (analyzer != null) {
analyzer.stop();
}
if (lensEngine != null) {
lensEngine.release();
}

Summary

As you can see, the development process is really simple and fast. In addition to adding hand gesture control to your mobile games, ML Kit's hand keypoint detection service also has many other useful applications. For example, users can add cute or funny special effects to their videos when using short-video apps with this service integrated. Also, users of smart home apps with this service integrated can customize the hand gestures they use to remotely control appliances.

Try the service out for yourself now by integrating HUAWEI ML Kit with your own apps.

Reference

Official website of Huawei Developers

Development Guide

HMS Core official community on Reddit

Demo and sample code

Discussions on Stack Overflow