[Huawei Developers] Basavaraj.navi:
Introduction

In this article, I will cover live yoga pose detection. In my last article, I’ve written yoga pose detection using the Huawei ML kit. If you have not read my previous article refer to link Beginner: Find yoga pose using Huawei ML kit skeleton detection - Part 1.

In this article, I will cover live yoga detection.

Definitely, you will have a question about how does this application help?

Let’s take an example, most people attend yoga classes due to COVID-19 nobody is able to attend the yoga classes. So using the Huawei ML kit Skeleton detection record your yoga session video and send it to your yoga master he will check your body joints which is shown in the video. And he will explain what are the mistakes you have done in that recorded yoga session.

Integration of Skeleton Detection

1. Configure the application on the AGC.

2. Client application development process.

Configure application on the AGC

Follow the steps.

Step 1: We need to register as a developer account in AppGallery Connect. If you are already a developer ignore this step.

Step 2: Create an app by referring to Creating a Project and Creating an App in the Project

Step 3: Set the data storage location based on the current location.

Step 4: Enabling ML. Open AppGallery connect, choose Manage API > ML Kit.

Step 5: Generating a Signing Certificate Fingerprint.

Step 6: Configuring the Signing Certificate Fingerprint.

Step 7: Download your agconnect-services.json file, paste it into the app root directory.

Client application development process

Follow the steps.

Step 1: Create an Android application in the Android studio(Any IDE which is your favorite).

Step 2: Add the App level Gradle dependencies. Choose inside project Android > app > build.gradle.

1

2	apply plugin: 'com.android.application'

apply plugin: 'com.huawei.agconnect'

Root level gradle dependencies.

1

2	maven { url 'https://developer.huawei.com/repo/' }

classpath 'com.huawei.agconnect:agcp:1.4.1.300'

Step 3: Add the dependencies in build.gradle

1

2

3	implementation 'com.huawei.hms:ml-computer-vision-skeleton:2.0.4.300'

implementation 'com.huawei.hms:ml-computer-vision-skeleton-model:2.0.4.300'

implementation 'com.huawei.hms:ml-computer-vision-yoga-model:2.0.4.300'

To achieve the Skeleton detection example, follow the steps.

1. AGC Configuration

2. Build Android application

Step 1: AGC Configuration

1. Sign in to AppGallery Connect and select My apps.

2. Select the app in which you want to integrate the Huawei ML kit.

3. Navigate to Project Setting> Manage API > ML Kit

Step 2: Build Android application

In this example, I’m detecting yoga poses live using the camera.

While building the application, follow the steps.

Step 1: Create a Skeleton analyzer.

1

2	private static MLSkeletonAnalyzer analyzer = null;

analyzer = MLSkeletonAnalyzerFactory.getInstance().skeletonAnalyzer

Step 2: Create SkeletonAnalyzerTransactor class to process the result.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65	import android.app.Activity

import android.util.Log

import app.dtse.hmsskeletondetection.demo.utils.SkeletonUtils

import app.dtse.hmsskeletondetection.demo.views.graphic.SkeletonGraphic

import app.dtse.hmsskeletondetection.demo.views.overlay.GraphicOverlay

import com.huawei.hms.mlsdk.common.LensEngine

import com.huawei.hms.mlsdk.common.MLAnalyzer

import com.huawei.hms.mlsdk.common.MLAnalyzer.MLTransactor

import com.huawei.hms.mlsdk.skeleton.MLSkeleton

import com.huawei.hms.mlsdk.skeleton.MLSkeletonAnalyzer

import java.util.*

class SkeletonTransactor(

private val analyzer: MLSkeletonAnalyzer,

private val graphicOverlay: GraphicOverlay,

private val lensEngine: LensEngine,

private val activity: Activity?

) : MLTransactor<MLSkeleton?> {

private val templateList: List<MLSkeleton>

private var zeroCount = 0

override fun transactResult(results: MLAnalyzer.Result<MLSkeleton?>) {

Log.e(TAG, "detect success")

graphicOverlay.clear()

val items = results.analyseList

val resultsList: MutableList<MLSkeleton?> = ArrayList()

for (i in 0 until items.size()) {

resultsList.add(items.valueAt(i))

}

if (resultsList.size <= 0) {

return

}

val similarity = 0.8f

val skeletonGraphic = SkeletonGraphic(graphicOverlay, resultsList)

graphicOverlay.addGraphic(skeletonGraphic)

graphicOverlay.postInvalidate()

val result = analyzer.caluteSimilarity(resultsList, templateList)

if (result >= similarity) {

zeroCount = if (zeroCount > 0) {

return

} else {

0

}

zeroCount++

} else {

zeroCount = 0

return

}

lensEngine.photograph(null, { bytes ->

SkeletonUtils.takePictureListener.picture(bytes)

activity?.finish()

})

}

override fun destroy() {

Log.e(TAG, "detect fail")

}

companion object {

private const val TAG = "SkeletonTransactor"

}

init {

templateList = SkeletonUtils.getTemplateData()

}

}

Step 3: Set Detection Result Processor to Bind the Analyzer.

1	analyzer!!.setTransactor(SkeletonTransactor(analyzer!!, overlay!!, lensEngine!!, activity))

Step 4: Create LensEngine.

1

2

3

4

5

6	lensEngine = LensEngine.Creator(context, analyzer)

.setLensType(LensEngine.BACK_LENS)

.applyDisplayDimension(1280, 720)

.applyFps(20.0f)

.enableAutomaticFocus(true)

.create()

Step 5: Open the camera.

Step 6: Release resources.

1

2

3

4

5

6

7

8

9

10

11	if (lensEngine != null) {

lensEngine!!.close()

}if (lensEngine != null) {

lensEngine!!.release()

}if (analyzer != null) {

try {

analyzer!!.stop()

} catch (e: IOException) {

Log.e(TAG, "e=" + e.message)

}

}

Result

Tips and Tricks

Check dependencies downloaded properly.

Latest HMS Core APK is required.

If you are taking an image from a camera or gallery make sure the app has camera and storage permission.

Conclusion

In this article, we have learned the integration of the Huawei ML kit, and what is skeleton detection, how it works, what is the use of it, how to get the Joints point from the skeleton detection, types of detections like TYPE_NORMALand TYPE_YOGA.

Reference

Skeleton Detection

Happy coding