[Huawei Developers] Basavaraj.navi:
​

Introduction

In this article, I will explain what is Skeleton detection? How does Skeleton detection work in Flutter? At the end of this tutorial, we will create the Huawei Skeleton detection in Flutter application using Huawei ML Kit.

What is Skeleton detection?

Huawei ML Kit Skeleton detection service detects the human body. So, represents the orientation of a person in a graphical format. Essentially, it’s a set of coordinates that can be connected to describe the position of the person. This service detects and locates key points of the human body such as the top of the head, neck, shoulders, elbows, wrists, hips, knees, and ankles. Currently, full-body and half-body static image recognition and real-time camera stream recognition are supported.

What is the use of Skeleton detection?

Definitely, everyone will have the question like what is the use of it. For example, if you want to develop a fitness application, you can understand and help the user with coordinates from skeleton detection to see if the user has made the exact movements during exercises or you could develop a game about dance movements Using this service and ML kit can understand easily whether the user has done proper excise or not.

How does it work?

You can use skeleton detection over a static image or over a real-time camera stream. Either way, you can get the coordinates of the human body. Of course, when taking them, it’s looking out for critical areas like head, neck, shoulders, elbows, wrists, hips, knees, and ankles. At the same time, both the methods will detect multiple human bodies.

There are two attributes to detect skeleton.

1. TYPE_NORMAL

2. TYPE_YOGA

TYPE_NORMAL: If you send the analyzer type as TYPE_NORMAL, perceives skeletal points for normal standing position.

TYPE_YOGA: If you send the analyzer type as TYPE_YOGA, it picks up skeletal points for yoga posture.

Note: The default mode is to detect skeleton points for normal postures.

Integration of Skeleton Detection

1. Configure the application on the AGC.

2. Client application development process.

Configure application on the AGC

This step involves a couple of steps, as follows.

Step 1: We need to register as a developer account in AppGallery Connect. If you are already a developer ignore this step.

Step 2: Create an app by referring to Creating a Project and Creating an App in the Project

Step 3: Set the data storage location based on the current location.

Step 4: Enabling ML. Open AppGallery connect, choose Manage API > ML Kit

Step 5: Generating a Signing Certificate Fingerprint.

Step 6: Configuring the Signing Certificate Fingerprint.

Step 7: Download your agconnect-services.json file, paste it into the app root directory.

Client application development process

This step involves a couple of steps as follows.

Step 1: Create a flutter application in the Android studio (Any IDE which is your favorite).

Step 2: Add the App level Gradle dependencies. Choose inside project Android > app > build.gradle.

1

2	apply plugin: 'com.android.application'

apply plugin: 'com.huawei.agconnect'

Root level gradle dependencies.

1

2	maven { url 'https://developer.huawei.com/repo/' }

classpath 'com.huawei.agconnect:agcp:1.4.1.300'

Step 3: Add the downloaded plugin in pubspec.yaml.

Step 4: Add a downloaded file into the outside project directory. Declare plugin path in pubspec.yaml file under dependencies.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28	dependencies:

flutter:

sdk: flutter

huawei_account:

path: ../huawei_account/

huawei_location:

path: ../huawei_location/

huawei_map:

path: ../huawei_map/

huawei_analytics:

path: ../huawei_analytics/

huawei_site:

path: ../huawei_site/

huawei_push:

path: ../huawei_push/

huawei_dtm:

path: ../huawei_dtm/

huawei_ml:

path: ../huawei_ml/

agconnect_crash: ^1.0.0

agconnect_remote_config: ^1.0.0

http: ^0.12.2

camera:

path_provider:

path:

image_picker:

fluttertoast: ^7.1.6

shared_preferences: ^0.5.12+4

To achieve the Skeleton detection example, follow the steps.

1. AGC Configuration

2. Build Flutter application

Step 1: AGC Configuration

1. Sign in to AppGallery Connect and select My apps.

2. Select the app in which you want to integrate the Huawei ML kit.

3. Navigate to Project Setting > Manage API > ML Kit

Step 2: Build Flutter application

In this example, I am getting image from gallery or Camera and getting the skeleton detection and joints points from the ML kit skeleton detection.

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

80

81

82

83

84

85

86

87

88

89

90

91

92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128	import 'dart:io';

import 'package:flutter/material.dart';

import 'package:huawei_ml/huawei_ml.dart';

import 'package:huawei_ml/skeleton/ml_skeleton_analyzer.dart';

import 'package:huawei_ml/skeleton/ml_skeleton_analyzer_setting.dart';

import 'package:image_picker/image_picker.dart';

class SkeletonDetection extends StatefulWidget {

@override

_SkeletonDetectionState createState() => _SkeletonDetectionState();

}

class _SkeletonDetectionState extends State<SkeletonDetection> {

MLSkeletonAnalyzer analyzer;

MLSkeletonAnalyzerSetting setting;

List<MLSkeleton> skeletons;

double _x = 0;

double _y = 0;

double _score = 0;

@override

void initState() {

// TODO: implement initState

analyzer = new MLSkeletonAnalyzer();

setting = new MLSkeletonAnalyzerSetting();

super.initState();

}

@override

Widget build(BuildContext context) {

return Scaffold(

body: Center(

child: Column(

mainAxisAlignment: MainAxisAlignment.center,

children: <Widget>[

_setImageView()

],

),

),

floatingActionButton: FloatingActionButton(

onPressed: () {

_showSelectionDialog(context);

},

child: Icon(Icons.camera_alt),

),

);

}

Future<void> _showSelectionDialog(BuildContext context) {

return showDialog(

context: context,

builder: (BuildContext context) {

return AlertDialog(

title: Text("From where do you want to take the photo?"),

content: SingleChildScrollView(

child: ListBody(

children: <Widget>[

GestureDetector(

child: Text("Gallery"),

onTap: () {

_openGallery(context);

},

),

Padding(padding: EdgeInsets.all(8.0)),

GestureDetector(

child: Text("Camera"),

onTap: () {

_openCamera();

},

)

],

),

));

});

}

File imageFile;

void _openGallery(BuildContext context) async {

var picture = await ImagePicker.pickImage(source: ImageSource.gallery);

this.setState(() {

imageFile = picture;

_skeletonDetection();

});

Navigator.of(context).pop();

}

_openCamera() async {

PickedFile pickedFile = await ImagePicker().getImage(

source: ImageSource.camera,

maxWidth: 800,

maxHeight: 800,

);

if (pickedFile != null) {

imageFile = File(pickedFile.path);

this.setState(() {

imageFile = imageFile;

_skeletonDetection();

});

}

Navigator.of(context).pop();

}

Widget _setImageView() {

if (imageFile != null) {

return Image.file(imageFile, width: 500, height: 500);

} else {

return Text("Please select an image");

}

}

_skeletonDetection() async {

// Create a skeleton analyzer.

analyzer = new MLSkeletonAnalyzer();

// Configure the recognition settings.

setting = new MLSkeletonAnalyzerSetting();

setting.path = imageFile.path;

setting.analyzerType = MLSkeletonAnalyzerSetting.TYPE_NORMAL; // Normal posture.

// Get recognition result asynchronously.

List<MLSkeleton> list = await analyzer.asyncSkeletonDetection(setting);

print("Result data: "+list[0].toJson().toString());

// After the recognition ends, stop the analyzer.

bool res = await analyzer.stopSkeletonDetection();

}

}

Result

Tips and Tricks

Download the latest HMS Flutter plugin.

Check dependencies downloaded properly.

Latest HMS Core APK is required.

If you are taking image from a camera or gallery make sure your app has camera and storage permission

Conclusion

In this article, we have learned the integration of the Huawei ML kit, and what is skeleton detection, how it works, what is the use of it, how to get the Joints point from the skeleton detection, types of detections like TYPE_NORMAL and TYPE_YOGA.

Reference

Skeleton Detection

Happy coding