[Huawei Developers] muraliameakula:
Introduction

In this article, we will learn how to implement Image Segmentation feature in flutter application. Using this we can segments same elements such as human body, plant and sky from an image. We can use in different scenarios, it can be used in photography apps to apply background.

About Image Segmentation

Image Segmentation allows developers two types of segmentation Human body and multiclass segmentation. We can apply image segmentation on static images and video streams if we select human body type. But we can only apply segmentation for static images in multiclass segmentation.

Huawei ML Kit’s Image Segmentation service divides same elements (such as human body, plant and sky) from an image. The elements supported includes human body, sky, plant, food, cat, dog, flower, water, sand, building, mountain, and others. By the way, Huawei ML Kit works on all Android phones with ARM architecture and as it’s device-side capability is free.

The result of human body segmentation includes the coordinate array of the human body, human body image with a transparent background, and gray-scale image with a white human body and black background.

Requirements

1. Any operating system (MacOS, Linux and Windows etc.)

2. Any IDE with Flutter SDK installed (IntelliJ, Android Studio and VsCode etc.)

3. Minimum API Level 19 is required.

4. Required EMUI 5.0 and later version devices.

Setting up the ML kit

1. First create a developer account in AppGallery Connect. After create your developer account, you can create a new project and new app. For more information, click here.

2. Enable the ML kit in the Manage API section and add the plugin.

3. Add the required dependencies to the build.gradle file under root folder.

maven {url'http://developer.huawei.com/repo/'}
classpath 'com.huawei.agconnect:agcp:1.4.1.300'

4. Add the required permissions to the AndroidManifest.xml file under app/src/main folder.

<uses-permission android:name="android.permission.CAMERA" />
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />

5. After completing all the above steps, you need to add the required kits’ Flutter plugins as dependencies to pubspec.yaml file. Refer this URL for cross-platform plugins to download the latest versions.

huawei_ml:
path: ../huawei_ml/

6. Do not forget to add the following meta-data tags in your AndroidManifest.xml. This is for automatic update of the machine learning model.

<application
... >

<meta-data
android:name="com.huawei.hms.ml.DEPENDENCY"
android:value= "imgseg"/>

</application>

After adding them, run flutter pub get command. Now all the plugins are ready to use.

Note: Set multiDexEnabled to true in the android/app directory, so the app will not crash.

Code Integration

We need to initialize the analyzer with some settings. If we want to identify only the human body, then we need to use MLImageSegmentationSetting.BODY_SEG constant.

class ImageSegmentation extends StatefulWidget {
@override
ImageSegmentationState createState() => ImageSegmentationState();
}

class ImageSegmentationState extends State<ImageSegmentation> {
MLImageSegmentationAnalyzer analyzer;
MLImageSegmentationAnalyzerSetting setting;
List<MLImageSegmentation> result;
PickedFile _pickedFile;
File _imageFile;
File _imageFile1;
String _imagePath;
String _imagePath1;
String _foregroundUri = "Foreground Uri";
String _grayscaleUri = "Grayscale Uri";
String _originalUri = "Original Uri";

@override
void initState() {
analyzer = new MLImageSegmentationAnalyzer();
setting = new MLImageSegmentationAnalyzerSetting();
_checkCameraPermissions();
super.initState();
}

_checkCameraPermissions() async {
if (await MLPermissionClient().checkCameraPermission()) {
Scaffold.of(context).showSnackBar(SnackBar(
content: Text("Permission Granted"),
));
} else {
await MLPermissionClient().requestCameraPermission();
}
}

@override
Widget build(BuildContext context) {
return Scaffold(
body: Column(
children: [
SizedBox(height: 15),
Container(
padding: EdgeInsets.all(16.0),
child: Column(
children: [
_setImageView(_imageFile),
SizedBox(width: 15),
_setImageView(_imageFile1),
SizedBox(width: 15),
],
)),
// SizedBox(height: 15),
// _setText(),
SizedBox(height: 15),
_showImagePickingOptions(),
],
));
}

Widget _showImagePickingOptions() {
return Expanded(
child: Align(
child: Column(
mainAxisAlignment: MainAxisAlignment.center,
children: [
Container(
margin: EdgeInsets.only(left: 20.0, right: 20.0),
width: MediaQuery.of(context).size.width,
child: MaterialButton(
color: Colors.amber,
textColor: Colors.white,
child: Text("TAKE PICTURE"),
onPressed: () async {
final String path = await getImage(ImageSource.camera);
_startRecognition(path);
})),
Container(
width: MediaQuery.of(context).size.width,
margin: EdgeInsets.only(left: 20.0, right: 20.0),
child: MaterialButton(
color: Colors.amber,
textColor: Colors.white,
child: Text("PICK FROM GALLERY"),
onPressed: () async {
final String path = await getImage(ImageSource.gallery);
_startRecognition(path);
})),
],
),
),
);
}

Widget _setImageView(File imageFile) {
if (imageFile != null) {
return Image.file(imageFile, width: 200, height: 200);
} else {
return Text(" ");
}
}

_startRecognition(String path) async {
setting.path = path;
setting.analyzerType = MLImageSegmentationAnalyzerSetting.BODY_SEG;
setting.scene = MLImageSegmentationAnalyzerSetting.ALL;
try {
result = await analyzer.analyzeFrame(setting);
_foregroundUri = result.first.foregroundUri;
_grayscaleUri = result.first.grayscaleUri;
_originalUri = result.first.originalUri;
_imagePath = await FlutterAbsolutePath.getAbsolutePath(_grayscaleUri);
_imagePath1 = await FlutterAbsolutePath.getAbsolutePath(_originalUri);
setState(() {
_imageFile = File(_imagePath);
_imageFile1 = File(_imagePath1);
});
} on Exception catch (e) {
print(e.toString());
}
}

Future<String> getImage(ImageSource imageSource) async {
final picker = ImagePicker();
_pickedFile = await picker.getImage(source: imageSource);
return _pickedFile.path;
}
}

Demo

​

Tips and Tricks

1. Download latest HMS Flutter plugin.

2. Set minimum SDK version to 23 or later.

3. Do not forget to add Camera permission in Manifest file.

4. Latest HMS Core APK is required.

Conclusion

That’s it!

In this article, we have learnt how to use image segmentation. We can get human body related pixels out of our image and changing background. Here we implemented transparent background and gray-scale image with a white human body and black background.

Thanks for reading! If you enjoyed this story, please click the Like button and Follow. Feel free to leave a Comment 💬below.

Reference

ML kit URL

Original Source