[Huawei Developers] Basavaraj.navi:
Introduction

In this article, we will learn how to integrate Huawei Video summarization using Huawei HiAI. We will build the Video preview maker application to share it on social media to increase your video views.

What is Video summarization?

In general Video summarization is the process of distilling a raw video into a more compact form without losing much information.

This Service can generate a 10 seconds, 15 seconds, or 30 seconds video summary of a single video or multiple videos containing the original voice.

Implementing an advanced multi-dimensional scoring framework, the aesthetic engine assists with shooting, photo selection, video editing, and video splitting, by comprehending complex subjective aspects in images, and making high-level judgments related to the attractiveness, memorability and engaging nature of images.

Features

Fast: This algorithm is currently developed based on the deep neural network, to fully utilize the neural processing unit (NPU) of Huawei mobile phones to accelerate the neural network, achieving an acceleration of over 10 times.

Lightweight: This API greatly reduces the computing time and ROM space the algorithm model takes up, making your app more lightweight.

Comprehensive scoring: The aesthetic engine provides scoring to measure image quality from objective dimensions (image quality), subjective dimensions (sensory evaluation), and photographic dimensions (rule evaluation).

Portrait aesthetics scoring: An industry-leading portrait aesthetics scoring feature obtains semantic information about human bodies in the image, including the number of people, individual body builds, positions, postures, facial positions and angles, eye movements, mouth movements, and facial expressions. Aesthetic scores of the portrait are given according to the various types of the body semantic information.

How to integrate Video Summarization

Configure the application on the AGC.

Apply for HiAI Engine Library

Client application development process.

Configure application on the AGC

Follow the steps

Step 1: We need to register as a developer account in AppGallery Connect. If you are already a developer ignore this step.

Step 2: Create an app by referring to Creating a Project and Creating an App in the Project

Step 3: Set the data storage location based on the current location.

Step 4: Generating a Signing Certificate Fingerprint.

Step 5: Configuring the Signing Certificate Fingerprint.

Step 6: Download your agconnect-services.json file, paste it into the app root directory.

Apply for HiAI Engine Library

What is Huawei HiAI?

HiAI is Huawei’s AI computing platform. HUAWEI HiAI is a mobile terminal–oriented artificial intelligence (AI) computing platform that constructs three layers of ecology: service capability openness, application capability openness, and chip capability openness. The three-layer open platform that integrates terminals, chips, and the cloud brings more extraordinary experience for users and developers.

How to apply for HiAI Engine?

Follow the steps

Step 1: Navigate to this URL, choose App Service > Development and click HUAWEI HiAI.

Step 2: Click Apply for HUAWEI HiAI kit.

Step 3: Enter required information like Product name and Package name, click Next button.

Step 4: Verify the application details and click Submit button.

Step 5: Click the Download SDK button to open the SDK list.

Step 6: Unzip downloaded SDK and add into your android project under libs folder.

Step 7: Add jar files dependences into app build.gradle file.

implementation fileTree(include: ['*.aar', '*.jar'], dir: 'libs')
implementation 'com.google.code.gson:gson:2.8.6'
repositories {
flatDir {
dirs 'libs'
}
}

Client application development process

Follow the steps

Step 1: Create an Android application in the Android studio (Any IDE which is your favorite).

Step 2: Add the App level Gradle dependencies. Choose inside project Android > app > build.gradle.

apply plugin: 'com.android.application'
apply plugin: 'com.huawei.agconnect'

Root level gradle dependencies.

maven { url 'https://developer.huawei.com/repo/' }
classpath 'com.huawei.agconnect:agcp:1.4.1.300'

Step 3: Add permission in AndroidManifest.xml

<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
<!-- CAMERA -->
<uses-permission android:name="android.permission.CAMERA" />
<uses-feature android:name="android.hardware.camera" />
<uses-feature android:name="android.hardware.camera.autofocus" />

Step 4: Build application.

First request run time permission

private void requestPermissions() {
try {
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.M) {
int permission = ActivityCompat.checkSelfPermission(this,
Manifest.permission.WRITE_EXTERNAL_STORAGE);
if (permission != PackageManager.PERMISSION_GRANTED) {
ActivityCompat.requestPermissions(this, new String[]{Manifest.permission.WRITE_EXTERNAL_STORAGE,
Manifest.permission.READ_EXTERNAL_STORAGE, Manifest.permission.CAMERA}, 0x0010);
}
}
} catch (Exception e) {
e.printStackTrace();
}
}

Initialize vision base

private void initVisionBase() {
VisionBase.init(VideoSummaryActivity.this, new ConnectionCallback() {
[USER=439709]@override[/USER]
public void onServiceConnect() {
Log.i(LOG, "onServiceConnect ");
Toast.makeText(VideoSummaryActivity.this, "Service Connected", Toast.LENGTH_SHORT).show();
}
[USER=439709]@override[/USER]
public void onServiceDisconnect() {
Log.i(LOG, "onServiceDisconnect");
Toast.makeText(VideoSummaryActivity.this, "Service Disconnected", Toast.LENGTH_SHORT).show();
}
});
}

Create video Async class

public class VideoAsyncTask extends AsyncTask<String, Void, String> {
private static final String LOG = VideoAsyncTask.class.getSimpleName();
private Context context;
private VideoCoverListener listener;
private AestheticsScoreDetector aestheticsScoreDetector;

public VideoAsyncTask(Context context, VideoCoverListener listener) {
this.context = context;
this.listener = listener;
}

[USER=439709]@override[/USER]
protected String doInBackground(String... paths) {
Log.i(LOG, "init VisionBase");
VisionBase.init(context, ConnectManager.getInstance().getmConnectionCallback()); //try to start AIEngine
if (!ConnectManager.getInstance().isConnected()) { //wait for AIEngine service
ConnectManager.getInstance().waitConnect();
}
Log.i(LOG, "init videoCover");
aestheticsScoreDetector = new AestheticsScoreDetector(context);
AEModelConfiguration aeModelConfiguration;
aeModelConfiguration = new AEModelConfiguration();
aeModelConfiguration.getSummerizationConf().setSummerizationMaxLen(10);
aeModelConfiguration.getSummerizationConf().setSummerizationMinLen(2);
aestheticsScoreDetector.setAEModelConfiguration(aeModelConfiguration);
String videoResult = null;
if (listener.isAsync()) {
videoCoverAsync(paths);
videoResult = "-10000";
} else {
videoResult = videoCover(paths);
aestheticsScoreDetector.release();
}
//release engine after detect finished
return videoResult;
}

[USER=439709]@override[/USER]
protected void onPostExecute(String resultScore) {
if (!resultScore.equals("-10000")) {
listener.onTaskCompleted(resultScore, false);
}
super.onPostExecute(String.valueOf(resultScore));
}

private String videoCover(String[] videoPaths) {
if (videoPaths == null) {
Log.e(LOG, "uri is null ");
return null;
}
JSONObject jsonObject = new JSONObject();
int position = 0;
Video[] videos = new Video[videoPaths.length];
for (String path : videoPaths) {
Video video = new Video();
video.setPath(path);
videos[position++] = video;
}
jsonObject = aestheticsScoreDetector.getVideoSummerization(videos, null);
if (jsonObject == null) {
Log.e(LOG, "return JSONObject is null");
return "return JSONObject is null";
}
if (!jsonObject.optString("resultCode").equals("0")) {
Log.e(LOG, "return JSONObject resultCode is not 0");
return jsonObject.optString("resultCode");
}
Log.d(LOG, "videoCover get score end");
AEVideoResult aeVideoResult = aestheticsScoreDetector.convertVideoSummaryResult(jsonObject);
if (null == aeVideoResult) {
Log.e(LOG, "aeVideoResult is null ");
return null;
}
String result = new Gson().toJson(aeVideoResult, AEVideoResult.class);
return result;
}

private void videoCoverAsync(String[] videoPaths) {
if (videoPaths == null) {
Log.e(LOG, "uri is null ");
return;
}
Log.d(LOG, "runVisionService " + "start get score");
CVisionCallback callback = new CVisionCallback();

int position = 0;
Video[] videos = new Video[videoPaths.length];
for (String path : videoPaths) {
Video video = new Video();
video.setPath(path);
videos[position++] = video;
}
aestheticsScoreDetector.getVideoSummerization(videos, callback);
}

public class CVisionCallback extends VisionCallback {

[USER=439709]@override[/USER]
public void setRequestID(String requestID) {
super.setRequestID(requestID);
}

[USER=439709]@override[/USER]
public void onDetectedResult(AnnotateResult annotateResult) throws RemoteException {
if (annotateResult != null) {
Log.e("Visioncallback", annotateResult.toString());
}
Log.e("Visioncallback", annotateResult.getResult().toString());
JSONObject jsonObject = null;
try {
jsonObject = new JSONObject(annotateResult.getResult().toString());
} catch (JSONException e) {
e.printStackTrace();
}
if (jsonObject == null) {
Log.e(LOG, "return JSONObject is null");
aestheticsScoreDetector.release();
return;
}
if (!jsonObject.optString("resultCode").equals("0")) {
Log.e(LOG, "return JSONObject resultCode is not 0");
aestheticsScoreDetector.release();
return;
}
AEVideoResult aeVideoResult = aestheticsScoreDetector.convertVideoSummaryResult(jsonObject);
if (aeVideoResult == null) {
aestheticsScoreDetector.release();
return;
}
String result = new Gson().toJson(aeVideoResult, AEVideoResult.class);
aestheticsScoreDetector.release();
listener.onTaskCompleted(result, true);
}

[USER=439709]@override[/USER]
public void onDetectedInfo(InfoResult infoResult) throws RemoteException {
JSONObject jsonObject = null;
try {
jsonObject = new JSONObject(infoResult.getInfoResult().toString());
AEDetectVideoStatus aeDetectVideoStatus = aestheticsScoreDetector.convertDetectVideoStatusResult(jsonObject);
if (aeDetectVideoStatus != null) {
listener.updateProcessProgress(aeDetectVideoStatus);
} else {
Log.d(LOG, "[ASTaskPlus onDetectedInfo]aeDetectVideoStatus result is null!");
}
} catch (JSONException e) {
e.printStackTrace();
}
}

[USER=439709]@override[/USER]
public void onDetectedError(ErrorResult errorResult) throws RemoteException {
Log.e(LOG, errorResult.getResultCode() + "");
aestheticsScoreDetector.release();
listener.onTaskCompleted(String.valueOf(errorResult.getResultCode()), true);
}

[USER=439709]@override[/USER]
public String getRequestID() throws RemoteException {
return null;
}
}
}

Result

Tips and Tricks

Check dependencies added properly

Latest HMS Core APK is required.

Min SDK is 21. Otherwise you will get Manifest merge issue.

If you are taking Video from a camera or gallery make sure your app has camera and storage permission.

Add the downloaded huawei-hiai-vision-ove-10.0.4.307.aar, huawei-hiai-pdk-1.0.0.aar file to libs folder.

Maximum video length is 10 minute.

Resolution should be between 144P and 2160P.

Conclusion

In this article, we have learnt the following concepts.

What is Video summarization?

Features of Video summarization

How to integrate Video summarization using Huawei HiAI

How to Apply Huawei HiAI

How to build the application

Reference

Video summarization

Apply for Huawei HiAI

Happy coding