[Huawei Developers] muraliameakula:
For some apps, it's necessary to have a function called sound detection that can recognize sounds like knocks on the door, rings of the doorbell, and car horns. Developing such a function can be costly for small- and medium-sized developers, so what should they do in this situation?

There's no need to worry about if you have the sound detection service in HUAWEI ML Kit. Integrating its SDK into your app is simple, and you can equip it with the sound detection function that can work well even when the device does not connect to the network.

Introduction to Sound Detection in HUAWEI ML Kit

This service can detect sound events online by real-time recording. The detected sound events can help you perform subsequent actions. Currently, the following types of sound events are supported: laughter, child crying, snoring, sneezing, shouting, cat meowing, dog barking, running water (such as from taps, streams, and ocean waves), car horns, doorbells, knocking on doors, fire alarms (including smoke alarms), and sirens (such as those from fire trucks, ambulances, police cars, and air defenses).

Preparations

Configuring the Development Environment

Create an app in AppGallery Connect.

For details, see Getting Started with Android.

Enable ML Kit.

Click here to get more information.

After the app is created, an agconnect-services.json file will be automatically generated. Download it and copy it to the root directory of your project.

Configure the Huawei Maven repository address.

To learn more, click here.

Integrate the sound detection SDK.

It is recommended to integrate the SDK in full SDK mode. Add build dependencies for the SDK in the app-level build.gradle file.

// Import the sound detection package.
implementation 'com.huawei.hms:ml-speech-semantics-sounddect-sdk:2.1.0.300'
implementation 'com.huawei.hms:ml-speech-semantics-sounddect-model:2.1.0.300'

Add the AppGallery Connect plugin configuration as needed using either of the following methods:

Method 1: Add the following information under the declaration in the file header:

apply plugin: 'com.android.application'
apply plugin: 'com.huawei.agconnect'

Method 2: Add the plugin configuration in the plugins block:

plugins {
id 'com.android.application'
id 'com.huawei.agconnect'
}

Automatically update the machine learning model.

Add the following statements to the AndroidManifest.xml file. After a user installs your app from HUAWEI AppGallery, the machine learning model is automatically updated to the user's device.

<meta-data
android:name="com.huawei.hms.ml.DEPENDENCY"
android:value= "sounddect"/>

For details, go to Integrating the Sound Detection SDK.

Development Procedure​

Obtain the microphone permission. If the app does not have this permission, error 12203 will be reported.

(Mandatory) Apply for the static permission.

<uses-permission android:name="android.permission.RECORD_AUDIO" />

(Mandatory) Apply for the dynamic permission.
ActivityCompat.requestPermissions(
this, new String[]{Manifest.permission.RECORD_AUDIO

}, 1);

Create an MLSoundDector object.

private static final String TAG = "MLSoundDectorDemo";

// Object of sound detection.
private MLSoundDector mlSoundDector;

// Create an MLSoundDector object and configure the callback.
private void initMLSoundDector(){
mlSoundDector = MLSoundDector.createSoundDector();
mlSoundDector.setSoundDectListener(listener);
}

Create a sound detection result callback to obtain the detection result and pass the callback to the sound detection instance.

// Create a sound detection result callback to obtain the detection result and pass the callback to the sound detection instance.

private MLSoundDectListener listener = new MLSoundDectListener() {
@Override
public void onSoundSuccessResult(Bundle result) {
// Processing logic when the detection is successful. The detection result ranges from 0 to 12, corresponding to the 13 sound types whose names start with SOUND_EVENT_TYPE. The types are defined in MLSoundDectConstants.java.
int soundType = result.getInt(MLSoundDector.RESULTS_RECOGNIZED);
Log.d(TAG,"Detection success:"+soundType);
}
@Override
public void onSoundFailResult(int errCode) {
// Processing logic for detection failure. The possible cause is that your app does not have the microphone permission (Manifest.permission.RECORD_AUDIO).
Log.d(TAG,"Detection failure"+errCode);
}
};

Note: The code above prints the type of the detected sound as an integer. In the actual situation, you can convert the integer into a data type that users can understand.

Definition for the types of detected sounds:

<string-array name="sound_dect_voice_type">
<item>laughter</item>
<item>baby crying sound</item>
<item>snore</item>
<item>sneeze</item>
<item>shout</item>
<item>cat's meow</item>
<item>dog's bark</item>
<item>running water</item>
<item>car horn sound</item>
<item>doorbell sound</item>
<item>knock</item>
<item>fire alarm sound</item>
<item>alarm sound</item>
</string-array>

Start and stop sound detection.

@Override
public void onClick(View v) {
switch (v.getId()){
case R.id.btn_start_detect:
if (mlSoundDector != null){
boolean isStarted = mlSoundDector.start(this); // context: Context.
// If the value of isStared is true, the detection is successfully started. If the value of isStared is false, the detection fails to be started. (The possible cause is that the microphone is occupied by the system or another app.)
if (isStarted){
Toast.makeText(this,"The detection is successfully started.", Toast.LENGTH_SHORT).show();
}
}
break;
case R.id.btn_stop_detect:
if (mlSoundDector != null){
mlSoundDector.stop();
}
break;
}
}

Call destroy() to release resources when the sound detection page is closed.

@Override
protected void onDestroy() {
super.onDestroy();
if (mlSoundDector != null){
mlSoundDector.destroy();
}
}

Testing the App​

Using the knock as an example, the output result of sound detection is expected to be 10.

Tap Start detecting and simulate a knock on the door. If you get logs as follows in Android Studio's console, they indicates that the integration of the sound detection SDK is successful.

More Information​

Sound detection belongs to one of the six capability categories of ML Kit, which are related to text, language/voice, image, face/body, natural language processing, and custom model.

Sound detection is a service in the language/voice-related category.

Interested in other categories? Feel free to have a look at the HUAWEI ML Kit document.

To learn more, please visit:

HUAWEI Developers official website

Development Guide

Reddit to join developer discussions

GitHub or Gitee to download the demo and sample code

Stack Overflow to solve integration problems

Follow our official account for the latest HMS Core-related news and updates.

Original Source