[Huawei Developers] muraliameakula:
Article Introduction

The article demonstrates the usage of Huawei ML Test to Speech (TTS) functionality in a News app. Users can play the news articles instead of reading the lengthy articles.

Huawei ML Kit

HUAWEI ML Kit allows your apps to easily leverage Huawei's long-term proven expertise in machine learning to support diverse artificial intelligence (AI) applications throughout a wide range of industries. Thanks to Huawei's technology accumulation, ML Kit provides diversified leading machine learning capabilities that are easy to use, helping you develop various AI apps.

Huawei ML Kit Text To Speech (TTS)

Huawei ML Kit Text To Speech (TTS) Converts text information into audio output in real time. Huawei TTS supports rich timbres, volume and speed options to produce more natural and human audible sounds. The service utilizes the deep neural network (DNN) synthesis mode and can be quickly integrated through the on-device SDK to generate audio data in real time.

Use Cases

HMS ML Kit Text To Speech (TTS) service can be widely utilized in so many everyday scenarios like:

TTS can be used in GPS and Sat Nav devices and voices can be clearly and accurately pronounce directions so you can reach your destination confidently and smartly.

TTS can provide the accessibility features to the disabled persons. Visually Impared persons can use the utility apps by experiencing the Text To Speech navigation features.

TTS can convert a large amount of text into speech output which can support the News Apps to read out the news articles.

Combining learning with entertainment makes it both fun and educational for children and adult learners alike. Edutainment can involve any learning field, with video games, TV shows, toys, and radio soaps designed to teach viewers about any topic and that can be made possible by using TTS features.

Development Preparation

Environment Setup

Android Studio 3.0 or later

Java JDK 1.8 or later

This document suppose that you have already done the Android Studio setup on your PC.

Project Setup

Create and configure app information in App Gallery Connect

Configuring the Signing Certificate Fingerprint inside project settings

Enable the ML Kit API from the Manage Api Settings inside App Gallery Console

Create a Android project using Android Studio

Copy the agconnect-services.json file to the android/app directory of your project

Maven Repository Configuration

1. Open the build.gradle file in the android directory of your project.

Navigate to the buildscript section and configure the Maven repository address and agconnect plugin for the HMS SDK.

buildscript {
repositories {
google()
jcenter()
maven { url 'https://developer.huawei.com/repo/' }
}

dependencies {
/*
* <Other dependencies>
*/
classpath 'com.huawei.agconnect:agcp:1.4.2.301'
}
}

Navigate to allprojects and configure the Maven repository address for the HMS SDK.

allprojects {
repositories {
google()
jcenter()
maven { url 'https://developer.huawei.com/repo/' }
}
}

2. Open the build.gradle file in the android/app/ directory and add apply plugin: 'com.huawei.agconnect' line after other apply entries.

apply plugin: 'com.android.application'
apply from: "$flutterRoot/packages/flutter_tools/gradle/flutter.gradle"
apply plugin: 'com.huawei.agconnect'

3. Set your package name in defaultConfig > applicationId and set minSdkVersion to 19 or higher. Package name must match with the package_name entry in agconnect-services.json file.

4. In your Android project directory, open the app.gradle file and add the following dependency to support Huawei ML TTS functionality.

implementation 'com.huawei.hms:ml-computer-voice-tts:2.2.0.300'

Adding the Permissions

To call the on-cloud API for recognition or detection, your app needs to apply for the Internet permission.

<uses-permission android:name="android.permission.INTERNET"/>

Development Process

1. When using on-cloud services of HUAWEI ML Kit, you have to set the API key or access token to call the TTS on Cloud capability.

MLApplication.getInstance().setApiKey("Your Api Key");

2. Create TTSEngine by providing the MLTtsConfig object. Customzied configuration class MLTtsConfig can be created to create a text to speech engine.

// Use customized parameter settings to create a TTS engine.
MLTtsConfig mlTtsConfig = new MLTtsConfig()
// Set the text converted from speech to English.
.setLanguage(MLTtsConstants.TTS_EN_US)
// Set the English timbre.
.setPerson(MLTtsConstants.TTS_SPEAKER_FEMALE_EN)
// Set the speech speed. The range is (0,5.0]. 1.0 indicates a normal speed.
.setSpeed(1.0f)
// Set the volume. The range is (0,2). 1.0 indicates a normal volume.
.setVolume(1.0f);

mlTtsEngine = new MLTtsEngine(mlTtsConfig);
// Set the volume of the built-in player, in dBs. The value is in the range of [0, 100].
mlTtsEngine.setPlayerVolume(20);
// Update the configuration when the engine is running.
mlTtsEngine.updateConfig(mlTtsConfig);

In the above code snippet we have create one customzied MLTtsConfig object. We can control the following factors while creating the TTSEngine.

Language : We have set the language to English by setting MLTtsConstants.TTS_EN_US

Person Voice : We have set the person voice to Female English Speaker by setting MLTtsConstants.TTS_SPEAKER_FEMALE_EN

Speech speed : The range is (0,5.0]. 1.0 indicates a normal speed.

Set the volume : The range is (0,2). 1.0 indicates a normal volume.

3. Create a TTS callback function to process the TTS result.

MLTtsCallback callback = new MLTtsCallback() {
@Override
public void onError(String taskId, MLTtsError err) {
// Processing logic for TTS failure.
Log.d("TTSNews", err.getErrorMsg());
}

@Override
public void onWarn(String taskId, MLTtsWarn warn) {
// Alarm handling without affecting service logic.
Log.d("TTSNews", warn.getWarnMsg());
}

@Override
// Return the mapping between the currently played segment and text. start: start position of the audio segment in the input text; end (excluded): end position of the audio segment in the input text.
public void onRangeStart(String taskId, int start, int end) {
// Process the mapping between the currently played segment and text.
Log.d("TTSNews", "OnRangeStart");
}

@Override
// taskId: ID of a TTS task corresponding to the audio.
// audioFragment: audio data.
// offset: offset of the audio segment to be transmitted in the queue. One TTS task corresponds to a TTS queue.
// range: text area where the audio segment to be transmitted is located; range.first (included): start position; range.second (excluded): end position.
public void onAudioAvailable(String taskId, MLTtsAudioFragment audioFragment, int offset, Pair<Integer, Integer> range,
Bundle bundle) {
// Audio stream callback API, which is used to return the synthesized audio data to the app.
Log.d("TTSNews", "onAudioAvailable");
}

@Override
public void onEvent(String taskId, int eventId, Bundle bundle) {
// Callback method of a TTS event. eventId indicates the event name.
switch (eventId) {
case MLTtsConstants.EVENT_PLAY_START:
// Called when playback starts.
isPlaying = true;
break;
case MLTtsConstants.EVENT_PLAY_STOP:
// Called when playback stops.
boolean isInterrupted = bundle.getBoolean(MLTtsConstants.EVENT_PLAY_STOP_INTERRUPTED);
isPlaying = false;
break;
case MLTtsConstants.EVENT_PLAY_RESUME:
// Called when playback resumes.
break;
case MLTtsConstants.EVENT_PLAY_PAUSE:
// Called when playback pauses.
break;

// Pay attention to the following callback events when you focus on only synthesized audio data but do not use the internal player for playback:
case MLTtsConstants.EVENT_SYNTHESIS_START:
// Called when TTS starts.
break;
case MLTtsConstants.EVENT_SYNTHESIS_END:
// Called when TTS ends.
break;
case MLTtsConstants.EVENT_SYNTHESIS_COMPLETE:
// TTS is complete. All synthesized audio streams are passed to the app.
boolean isInterrupted1 = bundle.getBoolean(MLTtsConstants.EVENT_SYNTHESIS_INTERRUPTED);
break;
default:
break;
}
}
};

Lets discuss about the TTS callback methods.

TTS callback provides the 4 call back methods.

onError() : In case of failure, the onError() method will be called and failure logic can be implemented here.

onWarn() : On case of any warning, the onWarn() method will be called and alarm handling can be done here.

onRangeStart() : This method returns the mapping between the currently played segment and text.

onAudioAvailable() : This is the Audio stream callback API, which is used to return the synthesized audio data to the app.

onEvent() : Callback method of a TTS event. eventId indicates the event name.

4. Play TTS Engine after setting the TTS callback.

mlTtsEngine.setTtsCallback(getTTSCallBack());

// Use the built-in player of the SDK to play speech in queuing mode.
String id = mlTtsEngine.speak(content, MLTtsEngine.QUEUE_APPEND);

In the above code snippet, we have to pass the article content in the string format which will be converted into the speech.

MLTtsEngine.QUEUE_APPEND is passed so that If playback is going on, the task is added to the queue for execution in sequence; if playback pauses, the playback is resumed and the task is added to the queue for execution in sequence; if there is no playback, the TTS task is executed immediately.

5. Add the TTS Engine controls.

public void controlTTSEngine(String action) {

switch (action) {
case "pause": {
// Pause playback.
mlTtsEngine.pause();
break;
}
case "resume": {
// Resume playback.
mlTtsEngine.resume();
break;
}
case "stop": {
// Stop playback.
mlTtsEngine.stop();
break;
}
case "shutdown": {
if (mlTtsEngine != null) {
mlTtsEngine.shutdown();
}
break;
}
}
}

The above method is used to control the TTS Engine on the basis of Android Life Cycle.

When app goes in the background state, the TTSEngine can be stopped.

override fun onPause() {
super.onPause()
onCloudTTSManager?.controlTTSEngine("stop")
}

When app is being destroyed then we can shutdown the TTSEngine and resources can be freedup which are occupied by the TTSEngine.

override fun onDestroy() {
super.onDestroy()
onCloudTTSManager?.controlTTSEngine("shutdown")
}

Outcome

Tips and Tricks

If targetSdkVersion is 30 or later, add the <queries> element in the manifest block in AndroidManifest.xml to allow your app to access HMS Core (APK).

<manifest ...>
...
<queries>
<intent>
<action android:name="com.huawei.hms.core.aidlservice" />
</intent>
</queries>
...
</manifest>

Currently, TTS for French, Spanish, German, and Italian is available only on Huawei and Honor phones; TTS for Chinese and English is available on all phones.

The text in a single request can contain a maximum of 500 characters and is encoded using UTF-8.

TTS depends on on-cloud APIs. During commissioning and usage, ensure that the device can access the Internet.

Conclusion

In this article we have developed News App which has the capability to convert the news articles to the speech using Huawei ML Kit Text To Speech functionality so that user can listen to the news articles.

So, by referencing the above guidelines, developers will be able to build the Huawei ML Kit TTS powered apps to support multiple daily life use cases.

References

Text to Speech

GitHub

Original Source