[Huawei Developers] ErtugSagman:
Hi everyone!

Today I will be briefing through how to implement a 3D Scene to display objects and play sounds in your Andorid Kotlin projects.

View attachment 5240503

All we need is Android Studio with version 3.5 or higher and a smartphone running Android 4.4 or later. The kits we need requires these specifications at minimum:

For Scene Kit alone:

JDK version: 1.7 or later

minSdkVersion: 19 or later

targetSdkVersion: 19 or later

compileSdkVersion: 19 or later

Gradle version: 3.5 or later

For Audio Kit alone:

JDK version: 1.8.211 or later

minSdkVersion: 21

targetSdkVersion: 29

compileSdkVersion: 29

Gradle version: 4.6 or later

That brings us to use Audio Kit’s minimum requirements as Scene Kit requirements are lower. So we should keep those in our minds while configuring our project. Let’s begin with implementing Scene Kit.

First of all, our aim with this Scene Kit implementation is to achieve a view of 3D object that we can interact with like this:

View attachment 5240505

We will also add multiple objects and be able to cycle through. In order to use Scene Kit in your project, start by adding these implementations to build.gradle files.

buildscript {
repositories {
...
maven { url 'https://developer.huawei.com/repo/' }
}
...
}

allprojects {
repositories {
...
maven { url 'https://developer.huawei.com/repo/' }
}
}

dependencies {
...
implementation 'com.huawei.scenekit:full-sdk:5.1.0.300'
}

Note that in the project I have used viewBinding feature of Kotlin in order to skip boilerplate view initialization codes. If you want to use viewBinding, you should add this little code in your app-level build.gradle.

android {
...
buildFeatures {
viewBinding true
}
...
}

After syncing gradle files, we are ready to use Scene Kit in our project. Keep in mind that our purpose is solely to display 3D objects that user can interact with. But Scene Kit has much more deeper abilities that is provided for us. If you are actually looking for something different or want to discover all abilities, follow the link. Else, let’s continue with a custom scene view.

Scene Kit - HMS Core - HUAWEI Developer

Simple purpose of this custom view is just to load automatically our first object into the view when it is done initializing. Of course you can skip this part if you don’t need this purpose. Bear in mind that you should use default SceneView and load manually instead. You can still find the code for loading objects in this code snippet.

import android.content.Context
import android.util.AttributeSet
import android.view.SurfaceHolder
import com.huawei.hms.scene.sdk.SceneView

class CustomSceneView : SceneView {
constructor(context: Context?) : super(context)
constructor(
context: Context?,
attributeSet: AttributeSet?
) : super(context, attributeSet)

override fun surfaceCreated(holder: SurfaceHolder) {
super.surfaceCreated(holder)
loadScene("car1/scene.gltf")
loadSpecularEnvTexture("car1/specularEnvTexture.dds")
loadDiffuseEnvTexture("car1/diffuseEnvTexture.dds")
}
}

Well we cannot display anything actually before adding our object files in our projects. You will need to obtain object files elsewhere as object models I have, are not my creation. You can find public objects with ‘gltf object’ queries in search engines. Once you have your object, head to your project file and create ‘assets’ file under ‘../src/main/’ and place your object file here. In my case:

View attachment 5240507

In the surfaceCreated method, loadScene(), loadSpecularEnvTexture() and loadDiffuseEnvTexture() methods are used to load our object. Once the view surface is created, our first object will be loaded into it. Now head to the xml for your 3D objects to display, in this guide case, activity_main.xml. In activity_main.xml, create the view we just created. I have also added simple arrows to navigate between models.

<?xml version="1.0" encoding="utf-8"?>
<androidx.constraintlayout.widget.ConstraintLayout xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:app="http://schemas.android.com/apk/res-auto"
xmlns:tools="http://schemas.android.com/tools"
android:layout_width="match_parent"
android:layout_height="match_parent"
tools:context=".MainActivity">

<com.example.sceneaudiodemo.CustomSceneView
android:id="@+id/csv_main"
android:layout_width="match_parent"
android:layout_height="match_parent"/>

<ImageView
android:id="@+id/iv_rightArrow"
android:layout_width="32dp"
android:layout_height="32dp"
android:layout_margin="12dp"
android:src="@drawable/ic_arrow"
android:tint="@color/white"
app:layout_constraintBottom_toBottomOf="parent"
app:layout_constraintEnd_toEndOf="parent"
app:layout_constraintTop_toTopOf="parent" />

<ImageView
android:id="@+id/iv_leftArrow"
android:layout_width="32dp"
android:layout_height="32dp"
android:layout_margin="12dp"
android:rotation="180"
android:src="@drawable/ic_arrow"
android:tint="@color/white"
app:layout_constraintBottom_toBottomOf="parent"
app:layout_constraintStart_toStartOf="parent"
app:layout_constraintTop_toTopOf="parent" />

</androidx.constraintlayout.widget.ConstraintLayout>

Now we are all set for our object to be displayed once our app is launched. Let’s add a few other objects and navigate between. In MainActivity:

private lateinit var binding: ActivityMainBinding
private var selectedId = 0
private val modelSceneList = arrayListOf(
"car1/scene.gltf",
"car2/scene.gltf",
"car3/scene.gltf"
)
private val modelSpecularList = arrayListOf(
"car1/specularEnvTexture.dds",
"car2/specularEnvTexture.dds",
"car3/specularEnvTexture.dds"
)
private val modelDiffList = arrayListOf(
"car1/diffuseEnvTexture.dds",
"car2/diffuseEnvTexture.dds",
"car3/diffuseEnvTexture.dds"
)

override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
binding = ActivityMainBinding.inflate(layoutInflater)
val view = binding.root
setContentView(view)
binding.ivRightArrow.setOnClickListener {
if (modelSceneList.size == 0) return@setOnClickListener
selectedId = (selectedId + 1) % modelSceneList.size // To keep our id in the range of our model list
loadImage()
}
binding.ivLeftArrow.setOnClickListener {
if (modelSceneList.size == 0) return@setOnClickListener
if (selectedId == 0) selectedId = modelSceneList.size - 1 // To keep our id in the range of our model list
else selectedId -= 1
loadImage()
}
}

private fun loadImage() {
binding.csvMain.loadScene(modelSceneList[selectedId])
binding.csvMain.loadSpecularEnvTexture(modelSpecularList[selectedId])
binding.csvMain.loadDiffuseEnvTexture(modelDiffList[selectedId])
}

In onCreate(), we are making a simple next/previous logic to change our objects. And we are storing our objects’ file paths as strings in separate lists that we created hardcoded. You may want to tinker to make it dynamic but I wanted to keep it simple for the guide. We used ‘selectedId’ to keep track of current object being displayed.

And that’s all for SceneView implementation for 3D object views!

Now no time to waste, let’s continue with adding Audio Kit.

Head back to the app-level build.gradle and add Audio Kit implementation.

dependencies {
...
implementation 'com.huawei.hms:audiokit-player:1.1.0.300'
...
}

As we already added necessary repository while implementing Scene Kit, we won’t need to make any changes in the project-level build.gradle. So let’s go back and complete Audio Kit.

I added a simple play button to activity_main.xml.

<Button
android:id="@+id/btn_playSound"
android:layout_width="wrap_content"
android:layout_height="wrap_content"
android:text="Play"
app:layout_constraintBottom_toBottomOf="parent"
app:layout_constraintEnd_toEndOf="parent"
app:layout_constraintStart_toStartOf="parent" />

I will use this button to play sound for current object. Afterwards, all we need to do is make these changes in our MainActivity.

private var mHwAudioManager: HwAudioManager? = null
private var mHwAudioPlayerManager: HwAudioPlayerManager? = null

override fun onCreate(savedInstanceState: Bundle?) {
...
initPlayer(this)
binding.btnPlaySound.setOnClickListener {
mHwAudioPlayerManager?.play(selectedId) // Requires playlist to play, selectedId works for index to play.
}
...
}

private fun initPlayer(context: Context) {
val hwAudioPlayerConfig = HwAudioPlayerConfig(context)
HwAudioManagerFactory.createHwAudioManager(hwAudioPlayerConfig,
object : HwAudioConfigCallBack {
override fun onSuccess(hwAudioManager: HwAudioManager?) {
try {
mHwAudioManager = hwAudioManager
mHwAudioPlayerManager = hwAudioManager?.playerManager
mHwAudioPlayerManager?.playList(getPlaylist(), 0, 0)
} catch (ex: Exception) {
ex.printStackTrace()
}
}

override fun onError(p0: Int) {
Log.e("init:onError: ","$p0")
}
})
}

fun getPlaylist(): List<HwAudioPlayItem>? {
val playItemList: MutableList<HwAudioPlayItem> = ArrayList()
val audioPlayItem1 = HwAudioPlayItem()
val sound = Uri.parse("android.resource://yourpackagename/raw/soundfilename").toString() // soundfilename should not include file extension.
audioPlayItem1.audioId = "1000"
audioPlayItem1.singer = "Taoge"
audioPlayItem1.onlinePath =
"https://lfmusicservice.hwcloudtest.cn:18084/HMS/audio/Taoge-chengshilvren.mp3"
audioPlayItem1.setOnline(1)
audioPlayItem1.audioTitle = "chengshilvren"
playItemList.add(audioPlayItem1)
val audioPlayItem2 = HwAudioPlayItem()
audioPlayItem2.audioId = "1001"
audioPlayItem2.singer = "Taoge"
audioPlayItem2.onlinePath =
"https://lfmusicservice.hwcloudtest.cn:18084/HMS/audio/Taoge-dayu.mp3"
audioPlayItem2.setOnline(1)
audioPlayItem2.audioTitle = "dayu"
playItemList.add(audioPlayItem2)
val audioPlayItem3 = HwAudioPlayItem()
audioPlayItem3.audioId = "1002"
audioPlayItem3.singer = "Taoge"
audioPlayItem3.onlinePath =
"https://lfmusicservice.hwcloudtest.cn:18084/HMS/audio/Taoge-wangge.mp3"
audioPlayItem3.setOnline(1)
audioPlayItem3.audioTitle = "wangge"
playItemList.add(audioPlayItem3)
return playItemList
}

After making the changes, we will be able to play sounds in our projects. I had to add online sounds available, if you want to use sounds added in your project, then you should use ‘sound’ variable I have given example of, and change ‘audioPlayItem.setOnline(1)’ to ‘audioPlayItem.setOnline(0)’. Also change ‘audioPlayItem.onlinePath’ to ‘audioPlayItem.filePath’. Then you should be able to play imported sound files too. By the way, that’s all for Audio Kit as well! We didn’t need to implement any play/pause or seekbar features as we just want to hear the sound and get done with it.

So we completed our guide for how to implement a Scene Kit 3D Scene View and Audio Kit to play sounds in our projects in Kotlin. If you have any questions or suggestions, feel free to contact me. Thanks for reading this far and I hope it was useful for you!

References

Scene Kit - HMS Core - HUAWEI Developer

Audio Kit - Audio Development Component - HUAWEI Developer