[Android Software Development] chdloc:
Unlocking the hardware filter in the Qualcomm Snapdragon?

Ignoring Bluetooth audio for the time being, there are two ways of playing audio on Android

Let Android do all the work (decoding, fx, volume control, and mixing) and send audio to the codec (ADM)

Let the aDSP do the heavy lifting (decoding, fx, volume control, and mixing) and have it send the final audio stream to the codec (ASM)

The ASM kernel-aDSP interface offers the ability to apply fx to the audio stream. One of the fx options are 12 biquads (they can be queried with tinymix). Unfortunately, the stock kernel on Nexus and Pixel devices cannot take advantage of the biquads as only the MultiMedia1, MultiMedia2, and MultiMedia3 DAI front-ends are defined/enabled. To use the biquads one needs to, at a minimum, enable the plumbing for MultiMedia4 EQ in the kernel, which is the one dedicated to offload processing.

To make a long story short, the biquads in the aDSP would even theoretically only be accessible when audio is being offloaded to the aDSP. That does not work, however, as the required plumbing is not present. So even if the plumbing were present, only very few apps would be making use of it as most apps do not offload.

This is very much work in progress. Some random thoughts and discoveries are listed below.

(The details refer to the Nexus 5, but should be generally applicable to any Snapdragon-based device)

There is another set of biquad (?) filters available inside the Snapdragon chip; they show up in tinymix (and in the kernel sources @ msm-pcm-routing.c) as MultiMediaX EQ BandY, where X={1,2,3} and Y=[1(1)12]. This means that the filters are only applicable to streams that make use of one of the first three MultiMedia streams. However, by default, music is routed through MultiMedia5, which is not eligible for filtering. This filter may be a 24-th order IIR filter which would allow more degrees of freedom than the IIR filters in the codec would.

Simply changing the stream from MulitMedia5 to MultiMedia1 in mixer_paths.xml (section: low-latency-playback) does not work, kernel log error: "MSM8974 LowLatency: asoc: MSM8974 LowLatency no valid playback route from source to sink". Where in the kernel sources is the backend-to-frontend forced to use MultiMedia5 for low-latency-playback?

logcat error:

D/audio_hw_primary( 193): enable_snd_device: snd_device(5: headphones)

D/audio_hw_primary( 193): enable_audio_route: apply and update mixer path: low-latency-playback

E/audio_hw_primary( 193): start_output_stream: cannot open device '/dev/snd/pcmC0D15p': Invalid argument

The device #15 is hard-coded in the audio HAL (audio_hw.c: [USECASE_AUDIO_PLAYBACK_LOW_LATENCY] = {15, 15} and msm8974/platform.h: #define LOWLATENCY_PCM_DEVICE 15)

# cat /proc/asound/card0/pcm15p/info

card: 0

device: 15

subdevice: 0

stream: PLAYBACK

id: MultiMedia5 (*)

[...]

MultiMedia1 is mapped to pcm0p, MultiMedia2 is mapped to pcm1p, MultiMedia3/4 are not mapped anywhere

If the music is offloaded, then it seems to always bypass the MultiMedia mixer and directly connect to MM_DL4

The only way to enable the filters seems to change the hardcoded device number in the audio HAL from 15 to either 0 (MultiMedia1) or 1 (MultiMedia2) and make the corresponding changes in mixer_paths.xml (point to MultiMedia1/2 for the low-latency-playback use-case).

Filters will only be available in the non-offloaded case.

UPDATE:

applied the above modifications using MultiMedia2; music plays fine, but kernel log reads

<3>[ 101.532335] q6asm_get_audio_client: invalid session: -1065112576

<3>[ 101.532453] msm_send_eq_values: Could not get audio client for session: -1065112576

and the filters never get applied; also, the maximum allowed value for the first filter parameter seems to be 65535; hence the filter implementation differs from the IIR in the WCD93xx codec.

Lots of debug information in /sys/kernel/debug/asoc/msm8974-taiko-mtp-snd-card and /sys/devices/sound.29/

Some new findings:

Indeed, the filters in the ADSP are a set of biquads, each of which expects the following five parameters (msm-pcm-routing-v2.c):

first parameter: band_indx: ranges from 0 to 11

second parameter: filter_type: the following types are defined in apr_audio-v2.h

/*	No equalizer effect.*/

#define ASM_PARAM_EQYPE_NONE 0

/*	Bass boost equalizer effect.*/

#define ASM_PARAM_EQ_BASS_BOOST 1

/*Bass cut equalizer effect.*/

#define ASM_PARAM_EQ_BASS_CUT 2

/*	Treble boost equalizer effect */

#define ASM_PARAM_EQREBLE_BOOST 3

/*	Treble cut equalizer effect.*/

#define ASM_PARAM_EQREBLE_CUT 4

/*	Band boost equalizer effect.*/

#define ASM_PARAM_EQ_BAND_BOOST 5

/*	Band cut equalizer effect.*/

#define ASM_PARAM_EQ_BAND_CUT 6

third parameter: center_freq_hz

fourth parameter: filter_gain: -12 to +12 dB in 1 dB increments; format unclear

fifth parameter: q_factor: expects the numerator of the Q8 representation; the numerator is always 2^8=256; for example 1.5=384/256, so 384 is the expected parameter

The basic problem persists: the filters are applied only to MultiMediaX, X={1,2,3}, streams. At least on the devices I've seen, music playback is routed through MultiMedia4 (compress-offload-playback or low-latency-playback use cases, see mixer_paths.xml).

There are two options, both of which require changes to the kernel sources

add support for EQ on MultiMedia4 path

modify front-end to back-end routes from MultiMedia4 to MultiMedia1