[Android Software Development] chdloc:
Some Theory

The following hidden text shows the original two posts of this thread for those of you who are interested in some gory details

Let me start off by pointing out that all of this is very technical and that there may be a steep learning curve. So please fire away with all your questions, so I can improve on the clarity of this and the following few posts.

Modern Qualcomm audio codecs, such as the WCD9310 (used, e.g., in the Nexus 4), the WCD9320 (used, e.g., in the Nexus 5), or the WCD9330 (used, e.g., in the Nexus 6, Nexus 5x, and Nexus 6p) offer the option to apply hardware filtering of the audio signals passing through the chip. Qualcomm offers two software-customizable 10-th order infinite-impulse response (IIR) filter, implemented as a cascade of five direct-form I biquad IIR filters, also known as second-order sections (SOS).

What are the advantages of using hardware filters instead of filters implemented in software at the Android level?

as the filters are implemented in hardware, there is no (maybe minute?) additional impact on battery life

as the filters are implemented at the lowest possible level, there is, in principle, no dependence on the media player, ROM, or kernel. Hence, a useful application of this IIR filter may be to compensate (within reasonable levels) one's favorite headphones for any media player. Please see this post for a suggestion on how you could measure your headphones (and hearing) to derive your own compensation filter.

the filter coefficients are supplied in the human-readable mixer_paths.xml file which can be modified by any knowledgeable user. More on that below.

if designed properly, the filters will not introduce any non-linear distortions; the user has full control over the design (see post #3)

a very simple channel cross-feed can be implemented. Cross-feed can help some folks that suffer from listening fatigue when listening to music via headphones.

As there is no documentation available that details the audio filtering options offered by the Qualcomm codecs, I spent some time reverse engineering.

After some quality time, I have succeeded in enabling the hardware audio filters inside the Qualcomm for equalization purposes which may be useful for any device that sports a Qualcomm codec. My examples below are tailored toward the Nexus 5, but porting this to other devices should be relatively straightforward.

(If you are reading this for the first time it may be instructional to jump to post #3 now, which details a practical example, before continuing to read this post)

The following details the current status of my investigations.

Qualcomm's biquad implementation is given by the following difference equation:

a0*y[n] = b0*x[n] + b1*x[n-1] + b2*x[x-2] - a1*y[n-1] - a2*y[n-2]

a0 is implicitly set to 1 internally, so it does not need to be defined explicitly.

Hence, each SOS is fully represented by five parameters, i.e. b0, b1, b2, a1, and a2.

The fixed-point numbers defining the filter coefficients seem to be stored in Q28 format, i.e. 1.0 corresponds to 2^{28}=268435456

Example: Highpass filter w/ cutoff @ 100Hz

Matlab design: elliptic IIR highpass; f_stop=80Hz, f_pass=100Hz, A_stop=40dB, A_pass=1dB; order = 5, sections: 3

SOS1: b0=1, b1=-1.99989100262078, b2=1, a0=1, a1=-1.99852109834172, a2=0.998692521205726

SOS2: b0=1, b1=-1.99994495162412, b2=1, a0=1, a1=-1.99015476348031, a2=0.990440326636932

SOS3: b0=1, b1=-1 b2=0, a0=1, a1=-0.966597321365327, a2=0

SOS fixed point <=> floor(SOS*268435456)

SOS1: b0=268435456, b1=-536841654, b2=268435456, a0=268435456, a1=-536473923, a2=268084482

SOS2: b0=268435456, b1=-536856136, b2=268435456, a0=268435456, a1=-534228102, a2=265869300

SOS3: b0=268435456, b1=-268435456, b2=0, a0=268435456, a1=-259468993, a2=0

id="0" <=> b0, id="1" <=> b1, id="2" <=> b2, id="3" <=> a1, id="4"<=> a2

keep all positive numbers as is and omit a0

negative numbers: two's complement, i.e. convert absolute number to binary, flip bits and add 1;

the two MSB are reserved (cf. wcd9320.c @ function set_iir_band_coeff() )!

example:

-536841654 (b1 of first SOS)

0001 1111 1111 1111 1000 1101 1011 0110 (converted absolute number, i.e. 536841654, to 32-bit binary)

1110 0000 0000 0000 0111 0010 0100 1001 (flipped bits of 32-bit binary)

0010 0000 0000 0000 0111 0010 0100 1010 (flip bits + 1; top two bits reserved, i.e. zero'ed)

536900170 (converted back to decimal)

What follows is a new "headphone" section for a Nexus 5 in mixer_paths.xml file with modified routing and the IIR filters set to be in "passthrough" mode, i.e. the data flows through the filters, but remain unmodified. In other words, b0=1.0, a1=a2=b1=b2=0.0, i.e. y[n]=x[n]. This new headphone section can now be used to accept any newly designed IIR filter on-the-fly for testing (Warning: make sure that the filters are stable before trying to route audio through them!). When a final filter is found the filter coefficients need to be hard-coded into the mixer_paths.xml file.

<path name="headphones">
<ctl name="SLIM RX1 MUX" value="AIF1_PB" />
<ctl name="SLIM RX2 MUX" value="AIF1_PB" />
<ctl name="SLIM_0_RX Channels" value="Two" />
<!-- begin stock routing (commented out) -->
<!-- <ctl name="RX1 MIX1 INP1" value="RX1" /> -->
<!-- <ctl name="RX2 MIX1 INP1" value="RX2" /> -->
<!-- end stock routing (commented out) -->
<ctl name="CLASS_H_DSM MUX" value="DSM_HPHL_RX1" />
<ctl name="HPHL DAC Switch" value="1" />

<!-- enabling IIR filter; coefficients can be modified on-the-fly with tinymix, see below -->
<!-- RX1 plugs into the IIR1 mux and RX2 plugs into the IIR2 mux -->
<ctl name="IIR1 INP1 MUX" value="RX1" />
<ctl name="IIR2 INP1 MUX" value="RX2" />
<!-- IIR1 filter cofficients (pass-through) -->
<ctl name="IIR1 Band1" id ="0" value="268435456" />
<ctl name="IIR1 Band1" id ="1" value="0" />
<ctl name="IIR1 Band1" id ="2" value="0" />
<ctl name="IIR1 Band1" id ="3" value="0" />
<ctl name="IIR1 Band1" id ="4" value="0" />
<ctl name="IIR1 Band2" id ="0" value="268435456" />
<ctl name="IIR1 Band2" id ="1" value="0" />
<ctl name="IIR1 Band2" id ="2" value="0" />
<ctl name="IIR1 Band2" id ="3" value="0" />
<ctl name="IIR1 Band2" id ="4" value="0" />
<ctl name="IIR1 Band3" id ="0" value="268435456" />
<ctl name="IIR1 Band3" id ="1" value="0" />
<ctl name="IIR1 Band3" id ="2" value="0" />
<ctl name="IIR1 Band3" id ="3" value="0" />
<ctl name="IIR1 Band3" id ="4" value="0" />
<ctl name="IIR1 Band4" id ="0" value="268435456" />
<ctl name="IIR1 Band4" id ="1" value="0" />
<ctl name="IIR1 Band4" id ="2" value="0" />
<ctl name="IIR1 Band4" id ="3" value="0" />
<ctl name="IIR1 Band4" id ="4" value="0" />
<ctl name="IIR1 Band5" id ="0" value="268435456" />
<ctl name="IIR1 Band5" id ="1" value="0" />
<ctl name="IIR1 Band5" id ="2" value="0" />
<ctl name="IIR1 Band5" id ="3" value="0" />
<ctl name="IIR1 Band5" id ="4" value="0" />
<ctl name="IIR1 Enable Band1" value="1" />
<ctl name="IIR1 Enable Band2" value="1" />
<ctl name="IIR1 Enable Band3" value="1" />
<ctl name="IIR1 Enable Band4" value="1" />
<ctl name="IIR1 Enable Band5" value="1" />
<!-- IIR2 filter coefficients; same as IIR1 -->
<ctl name="IIR2 Band1" id ="0" value="268435456" />
<ctl name="IIR2 Band1" id ="1" value="0" />
<ctl name="IIR2 Band1" id ="2" value="0" />
<ctl name="IIR2 Band1" id ="3" value="0" />
<ctl name="IIR2 Band1" id ="4" value="0" />
<ctl name="IIR2 Band2" id ="0" value="268435456" />
<ctl name="IIR2 Band2" id ="1" value="0" />
<ctl name="IIR2 Band2" id ="2" value="0" />
<ctl name="IIR2 Band2" id ="3" value="0" />
<ctl name="IIR2 Band2" id ="4" value="0" />
<ctl name="IIR2 Band3" id ="0" value="268435456" />
<ctl name="IIR2 Band3" id ="1" value="0" />
<ctl name="IIR2 Band3" id ="2" value="0" />
<ctl name="IIR2 Band3" id ="3" value="0" />
<ctl name="IIR2 Band3" id ="4" value="0" />
<ctl name="IIR2 Band4" id ="0" value="268435456" />
<ctl name="IIR2 Band4" id ="1" value="0" />
<ctl name="IIR2 Band4" id ="2" value="0" />
<ctl name="IIR2 Band4" id ="3" value="0" />
<ctl name="IIR2 Band4" id ="4" value="0" />
<ctl name="IIR2 Band5" id ="0" value="268435456" />
<ctl name="IIR2 Band5" id ="1" value="0" />
<ctl name="IIR2 Band5" id ="2" value="0" />
<ctl name="IIR2 Band5" id ="3" value="0" />
<ctl name="IIR2 Band5" id ="4" value="0" />
<ctl name="IIR2 Enable Band1" value="1" />
<ctl name="IIR2 Enable Band2" value="1" />
<ctl name="IIR2 Enable Band3" value="1" />
<ctl name="IIR2 Enable Band4" value="1" />
<ctl name="IIR2 Enable Band5" value="1" />
<!-- IIR1 plugs back into RX1 mux and IIR2 plugs back into RX2 mux -->
<ctl name="RX1 MIX1 INP1" value="IIR1" />
<ctl name="RX2 MIX1 INP1" value="IIR2" />
<!-- set IIR1 and IIR2 gain (84<=>0dB; lower this parameter appropriately if any of the biquads introduce gain) -->
<ctl name="IIR1 INP1 Volume" value="84" />
<ctl name="IIR2 INP1 Volume" value="84" />

<!-- cross-feed into second input of the IIR filter (needs kernel mod on hammerhead!) -->
<ctl name="IIR1 INP2 MUX" value="RX2" />
<ctl name="IIR2 INP2 MUX" value="RX1" />
<!-- set the gain of the cross-feed (here: cross terms down by 24 dB, i.e 84-60) -->
<ctl name="IIR1 INP2 Volume" value="60" />
<ctl name="IIR2 INP2 Volume" value="60" />

<!-- end test filter -->

<!-- back to stock below -->
<ctl name="RX1 Digital Volume" value="84" />
<ctl name="RX2 Digital Volume" value="84" />
<!-- lowered the HPHL[R] Volume from 15 to 10 to significantly reduce THD -->
<ctl name="HPHL Volume" value="10" />
<ctl name="HPHR Volume" value="10" />
</path>

Note that the entries pertaining to cross-feed are optional and do not work on the Nexus 5 stock kernel as the AOSP codec driver is missing the plumbing for multiple inputs.

I noticed that the "headphones" section on Shamu and Mako do look different. Other devices will likely differ from the Hammerhead one, too. I should be able to help folks getting their "headphones" section on non-Hammerhead devices in shape.

Once booted with the modified headphone section, the filters can be modified on-the-fly by using tinymix:

# first, disable the filter to make sure that the filter does not become unstable during the update process!
tinymix "IIR1 Enable Band1" 0
tinymix "IIR1 Enable Band2" 0
tinymix "IIR1 Enable Band3" 0
tinymix "IIR1 Enable Band4" 0
tinymix "IIR1 Enable Band5" 0
tinymix "IIR2 Enable Band1" 0
tinymix "IIR2 Enable Band2" 0
tinymix "IIR2 Enable Band3" 0
tinymix "IIR2 Enable Band4" 0
tinymix "IIR2 Enable Band5" 0
# second, write new IIR1 filter (for the left audio channel)
tinymix "IIR1 Band1" 268435456 537978230 268435456 778747325 114847078
tinymix "IIR1 Band2" 268435456 542902433 268435456 601441125 243151927
tinymix "IIR1 Band3" 268435456 0 0 0 0
tinymix "IIR1 Band4" 268435456 0 0 0 0
tinymix "IIR1 Band5" 268435456 0 0 0 0
# third, write new IIR2 filter (for the right audio channel); you typically want IIR2 to be the same filter as IIR1
tinymix "IIR2 Band1" 268435456 537978230 268435456 778747325 114847078
tinymix "IIR2 Band2" 268435456 542902433 268435456 601441125 243151927
tinymix "IIR2 Band3" 268435456 0 0 0 0
tinymix "IIR2 Band4" 268435456 0 0 0 0
tinymix "IIR2 Band5" 268435456 0 0 0 0
# fourth, enable the filter
tinymix "IIR1 Enable Band1" 1
tinymix "IIR1 Enable Band2" 1
tinymix "IIR1 Enable Band3" 1
tinymix "IIR1 Enable Band4" 1
tinymix "IIR1 Enable Band5" 1
tinymix "IIR2 Enable Band1" 1
tinymix "IIR2 Enable Band2" 1
tinymix "IIR2 Enable Band3" 1
tinymix "IIR2 Enable Band4" 1
tinymix "IIR2 Enable Band5" 1

NOTE: the filter coefficients shown above just provide a proof-of-concept. The designed filter in this particular example eliminates all signals below 1000 Hz to make the effect apparent. I will make Octave scripts (a free Matlab alternative) available with which you can design you own filters in a later post. To go back to passthrough mode use

tinymix "IIR1 Enable Band1" 0
tinymix "IIR1 Enable Band2" 0
tinymix "IIR1 Enable Band3" 0
tinymix "IIR1 Enable Band4" 0
tinymix "IIR1 Enable Band5" 0
tinymix "IIR2 Enable Band1" 0
tinymix "IIR2 Enable Band2" 0
tinymix "IIR2 Enable Band3" 0
tinymix "IIR2 Enable Band4" 0
tinymix "IIR2 Enable Band5" 0
tinymix "IIR1 Band1" 268435456 0 0 0 0
tinymix "IIR1 Band2" 268435456 0 0 0 0
tinymix "IIR1 Band3" 268435456 0 0 0 0
tinymix "IIR1 Band4" 268435456 0 0 0 0
tinymix "IIR1 Band5" 268435456 0 0 0 0
tinymix "IIR2 Band1" 268435456 0 0 0 0
tinymix "IIR2 Band2" 268435456 0 0 0 0
tinymix "IIR2 Band3" 268435456 0 0 0 0
tinymix "IIR2 Band4" 268435456 0 0 0 0
tinymix "IIR2 Band5" 268435456 0 0 0 0
tinymix "IIR1 Enable Band1" 1
tinymix "IIR1 Enable Band2" 1
tinymix "IIR1 Enable Band3" 1
tinymix "IIR1 Enable Band4" 1
tinymix "IIR1 Enable Band5" 1
tinymix "IIR2 Enable Band1" 1
tinymix "IIR2 Enable Band2" 1
tinymix "IIR2 Enable Band3" 1
tinymix "IIR2 Enable Band4" 1
tinymix "IIR2 Enable Band5" 1

Note that restarting the audio stream will default to the filters currently present in the mixer_paths.xml file.

Known issues

the above implementation seems to be incompatible with any type of hotword detection: if audio is playing through headphones and the user goes back to the home screen with hotword detection enabled causes the codec to freak out. The exact reason is as of yet unknown, but it is believed to be related to the new routing scheme. Bottom line is that the music will stop playing through the headphones (and it does not come back automatically) if hotword detection is being activated, no matter how. Audio will resume only after a swap of the audio route, e.g. by unplugging the headphones, has been forced. Everything will work as expected if audio is first stopped, Google Now invoked manually, followed by resuming audio playback after the Google Now interaction has ended. Hotword detection now works happily alongside IIR filtering. First, however, the reference to "IIR1 INP1 MUX" needs to be removed from the entire device path that is used for the hotword detection device in mixer_paths.xml. On hammerhead, it is the "voice-rec-mic" device. You may need to trace all the way back to one of the "adcX" devices, where X={1,2,3,...}

the above implementation may not work with any "advanced" audio player that is capable of direct manipulation of the soundcard and configuring it to sampling rates higher than 48 kHz. I did get it to work, however, while writing audio (up to 24bit/192kHz) to the audio codec directly via ALSA. One thing to keep in mind, however, is that the sampling frequency is a filter design parameter. Hence, if desired/necessary, separate filters need to be designed for 48, 96, and 192 kHz sampling rate.

going from 16 bit to 24 bit sample resolution causes a drop of 48 dB in level when playing back a stereo signal via ALSA directly (libasound and alsa_aplay; not sure about libtinyalsa and tinyplay as I have not gotten it to work yet).

I believe this is due to the fact that the ALSA library presents the 24 bit in the lower 3 Bytes while the codec looks at the top 3 Bytes for its digital-to-analog conversion. The codec's digital volume control ("RX Volume" and "IIR INP Volume") can be used to apply the missing 48 dB of gain, i.e. shifting the lower 3 Bytes to the top 3 Bytes.

audio needs to be played once with IIR filtering disabled immediately after the sampling rate has been changed (48 <-> 96 <-> 192).

The IIR filter gain parameter, "IIR1/2 INP1 Volume", seems to be ignored. Please see this post for further details.

The IIR filter seems to add a 180 degree phase shift to the signals, which is perceptually irrelevant as long as both channels undergo the same phase shift.

The IIR filter is implemented inside the Qualcomm audio codec as a cascade of five biquad filters. A straightforward design of the IIR filter can be achieved by designing each individual biquad separately, where each section "takes care" of a certain region of the frequency response to be modified. For example, imagine you have a headphone that exhibits a resonance (peak) in its frequency response. One of the five biquads can be designed to reduce the gain around that frequency to the desired level. I will be giving examples in the third post below.

This post discusses, how the biquads -- and the resulting IIR filter to be written to the codec -- can be designed.

Prerequisites

familiarize yourself with the types of biquads available and how each of them change the frequency response with a set of parameters. There is an excellent website available that offers a very nice interface for playing around with any of the biquad designs

once you have a rough understanding of biquads, download and install Octave. Once installed, you will also need the "signal package", see here and here

download the attached archive and extract the two Octave/Matlab scripts, wcd93xx_sos.m and wcd93xx_filter.m

depending on whether you are running a lollipop- or kitkat-based ROM, download the appropriate archive that contains the tinymix binary (part of AOSP, but not installed by default). Copy it to, e.g. /system/bin, and make executable (744).

Now, launch Octave and type (after navigating to the directory where the two scripts reside)

help wcd93xx_sos

which will show some details on how to use this function to design individual biquads. For example, let's assume we would like to reduce a resonance around 6.5 kHz by about 6 dB for 48 kHz sampling rate. One possible option would be

[biquad1, SOS1]=wcd93xx_sos(6500,3,-6,48000,7);

which will show you the magnitude and phase response of the biquad.

You can now design up to five biquads to "deal" with deficiencies in the frequency response you are trying to equalize (lowpass, highpass, bandpass, notch filter, or shelf filter). Make sure you call each biquad with different output parameters [biquadX, SOSX] where X={1,2,3,4,5}. Once you have a set of biquads you deem appropriate, call the script wcd93xx_filter which will simply ask you how many biquads you have designed to generate the cascade and to plot the overall magnitude and phase response. This script also prints a series tinymix commands with which the filters can be applied to the Qualcomm audio codec while music is playing through your headphones, but only if the "headphones" section from the previous post had been added to your mixer_paths.xml file. The series of commands as well as the above "headphones" section target the Nexus 5, but they may work with other phones either unaltered, or with only minor modifications. Please post your "headphones" section of /system/etc/mixer_paths.xml if you run into trouble.

I understand that all of this may be overwhelming and confusing. To address this, I will walk you through a complete design procedure in the next post in great detail.

Some basic design guidelines

be very careful when applying any gains (AdB>0) as you may end up with bad distortions. If you do need amplification of certain frequencies or ranges of frequencies, you may want to appropriately reduce the IIR filter gain first (mixer variable: "IIR1/2 INP1 Volume", see next post)

<to be completed>

Make sure you make a backup of your /system/etc/mixer_paths.xml file (and know how to restore it if things go south) first before attempting to change anything.

Update (07/31/2015): small update to the Matlab/Octave scripts. A bug has been fixed that would show the overall compensation filter using 48 kHz sampling rate only.