[Nexus 5 Android Development] chdloc:
Updates

Lollipop: The attached archive named camcorder_audio_fix_plus_stereo_playback_LMY47D_v1_1.tar.gz or this Aroma installer is fully supported on any AOSP-based Lollipop build greater than or equal to 5.1

M-Preview: The "simple" fix (without noise suppression) offered by this Aroma installer should work any of the M-Preview builds. The "advanced" mod, however, is incompatible with M-Preview builds.

October 8, 2015: Full Marshmallow support available here

August 26, 2016: Nougat support for simple fix available here

The Problem

The audio of a camcorder recording made by the Nexus 5 has often been described as "under-water" or "bubbly":

Speech signals sound distorted and low-level background sounds fade in and out. Overall, an embarrassingly bad audio quality.

As it turns out, the audio recording is fed to a noise suppression algorithm that is way too aggressive.

Also, the Nexus 5 is not able to record anything useful in very loud acoustic environments, such as a concert.

The mod presented here is aimed to fixing these issues.

Simple fix: Turning off noise suppression

I found a very simple way to turn off noise suppression altogether. All this fix does is to select a different audio device that bypasses noise suppression.

Unfortunately, this change requires modification of a system library which means that your phone needs to be rooted.

To experience the effect of the simple fix you can listen to some audio samples here.

Advantages of this "simple" version of the mod

- the recording is now devoid of any signal processing, i.e. no additional distortion is being introduced to the sound

- the mod should work with pretty much any Lollipop AOSP-based ROM. Separate mods are available for CM11 and CM12, see link below.

- the simple fix should not interfere with any other app

Disadvantages of this "simple" version of the mod

- since noise suppression has been disabled, the (white) noise that the acoustic front-end (microphone, preamp, analog circuitry) of the Nexus 5 produces is now very audible. While I do not see this a problem in high-noise and/or outdoor environments, the high level of white noise can be problematic in quiet environments

- by default, the volume of the recorded sound is now lower than stock. Refer to the section "Changing the microphone gain" below for more details

- aMGC (see below) is not supported

A new simpler "simple" version of the mod that does not require modification of a system library can be found here

Advanced Fix: Applying moderate noise suppression

After having spent a significant amount of time tinkering with the Android audio stack, I found a way to apply a moderate amount of noise suppression to the camcorder signal.

This new fix makes use of the webRTC noise suppression algorithm available in AOSP (but it was missing the required plumbing). For now, I'm giving you guys two options to play with: 6 dB and 12 dB of suppression.

Note that although 6dB of noise suppression is generally considered as "low", I can still hear some artifacts, but they are significantly reduced when compared to the stock experience. (The stock library adds about 20 dB of noise suppression)

For my taste 12 dB adds too much distortion, but it may fit the bill for some of you.

Keep in mind that webRTC's noise suppression algorithm has been optimized for speech signals and, as a result, other types of signals (music, nature sounds etc) may suffer from some distortion.

To experience the effect of the advanced fix, you can listen to some audio samples here.

Advantages of the advanced fix

- users now have the option to select the level of noise suppression that suits their needs.

I'm personally torn between no noise suppression and 6 dB of noise suppression due to some (barely audible) distortion still present when 6 dB of noise suppression is applied.

- aMGC is supported (see below)

Disadvantages of the advanced fix

- this fix may not work with ROMs other than Stock Android 4.4.4/5.0.x/5.1.x as fundamental changes to the Android audio subsystem (audioflinger) were necessary to feed audio to the built-in effects processor

- there may be problems with this fix in combination with other audio mods (e.g. DSP Manager). Please report back if you experience problems with this fix and other audio mods on stock Android

Known issues

- having either "hotword detection from any screen" or "touch sounds" enabled results in the audio bypass the effects processor, which means that either of these features will disable noise suppression completely!

Installation instructions:

Please refer to the README provided with the attached archive.

assisted Manual Gain Control (aMGC)

The Problem

By default, the microphone setup used for camcorder recordings is tuned to be fairly sensitive. The idea is that distant sources are still recorded with sufficient level. The disadvantage is that loud signals, such as music or screaming kids, will cause a great deal of distortions, thereby rending recordings in noisy environments virtually useless. Android does not offer any way to adjust the microphone gain by the user.

In looking into this issue, I have played around with automatic gain control (AGC) as offered by a webRTC component that ships with Android. I found that AGC as implemented (optimized for speech signals) is not suitable for camcorder recordings as it more often than not causes wild swings of level fluctuations.

An attempt to arrive at a solution

The idea as implemented in this "app" (more accurately a background service) is to show a UI with gain control during a camcorder recording along with a VU meter to assist the user to adjust the gain during a particular recording. I'm calling this approach assisted manual gain control (aMGC).

The attached apk for aMGC has been generated directly from a Tasker project by using the Tasker App Factory app.

A Tasker project that can be imported into your own Tasker setup is also supplied.

Note that the minimum API level for this app is 21. Hence, it is only compatible with Android > 5.0

The app does the following:

pretty much nothing if you click on the icon, as it is a service that starts automatically on boot. However, you may need to start it manually in case Android decides to kill it.

it kicks in whenever the Google Camera launches

it observes the state of the soundcard; if the microphone is opened (i.e. a camcorder recording is started), it displays a small floating and blocking UI in the lower right-hand corner of the screen with which the microphone gain can be adjusted

it uses the LED to display a VU meter, going from faint green (very low signal level detected) to yellow to orange to red (very high signal level detected)

by default it uses the gain as defined in mixer_paths.xml (indicated as 0 dB in the UI). Each short-press on the "down" button decreases the microphone gain by 6dB; a long-press decreases the gain by 12 dB; similarly each press on the "up" button increases the gain by 6 dB and a long-press by 12 dB. Additional gain is limited to +12 dB

Whenever a recording is stopped, the UI disappears and the LED VU meter stops. When the record button is pressed again, everything starts over again with the default values from mixer_paths.xml. Note that it is an Android implementation detail that mixer_paths.xml edits need a system reboot to be applied.

Prerequisites

latest "advanced" camcorder audio mod available here

tinymix (attached; uncompress, copy to /system/bin, and make executable)

NEW: all attached shell scripts, see shell_scripts.tar.gz; unpack, create a directory /data/local/tmp/camera, copy all scripts to that directory, and make all files executable. I had originally thought that Tasker automatically incorporates all shell scripts into the apk, but it does not. Currently, aMGC expects these files to reside in /data/local/tmp/camera . I'll try to fix that in a future revision of the app, by integrating the shell scripts into Tasker.

app needs root privileges

busybox

app needs Accessibility permissions. After installation go to Settings->Accessibility and enable CamcorderMGC. Unfortunately, this permission is required to detect Camera start/stop events. Don't worry, I'm not doing anything nasty

Android >= 5.0

If you are using Tasker (also needs Accessibility privileges) you need to run at least version v4.6. Running the attached Tasker project, i.e. CamcorderMGC.prj.xml, is generally preferred over using the app. Note that you will also need to copy the attached shell scripts to /data/local/tmp/camera when using the Tasker project.

Usage

Hit record and observe the VU meter. If the UI and VU meter do not start you may need to start the app manually as Android may have killed it.

You want the color of the LED to remain yellow/orange. Long periods of red means bad clipping. In that case you may want to lower the gain either gradually (6 dB steps) in moderately noisy environments or more aggressively (12 dB steps) in noisy environments until the LED keeps blinking yellow or orange. Short bursts of red are probably OK. I need you guys to give me feedback on that.

If the levels seem too low after a recording has been made, simply boost the gain by using a decent video editing app on your PC.

Known issues/limitation

Each interaction with the UI takes a couple of seconds to complete. Short-press or long-press the buttons once and be patient for the task to complete. Once completed, the current gain adjustment with respect to the default levels will be displayed.

aMGC currently does not work as expected when hotword detection is allowed from any screen.

aMGC currently does not work when "Touch sounds" are enabled in Settings -> Sound & notification -> Other sounds

Whenever the user changes the microphone gain(s) during a recording a very short click may be audible while the codec (WCD9320) changes the gains as requested. This occurs due to the discontinuity in the audio signal while the gain modification is being applied. Conventional remedies, such as cross-fading cannot be applied as the actual audio data is not available to the aMGC method. For "critical" recordings I recommend running a quick "sound check" first to find the appropriate gain and then quickly apply it at the onset of the actual recording. I suggest to be conservative with the gain as an overall low gain can be easily corrected for by post-processing while excessive gain may result in irreparable distortion.

Installation instructions:

Please refer to the separate README provided with the attached archive.

Aroma installer

Thanks to [MENTION=3163883]spacetaxi[/MENTION] there are extremely flexible Aroma installers available for added convenience.

The latest installer for Lollipop < 5.1.0 can be found here and the one for Lollipop 5.1 here.

The installer for KitKat (now deprecated) can be found here

Confirmed working ROM/kernel setups

stock Android 4.4.4/5.0.x/5.1 with stock kernel

Cataclysm (Jan 23) with Code Blue kernel

Cataclysm (Jan 23) with Cataclysm kernel

CM11 (simple fix only, no aMGC)

CM12 (simple fix only, no aMGC)

stock Android 4.4.x with stock kernel (no aMGC)

Additional mods

In this section I describe mods that don't have anything to do with camcorder recordings, but still appear (or will be appearing) in the modified mixer_paths.xml

Stereo playback

Based on prior work by [MENTION=5108383]sshafranko[/MENTION] (but originally discovered by [MENTION=4170892]dontbelive[/MENTION]) I enabled "true" stereo playback on our device by turning on and tuning the earpiece loudspeaker.

Refer to this post for details.

Sidetone for telephony

Closed-type headphones, noise canceling headphones, and IEM earphones are very good at shutting out environmental sounds. Unfortunately, they also block your own voice. While wearing these headsets you are actually hearing the sound of your own voice via means of bone conduction and resonances of head cavities. It sounds extremely low-pass filtered and it is very difficult to conduct a conversation while wearing headphones. This is where sidetone comes in. Please refer to this post for details.

Using external microphones

A tweak to the main mod is presented here, which allows an external microphone (via headset port) to be used instead of the built-in one. An important advantage of doing so is to be able to utilize a better-quality microphone. (The ultimate improvement in sound-quality can be achieved by using a (mono/stereo) microphone that connects via a good-quality preamp and ADC to the phone's USB port, if supported by an app.)

Additional information

Changing the microphone gain

Background:

The microphone path used by this mod is "voice-rec-mic-mod", which is derived from "voice-rec-mic" already present in the stock version of /system/etc/mixer_paths.xml .

The default microphone gain is set to 97 ("DEC6 Volume"). If one feels that microphone "sensitivity" is insufficient, this gain can be increased significantly. According to ALSA mixer data, the maximum level can be set 124. Measurements have shown that each "tick" (integer increase of this value) adds about 1 dB of gain).

You can actually also modify the "ADC1 Volume" parameter, which ranges from 0 to 19 (and adds about 1.5 dB per tick), but I have not seen any real advantage of modifying this value over the other in my tests. The abbreviation "ADC" (Analog-to-Digital Converter) suggests an analog gain which would result in a better

SNR (signal-to-noise ratio) if that gain is adjusted rather the one associated with DEC (which for sure is a digital gain and thus has no effect on SNR if used inside the signal's full digital scale). My tests, however, have shown no advantage of using the "ADC1 Volume" setting in terms of SNR.

Of course, there is a tradeoff between microphone gain and background noise, i.e. higher gain results in more audible microphone/preamp ("white") self-noise. As a matter of fact it is this noise that the original library tries to fight with its poorly implemented/configured noise suppression algorithm.

Static modification of microphone gain:

- method 1: The gain can easily be changed by editing the new mixer_paths.xml file directly via a file browser, like Root Explorer. The values can range from 0 to 124. Your phone needs to be rebooted for the changes to take effect.

- method 2: run the Aroma installer

Dynamic modification of microphone gain using tinymix:

Once the recording has started, the microphone gain can quite easily be modified with the help of the attached tinymix binary, which can be built from the AOSP tree.

The syntax (after it has been uncompressed), copied to, e.g., /system/bin, and made executable is:

> tinymix "DEC6 Volume" X

where X=[0,124]. Again, there will be a tradeoff between microphone gain and noise amplification.

The big caveat is that the gain is reset to the one listed in the file /system/etc/mixer_paths.xml every time a new recording is started.

If you are running the "advanced" mod, then you may want to try aMGC which helps you adjust the gain depending on the acoustic environments (see the section on aMGC).

Links to the simple mod for CM

Go here for CM11-based ROMs

and here for a CM12 (and CM12.1)-based ROMs

Thoughts on in-call voice quality

As the question regarding in-call audio quality has come up a few times, let me elaborate on my findings:

Most modern smartphones use two microphones when making phone calls. Dual-microphone processing can be very sensitive to the relative position of the talkers mouth and the microphone, in particular in varying background noise scenarios.

Especially when the main microphone on the N5 (the one behind the grille) is far away from the mouth the local talker may be mistaken for background noise and, hence, get suppressed.

As dumb as it sounds, try to hold your phone "right" when making calls, i.e. talk directly into the microphone and try not to move the phone much.

A second very important "good practice" suggestion is to keep the loudspeaker volume as low as possible and try to talk in low background noise environments (I know that the latter one is not always possible). The reason is the rather nasty subject of double-talk.

When a phone detects a double-talk situation, i.e. a loud RX signal combined with a strong TX signal (talker + background noise), the AEC (acoustic echo canceler) will typically start "clipping" the near-end speech, which results the other end not being able to hear you.

Other than these two basic suggestions, there are a couple of things a brave soul can try to fight this issue.

Note there is no (and possibly never will be a) real fix without Qualcomm's and LG's intervention. I'm currently using what's

behind door #2 below with success...

1. Changes in build.prop (not an endorsed workaround)

========================================

consider this change

- persist.audio.dualmic.config=endfire

+ persist.audio.dualmic.config=broadside

What this does is change the dual-microphone algorithm to an alternate one. In my (limited) testing, however, "broadside" was worse than the "endfire" configuration. Actually, I never got many complaints about other people not being able to hear me using "endfire"

However, it may work for you. If it does, great, and you don't need to move on to try other stuff, such as another change:

- persist.audio.dualmic.config=endfire

+ persist.audio.dualmic.config=none

This change turns off dual-microphone processing altogether. The big caveat is, however, is that this change seems to disable other vital signal processing as well, in particular AEC.

Other issues I have seen with these two changes is that the first call after a reboot is often "bad", which can be fixed by quickly switching to speakerphone mode and back. Not good.

2. A very complicated workaround (which works for me)

=========================================

As I mentioned earlier in the thread, I spent a lot of time trying to fix the in-call RX distortion issue when recording from the line.

I found a workaround that works for me that may partially address the issue of occasional low TX level as well. I say partial since low TX level may be caused by both the dual-microphone processing as well as the AEC in double-talk situations. This workaround only addresses (turns off) the dual-microphone processing while still enabling AEC as well as some noise suppression. Low TX levels in double-talk situations can only be improved by Qualcomm, but somewhat alleviated by an educated user (see suggestions above).

The fix requires:

* turning off dual-microphone processing (persist.audio.dualmic.config=none in build.prop)

* custom AOSP kernel (3.4.0-gd59db4e) that allows for injecting proper (?) audio calibration data via Qualcomm's acdb interface, which enables AEC as well as single-channel noise suppression

* a bunch of Tasker scripts

* a shell script

* a binary that commits the calibration data to the kernel/CPU upon a system reboot, or when needed as determined by the Tasker and shell scripts

Ultimately, in finding a fix for this issue, I got stuck at the lack of documentation of Qualcomm's audio interface.

My solution is just a huge hack to address a problem that really can only be fixed by Qualcomm and LG (and maybe Google) as

all processing during cellular calls is done by Qualcomm deep within the MSM/DSP. It is my understanding that the signal processing within that chip is controlled by some of the proprietary vendor blobs.

Some gory details

Up until now, the device identifier used for camcorder recordings in the audio HAL, i.e. audio.primary.msm8974.so, has been selected

as "SND_DEVICE_IN_VOICE_REC_MIC" (while stock uses the "CAMCORDER" device identifier with disastrous quality consequences).

Many people have been observing that the Nexus 5 is unable to produce any useable recordings in loud acoustic environments, in particular

club or concert hall settings. After spending a lot of time doing detective work, I found that the problem can be traced to a high-pass filter (HPF)

being active in either the codec itself (WCD9320) or the main chipset (MSM8974) while most available device identifiers are used, including

"CAMCORDER" and "SND_DEVICE_IN_VOICE_REC_MIC". I'm suspecting two problems with this HPF. One, I think that the HPF itself may be causing

distortions. Second, by filtering out low frequencies, the microphone level estimate that I'm performing inside the audioflinger service is biased in that it

cannot take into consideration the low frequencies that actually hit the analog-to-digital converter, which will eventually cause it to clip.

I found that the "ACDB_ID_VOICE_DMIC_EF_TMUS" identifier in the audio HAL disables the HPF filter. Now the level estimate that is being used by

aMGC is bias-less. Clipping can be properly detected and action can be taken with the aMGC UI to appropriately lower the microphone gain as needed.

An explanation why low-frequency clipping is much worse than clipping at higher frequencies:

In general, clipping (when a signal exceeds full-scale representation of a digital system) causes the introduction of even-order harmonics.

Assume you want to record a 50 Hz sinusoidal signal. Clipping of that signal will cause the recording to not only include a 50 Hz tone (0-th order harmonic),

but also a 150 Hz tone (2-nd order harmonic), a 250 Hz tone (4-th order harmonic), etc. In all, more than a hundred additional tones will show up in your

recording, albeit at progressively lower levels than the 0-th order harmonic. That original pure sine wave now sounds more like a "rectangular wave".

Imagine what happens when a more complex low-frequency signal is getting clipped. The recording is ruined.

On the other hand, a clipped 1000 Hz sinusoidal signals will "only" produce seven additional harmonics in the frequency range that the Nexus 5 is capable

of recording (approximately less than 16 kHz). Don't get me wrong, clipping is bad no matter the frequency (and a professional avoids clipping like the plague),

but the effect is much more dramatic when clipping is happening at low frequencies.

Thanks

Big thanks go to [MENTION=3163883]spacetaxi[/MENTION] for the Aroma installers and [MENTION=2859866]skvalex[/MENTION] without whom I would probably never have looked into any of this.

XDA:DevDB Information

[MOD] [ROOT] Camcorder Audio Quality Fix with Stereo Recording and Playback Support, Device Specific App for the Google Nexus 5

Contributors

chdloc

Version Information

Status: Testing

Created 2015-03-06

Last Updated 2015-03-06