[Spirit FM] mikereidis:
Page 500. Post 4991. :)

Spirit2 Design Details Update Jan 31, 2013:

I haven't updated this document/post since Dec. 19 and some things have changed.

CM appears to be supporting the Android FM API Spirit2 is based on: http://xdaforums.com/showpost.php?p=37434422&postcount=5291

Analog audio will be supported, so no "Digital audio only". But hopefully, audio will be built into CM reasonably soon and I can concentrate on other things.

The API sendExtraCommand() method changes FM state and may be un-suitable for audio configuration, and is likely very un-suitable for transferring large amounts of PCM digital audio data.

More info to come.

Spirit2 Design Details Dec 19, 2012:

Sections:

- Introduction.

- ST-Ericsson proposed Android FM API based.

- Simple API between Spirit2 GUI & Service.

- HAL/FM plugin shim support for audio/other needs via sendExtraCommand() and OnExtraCommandListener().

- Fully digital audio at all times.

- The rest.

Introduction::

This "document" is more for me than anyone else I think. Writing forces me to solidify my thoughts on the design.

But anyone can feel free to comment / question / critique anything or propose ideas. I may not open source all my code, but I'm not averse to "open sourcing" the ongoing design & development of Spirit1 & 2.

My managers & some co-workers have often considered my detailed design documents too long to read, but whatever, it works for me.

Sorry for the lack of explanatory graphics. This isn't a powerpoint presentation for MBAs and I'm not that skilled at graphics.

Feel free to ignore, skim or whatever, there's no essential user info here.

When appropriate, in most cases, my decades of experience has taught me:

A sufficiently elegant design, re-using as much existing work as it makes sense to, can drastically reduce the amount of effort and code it takes to build. On smaller projects, the coding can become almost trivial.

There are projects and areas in which this may not work well (such as the rapid time to market priority for Spirit1), and I've seen projects over-designed to the point of "paralysis by analysis", especially by "too many cooks and no leader".

But good design is very important IMO, especially for 2nd and subsequent major versions. Rushing into coding results in hard to maintain code. It's fine for prototyping but not for "production code" of a decent size project, expected to be maintained and enhanced for a year or more.

ST-Ericsson proposed Android FM API based:

No change. Spirit2's native FM API will be the ST-Ericsson proposed Android FM API, last updated Nov 22 in AOSP.

I think there is still much doubt about whether or not this API will move from experimental in AOSP to supported / approved / blessed / recommended by Google. I suspect the major chip and phone OEMs will have to agree to it before it will happen.

Further, ST-Ericsson is again having a new round of troubles related to monetary losses and lack of success in mobile chipsets. About a year ago, further development of this API was planned to be moved to India, but it's not clear what happened.

Regardless, this API is the best chance to encourage a standard FM radio API for Android devices. Spirit1 supports 11 different APIs now (including this one). Spirit2 will effectively just support this one API, which will give me the time and clarity of code to support more advanced features.

Multi-API support will still exist, but will be confined to API specific plugins that will be built by myself, where needed, or can come from 3rd parties. I've already been able to use a 3rd party plugin for Qualcomm WCN3660 support that Sony uses on Xperia T, and it works pretty well on my HTC OneX LTE. The details of how to build plugins are openly documented as part of this proposed Android FM API.

At present, the only plugin on the drawing board for me to build into Spirit2 is for the Samsung - Silicon Labs FM chip driver, usually accessed as /dev/fmradio . (Although CM kernels IMO incorrectly use V4L naming of /dev/radio0 for this Samsung specific Silicon Labs si470x FM chip kernel driver). Other are/will be considered, such as TI's /dev/tifm (but TI is getting out of mobile), or Broadcom via HCI or /dev/ttyHS0 (but there are no particularly significant high end devices at present using Broadcom for FM.)

Note that Spirit2 will not support proprietary high level OEM APIs as Spirit1 did, particularly for stock, un-rooted devices. The primary target is custom ROMs, although in many cases, it will likely work on rooted stock devices, using low level APIs.

The choice of this API also means it will run well on stock, un-rooted Sony 2012/2013 devices that use Radio3.apk. I don't expect to support these officially however, unless perhaps Sony releases some solidly competitive devices in 2013, with wide adoption. Seems horrible to "waste" this "free" capability, but, we'll see...

Simple API between Spirit2 GUI & Service:

FM radio apps and music players contain 2 major components: a user interface that the user sees when the screen is on and the app is in the foreground, and a background service that maintains the audio and other functions, even when the screen is off or another app is in foreground.

Note that the FM App Service is distinct from a System FM Service that may be running in the background. System FM Services provide a Service API to clients such as the FM app itself, or to other apps, widgets, screen-lock apps, wallpapers etc. A System FM Service acts as an intermediary between apps and low level kernel drivers.

In Spirit2, there WILL be a System FM Service that, for convenience, may be contained in the single common APK file used for the app. But a System FM Service COULD be placed in a separate APK file and will be running in a different process.

In Spirit2, the "System FM Service" is built from open source Android code in AOSP, with a small wrapper added to turn it into a "user level Android service". I've done this and it works well, able to run multiple FM app instances simultaneously. I will likely openly release this small bit of wrapper code, as it may assist others and is relatively trivial.

Note that the proposed Android FM API is conceived as a system level function, which is appropriate for Android. This means that a mere user level app & service, even with SU/root, can not implement it "properly", especially with regards to the dependencies on the Android audio system.

"Proper" support for this API, and the Android audio extensions required, most be baked into the ROM. On stock ROMs, OEMs such as Samsung would have to support this API. Even if phone OEMs do eventually support this API, they are less likely to back-port it to previous device generations.

The same applies to custom ROMs such as CM. But there don't seem to have been any noteworthy statements of coming CM support for FM, much less support for this new FM API.

And I think getting into the "business" of modifying & building custom ROMs is too much of a distraction from my core goals. So I will continue to effectively create what should be system level services, for FM and FM audio, from user level code enhanced with SU/root access.

An app that uses the official system level API, such as Sony's Radio3.apk, will not be able to work with Spirit2 on non-Sony devices without modification. Spirit2 will support a very slight variation of the proposed official API, mostly for initialization. But an app that's written to an official API, or the Spirit2 slight variant, will be almost trivial to modify in the source code to support both. One could probably even modify Sony's Radio3.apk, at the "smali" level, to work on Spirit2 supported devices.

Ideally, chip & phone OEMs and custom ROM devs will adopt the proposed Android API and build the needed support into their ROMs. I will openly release at least strategic parts of the source code, such as a simple app, in an effort to encourage adoption in custom ROMs.

Back to FM radio apps having 2 major components: a user interface and a background service. Almost (all ?) every FM radio app arranges the UI and service hierarchically. For example, a preset is selected and the UI code calls the app service, which calls the System FM Service, which calls the lower level drivers.

Spirit2 will not use this hierarchical architecture for most functions, and only a relatively simple API will be supported between the app UI and app service, for such functions as starting, stopping, and updating preset lists.

For most FM functions, such as setting frequency, the UI will directly call the System FM service, bypassing the app service. This has the advantage of not requiring the design of a complex private API between app UI and app service. The app service will be called back by the System FM Service when frequency changes, so a frequency set API is not needed between app UI and app service. (Not in present ST-Ericsson API, but their notes show they know it's needed, and I'll add it to my variant if they don't.)

So, to a large extent, there will be little "coupling" between app UI and app service. They will each act the same as if they were separate FM apps running simultaneously, without interfering with each other (if written properly). There is no reason that 2 very different FM apps couldn't be running simultaneously, each with one or more app UIs, and likely one app service (each). There should only be one System FM service, but there could be more than one app service.

Note that there are many things that an app service can do that may not be obvious, that aren't done by the GUI. They are the things that can be done while the screen is off. The Spirit1 app service can change frequencies, for AF or media button channel selections, can send RDS info to a notification window, can manage volume changes, and turn FM off with the sleep function.

HAL/FM plugin shim support for audio/other needs:

The ST-Ericsson proposed Android FM API provides a "plugin" mechanism to support different FM chips and APIs. This is a Linux shared library, such as libfmradio.iris.so for Qualcomm WCN3660 / 3680 FM chips. Code for these libraries is generally written in C or C++ and built using the Android NDK.

The "old way" for Android FM is that chip OEMs, such as Broadcom and TI, write proprietary Bluetooth and FM "stacks" for their chips, and provide sample apps to the phone OEMs. Phone OEMs, such as HTC, then write their user level FM apps on top of these proprietary APIs. Some phone OEMs end up getting kind of "stuck" with those chips and the associated code, and it can be hard to move to other chip brands.

Some will build their own phone OEM proprietary translation layers/stacks, as Spirit1 did, in order to accommodate different chips, but the result can be a lot of difficult to maintain code.

This "new way" allows chip OEMs to simply provide a new plugin for their FM chip, and the phone OEMs are no longer stuck with one chip supplier. This may help explain why "under-dogs", such as ST-Ericsson want to encourage open APIs with simple plugins, while established players, like Broadcom, may want to hold onto a system that enhances vendor lock-in.

One of the short-comings of this FM API, and especially the plugins, is that it mostly just covers FM chip functions, such as frequency and RDS. Audio functions must be implemented in Android system code (from Google/AOSP) and the audio HALs; the libraries that convert standard Android system/internal APIs into audio system specific APIs. (Eg. audio.primary.exynos4.so)

I first considered creating a new "Spirit2 proprietary" (but documented) audio system plugin mechanism for FM audio. But I now think that the Android FM APIs designed to extend OEM specific functionality should be used. Functions, such as "Enable FM on headset" can be sent with the sendExtraCommand() API, and results can be returned with the OnExtraCommandListener() callback.

So when I create a libfmradio.samsung_sl.so FM chip plugins for GS3 etc., I can add the extra audio functions to the standard FM chip functions. But these functions can't work when directly using an existing 3rd party FM chip plugin, such as the 3 known existing Sony plugins. But I think a "shim" or "adapter" model can be used.

For example, the Android System FM Service (Spirit2 variant), would link to a libfmradio.audioshim_wm1811.so library that handles the audio functions. FM chip control functions (& callbacks) would mostly be passed unchanged to (from) the libfmradio.samsung_sl.so FM chip plugin, which then deals with the low level FM driver /dev/fmradio .

Fully digital audio at all times:

The last major piece of the puzzle I've settled on, so far, is the idea of making FM Audio 100% digital, all the time. It seems that it could provide a very elegant means of solving the major FM audio problems. And it provides a very easy way of supporting digital audio features, such as FM recording / Tivo like PVR / PARing (live pause) and support for A2DP stereo Bluetooth headsets. (And AFAICT, there is not a single Android FM app that supports BT now, and recording is rare.)

FM audio has been a major headache for me. The first test release prototypes of Spirit1 in March 2011 worked OK on my HTC Legend, but without audio. I knew I would likely eventually figure out the audio problem, but I wanted to have a functional app once that was figured out, and the TI FM chip secrets seemed easier to initially crack than the audio chip secrets.

I was new to Android (but not embedded Linux) at that time. But still now, many audio issues plague Spirit1 on various phone & ROM combinations, and they are effectively unsolvable given my limited time and my focus on the app, and not custom ROM library modification. And AFAICT, there is very little public documentation on low level Android audio details. The open source Android code is available, but code alone is not proper documentation. Whatever documentation exists, appears to be limited to phone and chip OEMs, AFAICT.

Traditionally, Android FM audio does not pass through the audio system the same way as all other audio, which is digital. Usually, there are analog outputs from the FM chip to the audio chip, or system. The audio is injected late in the audio processing chain, near the end. FM audio is like an after-thought that was "bolted onto the side" of the existing audio system. Advantages may include simplicity and lower power consumption.

Disadvantages to using late injected analog audio include: no inherent ability to record or digitize audio for other reasons. Un-digitized audio cannot be equalized or otherwise processed by DSPs, can't be used for VU meters, spectrum analysis, visualizations or song recognition, nor can BT headsets be supported.

Fortunately, recent high-end devices are now supporting FM audio digitization, in hardware at least, using either ADCs (Analog to digital converters), or better yet, direct digital connections from the FM chip. Direct digital connections are preferred because most (all ?) FM chips do FM audio demodulation, de-emphasis & other processing digitally, internal to the FM chip. It's better and easier digitally. So to me, it almost seems a "crime" to convert this digital audio to analog, then convert it back again to digital. Each additional conversion adds noise and distortion to the audio. And considering the full chain, from original singers' microphone to digital CD to analog and digital conversions at the radio station, we IMO often have too many conversions happening already. Even the GS3 uses an si4705 chip with digital output, but sadly Samsung seems to not have connected it. Hopefully GS4 & Note3 etc...

Recent, high-end devices are the target for Spirit2, and since the most important/all of these all seem to support FM audio digitization (or direct access to digital FM audio, such as Qualcomm WCN3660), it becomes logical to make digital FM audio capability a requirement for Spirit2 supported devices.

If a device is capable of digital FM audio, then the audio can be kept in the digital domain at all times. I will prototype this to see how well it works. Higher power consumption could be an issue, as could noticeable audio quality degradation due to an added pair of analog/digital conversions. An analog bypass could be implemented if needed.

The biggest audio problems that Spirit1 has faced have been due to the custom ROM audio libraries not knowing or dealing with FM audio. If they provided FM audio control functions, such as the CM7 FM audio APIs, Spirit's job was easy. Otherwise, Spirit1 has to perform often undocumented low level functions to enable and support FM audio.

Spirit1 supports MANY audio "tricks" or hacks to deal with FM audio, including Debug-> Silent Loop, Play Audiotrack, Power Restart Media, etc, etc. The biggest problems can happen when phone calls, audio notifications or system sounds, such as camera clicks, happen. Spirit1 put the audio system in a state for FM audio, but the ROM audio HAL library doesn't know about that and bad stuff can happen. Desire HD and similar are particularly trouble-some and in some cases one must reboot the phone to get normal audio running again.

People have often reported an audio problem specific to Spirit (specific to FM audio on that ROM actually) that doesn't happen with streaming or other audio apps. I explain how FM audio is different and try to offer various hacks, none being perfect.

But if all FM audio is always digital, a huge number of problems become automatically solved. Booyah ! :) Spirit2 can work just like any other audio app. It calls the functions that accept digital audio, and everything "just works", like it does for streaming apps. Volume control: handled by the ROM libraries. Hacks to keep the audio chip running: not needed. Hacks to disable FM quicker than the audio library can mess things up: not needed. DSPs, equalization, etc: fine. A2DP Bluetooth headset output: no problem. (I originally thought I'd have to write code to send to BT myself.) In many cases, external apps, locks-creens, widgets, wallpapers etc. can do all the fun stuff they do with other audio. It may even be possible to combine these advantages with an analog bypass, if that provides better audio quality.

Spirit2's audio shim (where the FM audio stuff is done) would have to do low level initialization and reading of a PCM device to get the digital audio, as it would have had to for recording & other digital audio stuff anyway. And there is the possibility that this could interfere with an audio library unaware of what Spirit2 is doing. But I think (& hope) this shouldn't be too much trouble; dealing with this small area seems infinitely preferable to the Spirit1 methods. As I've said before, if I have to, I'll consider writing audio libraries, or "shims" at least to deal with this.

The rest:

I've seen people starting a band debate extensively over the bands' name. I'd worry about the music first.

That's why you won't see any fixed UI design here, as someone who "just wants to make an app" might. Hundreds of hours of low level work is required to make FM recording work, before anyone should worry about where or what the record button should look like.

As stated elsewhere, Android 4.2 Jelly Bean seems to be fine for a minimum Android version level for Spirit2. Spirit2 is intended / targeted for custom ROMs only on the latest devices. And the latest versions of Android have intriguing new audio capabilities (or potential) which may allow much better control of complex modern mobile audio systems.

A slight variant of the existing main screen UI of Spirit1 will likely be built into Spirit2 to start at least. I may create additional alternative UIs, and I hope some talented UI designers can use the proposed Android FM API to create interesting apps.

Spirit2 will try to "do things right" where Spirit1 was hastily built to get a functional FM app out quickly. Spirit2 will endeavour to avoid polling (repetitive checking to see if something has changed) and use appropriate callback and threading mechanisms.

As much as possible or feasible, Spirit2 will not perform long or blocking operations on UI threads. This will minimize the chance of ANRs, even under error conditions. Pressing most buttons will display some feedback, launch a thread and return. The thread will take care of the possibly long or blocking operations, and may use completion callbacks for longer operations. When the thread or callback completes or times out, the displayed feedback will be updated to show this.

At startup, the Spirit2 app UI will check to see if the app service is started, changing state, or in some error condition. If none of these are true, the app service will be started and the app UI will display a status message while waiting for the app service to complete startup or to timeout or encounter some error. The status message may contain counters for elapsed time or an estimated startup countdown to 0.

If at startup, the app service is started, changing state, or in some error condition, the UI will reflect this and do as above, except skipping the app service startup. When the app service completes startup, or when an error or timeout occurs, the app UI will update to indicate this.

These startup rules are intended to avoid the problems Spirit1 can have at startup, especially when screen orientation is changed during startup. This means, for example, that starting the app and quickly switching away from it (or changing orientation) will allow the app service to continue startup, even though the app UI is effectively dead. This, along with some app service rules, will allow external apps, such as alarm clock type apps, to start Spirit2 FM.

It is likely that a configuration item will control whether or not FM starts automatically when Spirit2 starts. The initial default will be to not start FM, to minimize the chances of crashes before the initial configuration can be changed. Initial startups will display an Introduction dialog, as does Spirit1, but with added options for trouble-shooting, configuration etc. A guided and semi-automatic approach may be taken for such settings as Regional Band, Alternate Frequencies and any setting that is commonly changed from default.

Because official support for Spirit2 will be limited to a smaller number of devices, it becomes feasible to rely less on "auto configuration" based on what files are found and other clues, and more on the "ro.product.device" which identifies the device. Spirit2 should be able to self-configure everything needed for at least minimal operation. But it's impossible to know ahead of time what bugs or variations Spirit2 might have to accommodate on various releases of various ROMs.

Debug logs will be supported, but with an emphasis on utility, and perhaps some trivia questions to drastically reduce all the accidental debug logs I get from pirates etc.

I expect at this time to only release one version of Spirit2. Multiple versions (Free, Light, Unlocked...) are work to maintain and I'd prefer to work on features. I may freely release Spirit2 on XDA at least, along with a paid/"donation"/"support included" version on Google Play. We'll see how that goes.

I may ban free redistribution on ROMs. But I may allow it if I can find a way to allow it, without interfering with someone purchasing it on Play. Again, we'll see.