by jauntywundrkind | 2025-05-20 20:18:34+00:00
Fun coverage of what all goes in to such a fascinating compelling topic, that so so so few people have any real idea about (computer × audio pipelines). I'd like to run through the major players as I see them right now: Sound Open Firmware and then the ever present widely adopted Cadence Tensilica Xtensa HIFI IP that's on a good number of systems-on-chip. Then gaze forward a little.

Based on the whose who of who contributors/users (Intel, MediaTek, Realtek, AMD (although it seems they dropped the hardware for it after Zen3?), my impression is that Sound Open Firmware is the 900 lb gorilla in this area. There's a ton of work & capabilities poured in here. No one wants to own this and differentiating yourself is extremely hard. So cooperation makes sense. So: SOF; started 2018 & quickly picked up by the Linux Foundation. With the 2.4 release (January 2023) they switched to Zephyr as the base embedded OS, which seems like an awesome win to offload development efforts & perhaps to prosper from subsystems like Bluetooth support. https://archive.fosdem.org/2024/schedule/event/fosdem-2024-2... https://thesofproject.github.io/latest/introduction/index.ht... https://www.phoronix.com/search/Sound+Open+Firmware

SOF is a software consolidation effort, and has done great. The somewhat wilder side is the hardware. Not my field but again my impression is that, at this point, Tensilica (acquired by Cadence in 2013)'s Xtensa IP is basically used by everyone, often the incredibly featurefully integrated DSP rocking HIFI 4/5/5s/1. Tensilica's Xtensa is famously used in Espressif ESP series (and now with RISC-V designs). But Tensilic (mostly "LX"?) cores are everywhere, almost always explicitly for audio (or vision, in the case of Hololense) processing. With nice chunky DSPs and now coming with 32 bit float, nicely parallelizing/auto-vectorising with Cadence's proprietary XCC compiler, this is everywhere. There's HIFI4 and HIFI5 (and now 5s) cores in a wild number of chips: almost all modern Intel laptops/desktops, to tiny little Cortex A7 chips to tiny little TWS systems. Qualcomm has some Xtensa but I'm not sure where. For example there's a great Defcon talk on hacking ath10k wifi, which is Xtensa based! https://media.defcon.org/DEF%20CON%2031/DEF%20CON%2031%20pre... https://www.cnx-software.com/2022/02/13/allwinner-t113-s3-du... https://semiwiki.com/forum/threads/wuqi-microelectronics-sel...

Cadence has the DSP market down right now, seemingly.

Thoughts on the frontier/future: I'm interested to learn what AMD did after walking away from TrueSound hardware (having a HIFI block on their designs). Ther was work they were trying to pursue GPGPU efforts, and maybe on APU's the latency wouldnt be an issue, but I sort of expect its mostly software now, and I'd love to know more about what that software.

SOF has had years of wanting to get more open. But that XCC compiler seems like a so-far holding moat, and I'm not sure who else is even trying to have more DSP like instructions or cores to target the market. SOF has been trying to get running on ARM or x86 or RISC-V for a while, but without some vectorization and nice low power parallel hardware it's hard to hope for too much. I keep hoping RISC-V RVV vectors might maybe do the thing. But how much success folks have had with the plans to get llvm doing vectorization is very unknown.