STM32 USB Audio (UAC) -> I2S buggy drivers - Page 1
Forum section: electronics / general

author: paulca
author_rank: Super Contributor
timestamp: November 13, 2022, 12:19:56 pm
content: STM32F411CE Black Pill + PCM5102 module.STM32CubeIDE ...enable USB_OTG_FS, enabled Audio device class, device only. Set sample rate to 48000s/s.enable I2S2 in half duplex with Master Clock output.Fiddle with the I2S PLL multiplier/dividers to get as close as you can to an I2S clock speed of 48K. Managing I believe the closest is 48.065K 0.49% of something off.Then.... the documentation stops. Most of the main line documentation either pretends the whole "Device library" doesn't exist or instead gives you a 150 page lecture on the theory of lower level USB and nothing about how to use the library. Their Wiki entry on USB is one of the later. It has all the theory about the low level USB protocol, a tiny bit about the actual device classes behaviour and absolutely nothing on the library, except for a list of functions in each "class". No explanation, no code examples. Sorry there are two sample projects which they throw links for at you. Both are hacky, half written examples for Discovery boards. Both use "BSP" Board support packages, which look like Macro soup, but importantly are setup bespokely for the boards hardway. They literally wrote functions in the BSP to do everything for you and then they say, "Here! This is how to do it." Non-sense.So blind, I trawled google, github, wikkis and then spent an entire evening in the debugger stepping through library code to find what actually happens in there and how to get Data out. It's a warren of badly managed C code. Almost all of the typedef structs are anonymous, so CTRL+Click doesn't work. CTRL+SHIFT+G to find references (say as to where a call back gets called), either don't work at all or in fact... after a text search through the entire framework... is NOT called anywhere.Finally. I found it's (apparently) so simple they don't need documentation. There is a single function called when an Audio event occurs. Audio event is either, start, play or stop. The circular buffer is already provided for you, it's all abstracted away in a sea of casted function pointers that look like someone trying to write C++ code in C. Structs of function pointers = static class.So blindly I stuck an HAL_I2S_DMATransmit() in the START case and low and behold sound came out! After fixing the word alignment and buffer size it even sounded like music for about 1 milisecond. Setting the DMA to circular and we actually had music. Sounded brilliant until "sqqeellchpop", then back to normal. Then another half second or so of distortion.I tried removing the circular DMA and respawning it in the "PLAY" state, I tried all manor of things, but I could not get it to not distort every 10 seconds or so. Another long few hours trying to randomly read around the problem, checking other peoples git repos etc. etc. I finally came to the realisation my distrust for opague buffer handling code and "its all taken care of for you" code, was valid and it's a framesync buffer over/under run scenario caused by the clock rate skew between the USB host 48K and the I2S PLL's 48.065K.Then the penny dropped the call backs they provide are not for their library to call "you" but for you to call back to it as "your" view of the buffer changes. HALF complete, FULL complete callbacks call a function called "USB_AUDIO_Sync()".I quickly implemented that, but no, it still squelched periodically.I spent quite a while debugging through it. Obviously with a stream like that as soon as you hit a break point it all crashes to a halt and may or may not start up again when you hit resume. But with many, many, many restarts I did as much as I could to confirm it's a DMA buffer colision and the audio sync method doesn't work.Debugging it and trying to understand what it is doing and I came to the conclusion that it's not doing what it should be and probably not even what the developer intended it to do. While the concept of frame stuffing and dropping isn't that contraversial, the fact they don't actually fill the extra frame with anything is, as is the fact the state machine it runs is completely broken on one side and ineffective at best and a libility to crash on the other. When they "add a frame" they just add 2x16bit length to the DMA buffer.... which is declared as a fixed array, so they literally over run off the end into probably the ->rd_ptr or struct padding.Here is the function. I think this needs to be redone somehow. In my application, going forward, I want multiple discrete I2S devices exchanging synchronised streams. I cannot change the master clock to use the USB. So I either have to fix the frame stuffing or try one of the other techniques, like measuring the delay with a timer and programmatically inserting and removing actually populated frames at the end of the buffer. There is also the approach of providing feedback to the USB Audio stream. I believe Synchronous (or Isochronous?) mode allows the device to ask the host to modify it's clock. Again... I can respond to the USB SOF (start of frame) interrupts and try and so something there.Not least the comment is for the function above, some comment copy pasta is nice to see.The main thing to notice is that in the event of a OFFSET_HALF the BufferSize variable is unused and discarded. It is during the OFFSET_HALF callback that it detects the buffer overrun WILL occur, then it does nothing about it. In the OFFSET_FULL callback it does change the buffer size and resets the pointers to 0 and the stream returns to perfect.... until it goes the other way, when it underruns due to inactive at the half way callback and this repeats.Code:/*** @brief USBD_AUDIO_SOF* handle SOF event* @param pdev: device instance* @param offset: audio offset* @retval status*/void USBD_AUDIO_Sync(USBD_HandleTypeDef *pdev, AUDIO_OffsetTypeDef offset) {USBD_AUDIO_HandleTypeDef *haudio;uint32_t BufferSize = AUDIO_TOTAL_BUF_SIZE / 2U;if (pdev->pClassDataCmsit[pdev->classId] == NULL) {return;}haudio = (USBD_AUDIO_HandleTypeDef*) pdev->pClassDataCmsit[pdev->classId];haudio->offset = offset;if (haudio->rd_enable == 1U) {haudio->rd_ptr += (uint16_t) BufferSize;if (haudio->rd_ptr == AUDIO_TOTAL_BUF_SIZE) {/* roll back */haudio->rd_ptr = 0U;}}if (haudio->rd_ptr > haudio->wr_ptr) {if ((haudio->rd_ptr - haudio->wr_ptr) < AUDIO_OUT_PACKET) {BufferSize += 4U;} else {if ((haudio->rd_ptr - haudio->wr_ptr)> (AUDIO_TOTAL_BUF_SIZE - AUDIO_OUT_PACKET )) {BufferSize -= 4U;}}} else {if ((haudio->wr_ptr - haudio->rd_ptr) < AUDIO_OUT_PACKET) {BufferSize -= 4U;} else {if ((haudio->wr_ptr - haudio->rd_ptr)> (AUDIO_TOTAL_BUF_SIZE - AUDIO_OUT_PACKET )) {BufferSize += 4U;}}}if (haudio->offset == AUDIO_OFFSET_FULL) {((USBD_AUDIO_ItfTypeDef*) pdev->pUserData[pdev->classId])->AudioCmd(&haudio->buffer, BufferSize, AUDIO_CMD_PLAY);haudio->offset = AUDIO_OFFSET_NONE;}}

author: paulca
author_rank: Super Contributor
timestamp: November 13, 2022, 12:52:29 pm
content: Part of me has already been raiding my AliExpress account and ordered a few dev board for chips like the PCM2706 which at least provide a hardware USB->i2s device. To be confirmed is their slaving to a master clock and performing the USB feedback/frame syncing.Similarly I've been having issues with the ESP32 and it's bluetooth to I2S stuff. Not giving up on it yet, but I did also order a BlueTooth to I2S dev board, can't remember the RF daughter board IC name now. If they work, there is something to be said for handling discrete handware devices with clear data sheets over hacky library code in software you have little documentation for.

author: DavidAlfa
author_rank: Super Contributor
timestamp: November 13, 2022, 01:25:49 pm
content: Check:https://github.com/deividAlfa/STM32F411-Black-pill-USB-WAV-MP3-player

author: paulca
author_rank: Super Contributor
timestamp: November 13, 2022, 07:32:10 pm
content: Thanks, but that's USB Mass storage, reading files to play back. I need stream handling and USB Audio.

author: PCB.Wiz
author_rank: Super Contributor
timestamp: November 13, 2022, 08:49:09 pm
content: Quote from: paulca on November 13, 2022, 07:32:10 pmIt may not be much help, but a while back I played with Audio on the PC side, and it was a royal pain to avoid unwanted clicks and artifacts, until I changed from a 2 buffer ping-pong system to a 4 buffer quadrant design.The 4 buffer design is just a small step up from ping-pong and works like a clock, with 4 areas/quadrants and the pointers are checked so the read pointer is always 'diagonally opposite' the write pointer.That means read is always on a clean buffer and the next buffer is also clean, and the read can fast burst across the whole buffer.This keeps the pointers 'well out of each others way', and seems to solve the unexpected artifacts.It may be that the 'clean block' approach is better suited to flows with buffering and OS delays in the mix.Maybe another way to look at this, is the inherent extra valid pipeline data in this scheme, avoids under-run issues and is more tolerant of 'flag jitter' effects ?

author: paulca
author_rank: Super Contributor
timestamp: November 14, 2022, 10:37:04 am
content: Yea, USB audio can be a bit weird. It depends on the source as to how much it is willing to buffer for the device and what rate it's willing to send it. There are modes designed for listening to music which is sourced from a file and allows the device to just ask for as much data as it likes at whatever rate it wants. Obviously that isn't going to work on a live streamed video or similar "live" realitime stream.For the later, which you have to assume all of your streams will be, because they can be. We have to avoid long buffers. Buffers in live realitime streams are only there to buy you time, time enough to realign/sync the stream etc. Holding data for longer doesn't really help, it hinders with latency.I did a bit more digging around in their code. I can see the other avenue might be available to me. Responding to SOF (Start of Frame) sync messages. In there I have a 1000Hz timer basically originating from the host. On triggering that event handler, I can check the read pointer and write pointer and determine if I'm gaining or lossing on the stream and by how much. Just count the bytes written since last SOF (should be fixed) and the count number of bytes read since the last SOF. If the buffer is dissappearing, send a new clock rate request to the host.The maximum the F411 supports is USB_FullSpeed and 48K sample rate. I want 48K data. So there is a bit of a problem. I can't speed the stream up any. I can only slow it down.The plan is to choose the I2S PLL multiplier to give me the lower of the two closed rates. I think the upper is 48.065K the lower is like 47.851K.First off, I can see if the PC will accept a sample rate of 47851 s/s in it's initial setup. If not I will assume my buffers will always be moving towards overrun. When I detect this is imminent I can send a new rate request to the other end. Say, 47.7K and then wait for it to move towards underun, when I can send a new clock rate request for 48.000K again.So my buffer will always be moving towards overrun.On USB Audio and latency. For playback of other sources the USB Device spec (some version) supports the ability to report the device latency back to the host and it will adjust it's A/V buffers to remove the latency if it can. I believe I seen mention of support for this in the STM32 docs, but I wasn't paying much attention on the way past. You are more likely to notice this behaviour on BlueTooth headsets/phones. While watching a YouTube video, connect your BT headphones. In my case the instant the audio starts on the phones the youtube video glitches very slightly as it stalls video playback for however many milliseconds the headphone latency is, to realign the A/V sync.

author: paulca
author_rank: Super Contributor
timestamp: November 14, 2022, 10:55:39 am
content: Trying to stay forward thinking and where this project is ultimately going.Another option is to take the master clock duty away from the STM32 and give it to another device that has a properly aligned audio rate crystal in it and will produce a near perfect I2S clock at 48K.I will still need to address the drift with will always occur even with an expensive 10ppm xtal or better. It's just a matter of time and with short buffers that's not long at all.For the final project I will either be investing in an expensive audio xtal for the master clock OR I will invest in an I2S Audio Controller chip as the master. Depends on whether that will be overkill and what other features it provides asides providing the master clock (or clocks).I have a ADC module to test which has it's XTAL MCK output, it has something like a 25.145Mhz xtal. I'm going to assume the clock rate is sensibly choosen for each divison for 192/96/48 KHz clocks.

author: PCB.Wiz
author_rank: Super Contributor
timestamp: November 15, 2022, 12:06:04 am
content: Quote from: paulca on November 14, 2022, 10:37:04 amSurely the PC side behaves much like any UART, so far as USB data blocks are concerned ? - ie the OS accepts the BAUD value, but the data flow has no idea what the actual pin baud rate is, and the end device is the only item with a precise timing.I coded/hacked a USB UART once, where the driver had a SW imposed bug/ceiling and refused to pass on values > 4Mbd, so I simply trapped values < 6, (which it was happy to pass on), and took those as /N for 24MHz/N Baud choices.That worked just fine, the PC side number appeared to be 4 in the Baud value, but data flowed at 6MBd.

author: paulca
author_rank: Super Contributor
timestamp: November 15, 2022, 09:31:59 am
content: The USB VCom port that the STM32 HAL gives you has endpoint feedback, so if you use it as a UART you would be surprised when the other end pushes back and say, "No thank you, I'm stalled.". So if you create a VCom port as a debug output and then don't connect a serial terminal to it, the CDC write function will block eventually.USB Audio Class has three sync modes. Asynchronous. Host has a clock, device has a clock, it's up to them. Synchronous, the host has the clock, the device is expected to synchronise with the host. (The preferred option if you don't need to sync with anything else and can let the USB be the master clock. Or... Isochronous which is Asyncro but with feedback. The host notifies you ever 1 millisecond with the "SOF" event. Start of Frame. In that event you can respond with advice for the host regards it's rate etc. In the USB spec and lower level docs it's referred to as "Endpoint feedback". The end point in those terms is the bit of the driver handling the buffers more or less, it's where the data goes.The project is back on the rotation list, I'll probably get back to it an evening this week. Other projects to move forward today.