Custom Audio DSP with USB and I2S or TDM - Page 1
Forum section: electronics / projects

author: AaronD
author_rank: Frequent Contributor
timestamp: September 21, 2021, 07:40:17 pm
content: Looking at this or similar, but not married to anything in particular:https://www.microchipdirect.com/product/ATSAM4S16CB-ANI think my biggest problem will be to graduate up from PIC16's and ATmega's with all custom code and no libraries*, to something that can handle the throughput that something like this requires! (I looked briefly at a PIC16F1454, but when the maximum instruction rate is the same as an external bit clock...ehh, no.)(*It took some effort to convince the PIC port of SDCC to not add the C-standard init code, but go directly to my main() function instead. Turns out you just name it something else so that there is no main(), and then map an inline assembly function at the reset vector to long-jump to wherever the linker decides to put your renamed_main(). Then YOU are responsible to initialize everything...or not as the case may be.)Is the above a good chip to pick? Is there a better one? Why for both of those questions? What (free!) toolchain and possible libraries do I need, considering the possibility of some commercial projects later on? (GCC and Code::Blocks would be nice, as I already know them from some AVR projects, but others would be okay too) How to manage a DMA? Etc.I want to start with this block diagram, just to get my feet wet, and I have a project that can use it anyway:+------------------------------+| USB Composite Device | +----------+ +------------------+| - 2-ch Audio |------->| DSP |------->| 2.1 Speaker Out || - HID for custom control app | +----------+ +------------------++------------------------------+ ||+-------------------+| Physical Controls || - Potentiometers || - Encoders || - Buttons || - LED's |+-------------------+It's pretty much what you'd think: a stereo-input USB sound card with EQ and crossover to stereo tops and a mono sub, plus some direct physical controls. If it weren't for the physical controls, I could just buy that instead of making one, but the bigger purpose is to learn the building blocks to then do this:+------------------+ +----------+ +----------------------+| Stereo Mic In |------->| |------->| Stereo Line Out |+------------------+ | | +----------------------+| |+------------------+ | | +----------------------+| Stereo Mic In |------->| |------->| Stereo Speaker Out |+------------------+ | | +----------------------+| DSP |+------------------+ | | +----------------------+| Stereo Line In |------->| |------->| Mono Line Out |+------------------+ | | +----------------------+| |+------------------------------+ | | +----------------------+| USB Composite Device |<------>| |------->| Mono Speaker Out || - 8-ch in / 8-ch out Audio | +----------+ +----------------------+| - HID for custom control app |+------------------------------+That's a total of 6 channels in and 6 channels out of analog audio (counting amplified speakers as analog to allow for digital-input class-D amp chips), for either 3x I2S streams between the ADC's and DAC's/amps, or 1x TDM-8 for all of it.I2S looks similar enough to SPI that I might be able to use a SPI peripheral to fake it? And TDM is based on I2S with the same similarities, so maybe I can still use a SPI peripheral for that? The bit clock of TDM-8, 32-bit, 48kHz, is just a hair above 12MHz, so it'd have to be a pretty fast chip compared to what I'm used to, just for that.The analog-to-analog signal path of this one has a very tight constraint on the overall latency - there are two mic-to-speaker paths in there, with just a handful of samples' worth of distance at 48kHz through some foam, and the DSP's purpose is mostly to undo the foam's acoustic lowpass (plus some transducer correction and limiting) - so I think I need to write my own transport and DSP code to operate on single samples instead of a block of 64 or so like I see being common, just to have the acoustic and electronic wavefronts line up. I think I'm okay with writing the DSP code; from what I've seen on Wikipedia and some dedicated DSP coding sites, that part should be easy once I get a new chip up and running in the first place.USB has so such requirement: it's pretty much a monitor for recording/debugging at several different points simultaneously along the processing chain, so it can fill up a buffer and send it off whenever it gets around to it. Likewise in the other direction, it can get a buffer from USB and then play it out. That's fine; the critical latency doesn't apply to this connection.And like the diagram says, drop the physical controls for this one. But I might add some other functionality, unrelated to audio, depending on the amount of CPU time and other resources that I have left over.For both projects, I think I want the DSP chip to have the master clock (and give it a good one!), and everything else slaves to that, including USB, which would then need to run its audio interface in "asynchronous" mode. Don't need any options for USB Audio: just 2 or 8 channels (depending on the project) of 32-bit integer at 48kHz, and that's it. Not even any controls, as the DSP code (with its custom HID connection and corresponding app) covers that and more.

author: moffy
author_rank: Super Contributor
timestamp: September 22, 2021, 01:08:20 am
content: The NXP iMXRT1010, iMXRT1020 ... all the way up to the iMXRT1060(there are last digit variations within each group) have a 500/600MHz ARM M7 core with lots of peripherals. The 1010 and 1020 are LQFP the rest are BGA. Value for the range is exceptional and the 1020 EVK board comes with an on board audio CODEC with audio software examples via USB. The grandest chip is the iMXRT1172 which is dual core, one at 1000MHz and the other at 400Mhz and heaps of audio interfaces. You could use the 400MHz core to control the peripherals and the 1000MHz core as your DSP. Hope that helps.

author: ataradov
author_rank: Super Contributor
timestamp: September 22, 2021, 04:23:50 am
content: If you a just starting and planning on using vendor libraries, I would not use SAM4S. It is an older device, which is not supported by modern libraries. You would have to deal with old libraries that are mostly out of support now. They are still functional, of course, but may not be the best starting point.I would also try to avoid using SPI for I2S emulation. It would be hard to get the timings consistent.Start with more recent devices.

author: Marco
author_rank: Super Contributor
timestamp: September 22, 2021, 04:23:36 pm
content: Does it have to work with windows <10?

author: tszaboo
author_rank: Super Contributor
timestamp: September 22, 2021, 04:50:53 pm
content: SPI as I2s: Nope, you have more signals, so DMA wouldn't work, and your micro would spend most of it's time in interrupt for that many channels. And I2S carries playback speed, so you need precise control over it's speed, so extra hardware in the micro.USB: Just note that most microcontroller have USB 1.1 speed, 12MBPS. If you playback 16x channel at 16 bits @ 48 KHz, you are above that, AFAIK you can fit maybe 6 channel on a Audio class 1 device. Going to Audio class 2 limits the available micros a lot, simply because they just don't have example firmware.

author: AaronD
author_rank: Frequent Contributor
timestamp: September 22, 2021, 08:38:23 pm
content: Quote from: moffy on September 22, 2021, 01:08:20 amI'll have to poke around that family for a while and see what's there. Thanks! That last one looks especially interesting, but probably overkill too. Not necessarily bad though.What does that ecosystem look like? Supported IDE's, compilers, debuggers, etc. Preferably all free except for the hardware, and even that's fairly cheap to buy or easy to make (or universally compatible with everyone to justify a higher cost), and take advantage of any "special instructions" that a chip may have, instead of taking 10 or so to beat around the bush "the old fashioned way" that the special instructions are specifically meant to avoid.I'm not so sure about BGA packages, but that could just be a residual phobia from when I only did through-hole because that's all I could solder by hand with one iron. Now that I've made a toaster reflow oven (and made a custom surface-mount control board for it by hand with hot air) and done a few boards in there, and hot-air-reflowed a QFN, maybe BGA isn't off-limits anymore either. Just never tried it yet.Quote from: ataradov on September 22, 2021, 04:23:50 amAh! Okay, that makes sense. That was just what came to the top in a preliminary search for what I thought might be a decent guess at specs, then sorted by price. Like I said, I'm not married to it at all.Quote from: ataradov on September 22, 2021, 04:23:50 amI'm not entirely sure what you mean there. The only thing I can think of might be the left/right clock for I2S or the sample clock for TDM, which would probably have to be bit-banged or generated by a different peripheral if I used SPI to emulate the data. I can see that being tricky, but the bit-clock should be fine as that's still part of SPI. It's also a slight difference that SPI is byte-oriented and I2S/TDM are word-oriented, but a DMA to my understanding (and maybe a union in C, between the two data structures) should sort that out automatically, right?Quote from: Marco on September 22, 2021, 04:23:36 pmFully-updated Raspbian / Raspberry Pi OS for both. (based on Debian, like some other popular Linux distros) But if it's actually USB-standards-compliant, that shouldn't matter at all, I would think.Quote from: tszaboo on September 22, 2021, 04:50:53 pmYes, there's also a level-triggered left/right clock for I2S, or an edge-triggered frame/sample clock for TDM. (and there are far more bits per frame in TDM, all concatenated together so that a receiver has to count bits to separate the channels) That higher-level, 48kHz clock would have to be created separately if I used SPI as I2S or TDM. But a DMA can still reduce the number of interrupts by transferring an entire 32-bit channel or even a 256-bit TDM frame without CPU intervention, right?Quote from: tszaboo on September 22, 2021, 04:50:53 pmI2S carries playback speed? So extra hardware? How? I thought it was just a synchronous serial line with two clocks: a bit clock to run the shift registers, and a frame clock to latch the SR's into their respective sample buffers. Likewise for TDM, except that a frame is now many samples concatenated together instead of just two, and the frame clock only cares about the leading edge of the entire frame instead of denoting immediately what channel is being transferred at the moment. So the "playback speed" is simply the sample rate, right?Given that, the precise speed can simply be bit-banged using a top-priority interrupt if necessary. Yes, it takes some CPU time to do that, counting interrupt latency and a handful of instructions to actually do it, but surely not excessive.Or perhaps use a different hardware peripheral to generate a free-running 48kHz output, and then slave the DMA to that to feed the SPI peripheral? Then the whole thing just runs on its own with no CPU intervention at all except to run the DSP code between the single-sample-for-all-channels endpoint buffers. (in addition to triggering the DMA to transfer one entire frame and then wait for the next trigger, that free-running output also triggers a high-priority interrupt that has all of the DSP code in it)Of course, if there IS a complete TDM peripheral, or multiple I2S's, THEN USE THAT!! I just haven't seen one yet, except in a purpose-designed DSP chip that is effectively locked to its vendor IDE* and doesn't have USB**.(*Analog Devices has a good one, if it weren't for the block-size that equates to that many samples of latency. Most applications won't notice that, but I think that my second one will, as phasing issues.)(**That's almost a deal-breaker because of the ridiculous overkill that it seems to take just to convert a bidirectional TDM-8 stream into a bidirectional 8-ch USB Audio stream. There seem to be no drop-in chip that "just does that", so it needs a microcontroller and firmware for just that one job...or as the entire DSP itself, which is what I'm looking at here.)Quote from: tszaboo on September 22, 2021, 04:50:53 pmHmm...okay. I hadn't run into that yet, but it makes sense. And I probably *will* run into it during the second, more ambitious part. Thanks for the warning!

author: ve7xen
author_rank: Super Contributor
timestamp: September 22, 2021, 09:03:02 pm
content: Quote from: AaronD on September 22, 2021, 08:38:23 pmI would also avoid this. Generating the LRCLK with one channel of a multi-channel SPI peripheral (or divided down from your bit clock) is probably possible, and maybe you can use a PLL to generate the correct bit clock to avoid external clock generation. If using 32bit I2S it's probably notthatbad, but also sounds like there's a lot of potential for gotchas and it's just another thing to waste time on niggly details. Lots of micros have I2S peripherals, or an I2S mode in their SPI peripherals. It shouldn't be hard to find something suitable, so I wouldn't consider this.Quote from: AaronD on September 22, 2021, 08:38:23 pmYou'll definitely want Audio Class 2 to do multichannel. In theory you could implement this on any USB2.0 High Speed controller, but as pointed out, there's not great example firmware out there.You may consider XMOS XU series devices. They are basically designed for this purpose, there is existing AC2 demo code that works well (though I wouldn't really call it 'good code'), an App Note that covers how to add basic DSP to it, and they can drive at least 4in/4out I2S (or probably TDM) channels (ie. 8in/8out audio channels) quite easily. Unlike most micros they don't have fixed I/O peripherals. It's a bit weird to work with (it's an extended C with some additional meta files), but if your DSP processing needs aren't too heavy, probably quite suitable. It's used in a fair number of commercial products similar to what you're proposing.If anyone knows of any other USB AC2 demo code for other platforms I would be really excited to learn of it, as I haven't really found any, and the XMOS is really overkill for the basic applications I have been using it for.

author: tszaboo
author_rank: Super Contributor
timestamp: September 22, 2021, 09:42:14 pm
content: Quote from: AaronD on September 22, 2021, 08:38:23 pmYes, SCK is continuous, and DACs and other peripherals will use it to do the conversion. So it's not like SPI, that you can send a byte, wait a bit and send the next byte when ready. If your SCK has gaps or it is not exactly 32x 48 KHz, your playback is wrong.

author: moffy
author_rank: Super Contributor
timestamp: September 23, 2021, 12:17:21 am
content: The IDE's are free and good, MCUXpresso, and the development boards have SWD built in, but then the micro's all support JTAG and SWD. I've been recently using a SEEED ArchMix board (iMXRT1052) with RAM, IO pins, USB, Flash and an LCD connector and I had no problem getting the SWD working for programming etc. I used the LPC LinkII as the interface board between the IDE and the ArchMix. I too have a phobia about BGA hence the choice of an reasonably inexpensive module/board, the ArchMix.

author: SiliconWizard
author_rank: Super Contributor
timestamp: September 23, 2021, 02:22:46 am
content: Quote from: ataradov on September 22, 2021, 04:23:50 amNot just hard, but IMO impossible. I2S is a continuous serial stream. However you try to beat up a SPI peripheral, I doubt you'll be able to do the same. Maybe with some clever programming, and if you're lucky with a given SPI peripheral allowing continuous clocking without any interruption.

author: AaronD
author_rank: Frequent Contributor
timestamp: September 23, 2021, 02:42:02 am
content: Quote from: ve7xen on September 22, 2021, 09:03:02 pmGood to know that the peripherals are more common than I've seen so far. But of course I've been looking at the very low end of things too, so that's probably why I haven't seen them.(there'ssomepoint in a multibyte-oriented protocol that has to be kept running constantly, on an 8-bit slow micro, but not much)Quote from: ve7xen on September 22, 2021, 09:03:02 pmYeah, that's the overkill that I was referring to too. My introduction to their existence was simply to connect a bidirectional TDM stream from an Analog Devices DSP, to multichannel USB. That was it. No DSP in the XMOS, just a dumb conversion of protocols.Just 4 pins for TDM, assuming shared clocks for both directions, plus 2 pins for USB, plus power and a CPU clock, out of a 64-pin package! A dedicated all-slave hardware device could probably do that with 2 power pins and no additional clock for a total of 8! Maybe in a SOIC package?Quote from: tszaboo on September 22, 2021, 09:42:14 pmSo keep it running!Like I said before, set it up to just run constantly, regardless of any other activity. Free-running 48kHz clock, which repeatedly triggers both the start of an exchange and another trip through the processing code. Instead of waiting for the next data to be available, just blindly send whatever's in the buffer each time around. If it hasn't been updated by then, the same data gets sent again. No waiting, just run. (that puts an absolute hard maximum on the total processing time)Each cycle only triggers one sample's worth of activity. The next sample requires another trigger, so the bit-clock could be set either exactly or just slightly fast so that it's guaranteed to barely finish in time.Of course, that's all a moot point if there's a for-real I2S or TDM peripheral available.Quote from: moffy on September 23, 2021, 12:17:21 amI'll have a look at those too. Thanks!Quote from: SiliconWizard on September 23, 2021, 02:22:46 amThe SPI's that I've used (admittedly on 8-bit slow micros, so there might be a difference there) have a separate buffer from the shift register. As long as there's something new in that buffer, the SCK keeps running, so you can send a 16-bit word all at once just by writing twice and moving on. Or you can set up an interrupt (or poll the flag) so that you keep feeding it one byte at a time while the shift register is still clocking the previous one out, and thus send a continuous stream with no interruptions. (again, that's on an 8-bit micro with no DMA)Likewise for receiving, that same memory address can be read to get the last complete byte received, even if there's another one in the process of coming in. Thus, a continuous stream of input data as well, with no interruptions.To run a continuous stream in both directions as master, just do both operations in the same "buffer-not-full" interrupt. It's that easy.Of course, the additional frame clock could be tricky as described above, and it's all moot anyway if there's an exact-match peripheral to use instead.

author: Marco
author_rank: Super Contributor
timestamp: September 23, 2021, 03:06:40 pm
content: Quote from: AaronD on September 22, 2021, 08:38:23 pmAFAIK only from Windows 10 is there generic async USB audio support, before that you needed custom drivers. But that won't matter for Linux.

author: Bassman59
author_rank: Super Contributor
timestamp: September 23, 2021, 04:58:51 pm
content: A self-assigned background task for me is a High Speed USB Audio interface -- Audio Class 2 required -- in an ARM Cortex-M4. It's part of a larger design idea.I've sort of given up on trying to use I2S in the micro, and instead take advantage of a micro with a external memory bus, and connect that bus to a small FPGA to manage I2S or TDM to converters as well as build in S/PDIF and ADAT ports, and even MADI. This lets me do all of the audio stuff in its own little island, with the clocking, the analog/converters, all of it decoupled from the processor. The processor just sees a port as a destination for DMA transfers.The FPGA is the easy part. Really. A college student can do it as a senior design project. Hell, if you want, you can choose an FPGA with the resources to do DSP. (Or just buy a DSP chip with I2S, TDM and USB ...)Digging through every vendors' USB stack is painful. NXP offers a lot of USB Audio examples, but working through the code is like chewing glass. Part of the problem is that the vendors like to give canned solutions that you add to your project, so bang! Now you have an MSC device! Just add it from the list of our middleware! I asked on NXP's forum, "what's the best way to start implementing a standard class for which you guys don't provide code?" and the answer is "start with something existing, and modify it ... " Oh, and there's always the RTOS problem.I like the Silicon Labs USB stack. It's straightforward, and pared down: basically you get an endpoint interrupt callback and a packet of data for USB OUT endpoints or an endpoint interrupt callback saying "last packet handled!" for IN endpoints. If you read the class specs, you know what to expect and what messages you have to handle and all of it. So it's not all that hard to do an Audio Class device in their parts. I was able to get a USB MIDI device working in short order. EXCEPT they seem to have zero interest in adding a High Speed USB interface to their parts. I guess that makes sense, because their parts are all slow compared to the other vendors. (I guess the Internet Of Things That Go Bump In The Night don't need High Speed USB interfaces.)TI's Tiva and MSP432 devices don't directly support High Speed USB, but their stack and the chips allow you to connect an external ULPI PHY and it should work. Their USB stack is more complicated than SiLabs' but manageable.I keep thinking about looking at STM32 but haven't had the time.Of course there's always the option: write your own USB stack! That's what they're doing with the TinyUSB project.I suppose if you were clever and motivated and didn't otherwise have a life you could take the High-Speed USB PHY and glue it to an FPGA and do all of the stuff in the FPGA. Or maybe use a processor core in the FPGA. Or ...Anyway, good luck.

author: ve7xen
author_rank: Super Contributor
timestamp: September 23, 2021, 07:05:52 pm
content: Quote from: AaronD on September 23, 2021, 02:42:02 am'Overkill' is just going to be the reality of the situation though, I think. You're going to need USB HS, and that means a fairly beefy processor core to drive it, and those tend to come with a bunch of peripherals and whatnot. The XMOS isn't really that much more overkill than the alternatives, and it comes with ready-made AC2 code you can use to get a leg up on your end application. But it may be an overkill in 'new things to learn' as the architecture is quite different from most microcontrollers.Your post inspired me to take a poke around at this again. It looks like NXP may have some suitable parts, I was looking at LPC5514 as a contender. Cortex M-33 core, USB HS, and up to 6 half-duplex I2S ports and 2 more that have 4 I/O ports each, also supports TDM though I've never used this mode. It also looks like the Azure RTOS includes a USB AC2 implementation, and can be used license-free on these processors... I may try this on one of their dev boards. It's quite a bit cheaper and requires less support hardware (supply rails...) than the XMOS.Minimum viable for a dedicated IC would probably be something like a 'successor' to PCM2707, which is packaged in TQFP 32. The 'problem' is that such a trivial product isn't really viable in an end application. At a bare minimum in a typical product, you almost certainly need I2C to control your CODEC, and some code to decide what to send it and when. Such chips do exist (see CMedia CM6212 and a couple others, which is an 8051 core strapped to a USB-I2S converter), but you can't really buy them and there is no documentation or code available. If you have a volume product in mind though it might be an option.

author: Bassman59
author_rank: Super Contributor
timestamp: September 23, 2021, 08:20:20 pm
content: Quote from: ve7xen on September 23, 2021, 07:05:52 pmI'm looking at the LPC5528, same thing as the 5514 but without CAN. I like how the FLEXCOMMs are all identical and you just configure each for the mode you need. (Unlike TI TM4C1294 which for some reason has ten I2C ports!)re: Azure RTOS, looking at the Github I see for a demo they have descriptors which are just lists of hex bytes, as opposed to some of the crazy macros and defines you see everywhere else.QuoteTI has shown no interest in a High Speed follow-on to the PCM27xx devices.

author: ve7xen
author_rank: Super Contributor
timestamp: September 23, 2021, 11:20:25 pm
content: Quote from: Bassman59 on September 23, 2021, 08:20:20 pmI ordered the LCP55S28 devboard and will give this a shot. I also noticed that the NXP 'MCUXpresso' SDK includes USB Audio Class 2 support, so that might be a winner too, as it looks a little more fleshed out with examples and such than the Azure code. I'll probably start there, it's also all under a BSD-3 Clause license, including the examples, which is much more permissive than the Microsoft license.Wish I found this earlier.

author: moffy
author_rank: Super Contributor
timestamp: September 24, 2021, 01:13:01 am
content: Quote from: ve7xen on September 23, 2021, 11:20:25 pmLet us know how you get on, a USB Audio class 2.0 solution would be interesting. Pity about Windows 10 though not supporting class 2.0.

author: SiliconWizard
author_rank: Super Contributor
timestamp: September 24, 2021, 01:32:31 am
content: Quote from: moffy on September 24, 2021, 01:13:01 amIt does since Windows 10 1703, which is like 4 years old.https://docs.microsoft.com/en-us/windows-hardware/drivers/audio/usb-2-0-audio-drivers

author: moffy
author_rank: Super Contributor
timestamp: September 24, 2021, 01:59:14 am
content: Quote from: SiliconWizard on September 24, 2021, 01:32:31 amThanks for the info. Is the fact that it is WaveRT the reason XMOS require their own custom driver for Windows 10?

author: SiliconWizard
author_rank: Super Contributor
timestamp: September 24, 2021, 02:13:55 am
content: Quote from: moffy on September 24, 2021, 01:59:14 amI frankly don't know. It took so long for Windows to finally get USB audio 2.0 that vendors had their own drivers for their audio interfaces anyway. The XMOS solution for instance is much older than 4 years. And then they keep distributing their drivers even for Windows 10 just so people are able to use the device on *any* version of Windows 10.Additionally, they may have implemented some specific features in their own drivers compared to the generic one, but I do not know.

author: Bassman59
author_rank: Super Contributor
timestamp: September 24, 2021, 02:29:39 pm
content: Quote from: SiliconWizard on September 24, 2021, 02:13:55 amThe XMOS developer kit implementation of USB Audio Class 2.0 does nothing particularly special that requires a non-generic Class driver. It has been well-supported in macOS for, oh, FIFTEEN YEARS now. I did test that hardware on an up-to-date Windows 10 machine without the custom XMOS driver and it worked as expected.

author: Bassman59
author_rank: Super Contributor
timestamp: September 24, 2021, 02:31:44 pm
content: Quote from: ve7xen on September 23, 2021, 11:20:25 pmI have that same board, as well as the LPC54628 which adds 10/100 Ethernet.Good luck unraveling that spaghetti code.

author: SiliconWizard
author_rank: Super Contributor
timestamp: September 24, 2021, 04:52:23 pm
content: Quote from: Bassman59 on September 24, 2021, 02:29:39 pmOkay, so the reason is probably just my first point.And yes, USB Audio Class 2.0 has been supported both on MacOS and Linux for a MUCH longer time than on Windows. Maybe a "market" they didn't care about, or maybe they also just didn't dislike the fact all vendors had to pay for getting their drivers certified...

author: moffy
author_rank: Super Contributor
timestamp: September 25, 2021, 01:30:49 am
content: Thanks for the info, it's encouraging to know that you can use the XMOS audio without their proprietary driver. Interesting observation from Microsoft:"This USB Audio 2.0 class driver was developed by Thesycon and is supported by Microsoft." The same people that developed the XMOS driver.

author: SiliconWizard
author_rank: Super Contributor
timestamp: September 25, 2021, 01:47:26 am
content: Quote from: Bassman59 on September 23, 2021, 04:58:51 pmI had taken a brief look at it, but not done anything yet with it. Do you have any reason not to consider it?https://github.com/hathach/tinyusb

author: ataradov
author_rank: Super Contributor
timestamp: September 25, 2021, 03:56:34 am
content: Quote from: SiliconWizard on September 25, 2021, 01:47:26 amIt is just another stack you will have to learn. It may be less bloated than a typical manufacturer stack, but it is still more code than you would likely need.Also, I expect it to be solid for standard stuff, like virtual COM ports and HID, but I'm not sure how ready it is for real time audio and isochronous transfers. Such things are typically hard to abstract into a nice universal API, since they often involve MCU-specific buffer management and DMA access.Also, if you need to pass USB certification tests with your device, then smaller USB stacks may not be an easy option. There is a lot of stuff that works in a real world, but does not pass the official tests. And USB tests are such a PITA to run (pretty much require a dedicated PC not used for anything else) than I have not seen people do that a lot.

author: SiliconWizard
author_rank: Super Contributor
timestamp: September 25, 2021, 05:40:47 pm
content: Quote from: ataradov on September 25, 2021, 03:56:34 amYes, looking at it, there seems to be a lot of code in there. But vendor stacks are often just huge messes with a lot of intricate and unreadable code, and TinyUSB's code just looks a bit better in that regard...In particular, I considered using NXP's libraries for implementing an USB audio 2.0 + Midi device. The example projects are a huge mess with a crap ton of preprocessor directives, globals everywhere, uh. Crap.Quote from: ataradov on September 25, 2021, 03:56:34 amI don't know. Yeah. They certainly do support Audio 2.0 class, but from the little I've seen, TinyUSB is all based on some event loop and callbacks. I don't know how well it would play with DMAs and such.Another concern I would have is, I don't know how it would interact with hardware beyond the USB peripherals; so I don't know for sure how it could "clash" with how I set up things outside of it, like clocks, interrupts, etc. Actually, that's a concern I also have with many vendors-provided stacks. That's really a big mess!

author: moffy
author_rank: Super Contributor
timestamp: September 27, 2021, 12:25:36 am
content: I had a look at the code for USB Audio, for the RT1020EVK and noticed the Audio Class 2.0 directives. I had an old board which I then compiled the HID+Audio code then plugged it into my Windows PC. Sure enough it loaded the "usbaudio2.sys" driver, so it is recognised as USB Audio Class 2.0. I know it involves a lot of hacking of the code, but I have been doing that for a while with these chips, I might look into this further.

author: Bassman59
author_rank: Super Contributor
timestamp: September 27, 2021, 06:05:33 am
content: Quote from: SiliconWizard on September 25, 2021, 05:40:47 pmTotal fsckin' crap!It took me about a week to get USB MIDI working on the SiLabs EFM32GG with their library.It took a bit longer to get it to work on TI TM4C1294 (based on their "bulk" example).I gave up on the LPC55S28. Good luck unraveling the crap to see when you get the interrupt saying the packet has arrived.As for TinyUSB, ataradov's answer is the same as mine!QuoteThere's another NXP fail -- why can't their examples have the .MEX configuration file included? It seems to me that if you use that it wrecks the examples. Arrrgh.

author: moffy
author_rank: Super Contributor
timestamp: September 29, 2021, 01:08:03 am
content: I found a USB MIDI example for the NXP RT1020 from a developer called "IMXRT1020_USBMIDI" on github. When imported it at least compiled without errors. I have worked for a while with the NXP USB stack, and there is a lot of undocumented stuff going on in the background, but with perseverance and patience I have gotten it working reliably with grblHAL, but it took a while.

author: ve7xen
author_rank: Super Contributor
timestamp: September 29, 2021, 02:10:24 am
content: The basic USB 2.0 audio example seemed to work out of the box. I was able to switch it to use HS USB without much trouble, and after realizing that the descriptors for AC 2.0 are not defined based on the AUDIO_ constants like the AC 1.0 ones and updating those, also run the DAC at 32bit/96khz. However I think some under/overruns are occurring sometimes, occasionally it sounds 'digitally' for a few seconds. I didn't bother using it long enough at 16/48 to know if that's knew or existed in the demo.The code is definitely not 'clean', especially with respect to factoring out number of channels, bit width, etc., which are pretty fixed in the callbacks and such, and in that respect is worse than the XMOS code. The demo code also doesn't include any support for variable sample rate. However I'm having a somewhat easier time understanding what it is doing...

author: AaronD
author_rank: Frequent Contributor
timestamp: September 30, 2021, 08:23:16 pm
content: Well, I've looked at a bunch of NXP chips, and found another problem: I'm sure that they'll do the job, but the documentation that I'm used to for the small slow stuff, doesn't seem to exist at this level. Sure, I understand that there's so much more to document that it can't possibly fit in a single 1,000-page PDF like what I'm used to seeing. But still, where is it?!(and XMOS doesn't seem very accessible at all without an account)I found plenty of electrical specs and top-level-marketing type feature lists, but nothing about how to drive stuff in software. ("this" bit at "this" address controls or indicates "this", for every one of them that it has) That's the section of an MCU's datasheet that I spend the vast majority of my time in, before I even start on a PCB design, let alone write any code. Are you supposed to trust at this level that the manufacturer's libraries "just work" so that nobody else *needs* to know how to drive it?What I'm looking for when I do that, are the specific details of how this peripheral can or can't (sometimes must) connect to that one, and how to make that connection in hardware or software. It's kind of a LabVIEW-type approach, with "wires" between "nodes" that each carry a specific datatype. (8-bit PWM, 1-bit enable or periodic trigger, 10-bit ADC readings that are lowpassed in the ISR into 16-bit internal variables, etc.) Some of those logical "wires" are baked into the silicon, ready to connect by the peripherals' configuration registers; others I need to create in software. Once I have that, I can translate the peripheral configuration and the remaining software-connections to C.For the ones that I need to create, how much CPU time is it going to take, compared to what I have, including an estimate of interrupt (and/or main loop) frequency and latency? Do I have enough timers or other clock/time sources to drive everything that needs one? (some can be shared, others can't) Given a complete list of everything that needs to be done *somehow*, can this chip work, that I'm looking at right now, as shown by a complete logical connection between its peripherals and some conceptual software? Or do I have to use a different one that has a different pinout and maybe other requirements, and thus invalidate everything else like the PCB design and whatever software I might have tried to write prematurely?As an example of using software as a module, I have a digitally-controlled analog signal path to modify a VGA video signal. Most of the processing is time-agnostic, but one function - to invert the colors, or produce a "negative" image - is only allowed while the picture itself is actually being sent. The porches and blanking intervals must still be un-inverted or "positive".I'm using two of this chip's 16-bit timers for this (choice of 16-bit or 8-bit), and all 4 of its capture/compare modules. 2 of those modules are set up to put rising and falling "timestamps" on the horizontal sync signal, and another is constantly reconfigured in software to set or clear (one at a time) a "horizontal okay" signal and also trigger an interrupt so it can do that reconfiguration before it's needed. The last capture/compare module is connected to the other timer and reconfigured for all 4 of those tasks on the slower vertical sync signal. And there's some monitoring and correction code as well. (effectively a "software-locked loop" or SLL?) The two physical-bit outputs of all that - "H okay" and "V okay" - go to a hardware logic peripheral that also takes the top-level "invert" command and produces the actual control signal to the circuitry.That's set up as a combination of a single interrupt and polled flags, all non-blocking, so that the other functions can flow around this critical-priority driver. (that chip only has one interrupt vector, which then needs to poll all of the relevant flags and enables to see what it was; so I only have that one enabled to save some ISR time by not checking) Otherwise, it's just a glorified shift register by copying the SPI's input buffer directly to a GPIO port. (it *was* just a shift register, with no software at all, until I found out about the requirements to invert a VGA picture)Thus, software as a custom "pseudo-peripheral" of sorts, to augment what the existing peripherals couldn't be made to do (*) or to create a real-time connection or "wire" that the silicon itself doesn't have, that is only made possible by direct and detailed knowledge of how each peripheral actually works. And that's the part that I'm missing in the more capable documentation. I feel lost without it.(*) The so-called "hacker mentality": instead of "What does this do?", it's "What can I make this do?"I guess if there's an example project that "just works" already and includes all of the external connections that I need, then I could "add my code here" without having to know the details. But I'd really prefer to understand the details before I buy a lemon, and then use them to "know that I know that I know" that it's actually going to work like I think it does.

author: ve7xen
author_rank: Super Contributor
timestamp: September 30, 2021, 09:25:36 pm
content: For larger MCUs, this type of documentation is typically part of a User Guide or Programming Guide or something with a similar name that's separate from the Datasheet. Sometimes it applies to a larger family of chips than the Datasheet does.In the case of the LPC5528 for example, the document title is "LPC55S6x/LPC55S2x/LPC552x User manual". It can be found on the product page's Documentation tab under the User Guide category.

author: Bassman59
author_rank: Super Contributor
timestamp: October 01, 2021, 01:40:45 am
content: Quote from: moffy on September 29, 2021, 01:08:03 amOne problem I have with generic USB-MIDI examples is that they're not very useful. Here's what I mean.There's really no such thing as a "generic MIDI device" other than something like a USB to UART MIDI dongle like the M-Audio Uno. That thing is actually pretty specific, not "generic," because all it does is to take MIDI messages from the computer and spits them out on the 31.25 kbps UART. It does the converse, too, by taking messages coming in on the UART and packetizing them into the four-byte USB MIDI format and sending them off to the computer.Apart from that, you need to design the whole device. What good is accepting a message from the computer if you just drop it on the floor? Going the other way, what causes a message to be sent?So think about one of those neat Akai MIDI performance pad things. They're a collection of buttons and knobs and some LEDs. Tap a button or wiggle a knob and it sends the appropriate messages to the computer. The computer can send messages to the device to light the LEDs.So what's involved? You need something to scan the buttons and knobs and detect changes and build the packets from them. You also need to grab incoming messages from the computer and parse them and do something appropriate in response.That's why I think a decent demo project for USB-MIDI is this: design a device with two USB-MIDI "cables" in each direction -- roughly equivalent to the physical ports. (In fact, on macOS they show up in Audio MIDI Setup as two Ports.)Declare that Port 1 in both directions talks to a serial MIDI jack. Implement a UART at 31.25 kbps, wire up the optoisolator and two DIN jacks, and get it to work.Most eval board kits have some LEDs and buttons. So there's your "MIDI keyboard." Design it so that when you press a button, it sends a Note On message to the computer. When you release the button, it sends a Note Off message. A different button sends a different message. In the other direction, look for messages that you decide will turn on an LED, or set the LED blink rate, or set the LED intensity.Along the way, you'll learn about MIDI Jacks.What's great about this example is that it doesn't require host OS drivers. Software to send and receive messages is readily available, or you can use Python or whatever. Think of it as an easier-to-use HID class.Fun things you can do include setting up strings of RGB LEDs and using a MIDI sequencer program on your computer to "play" the lights.

author: AaronD
author_rank: Frequent Contributor
timestamp: October 07, 2021, 04:31:43 pm
content: Quote from: ve7xen on September 30, 2021, 09:25:36 pmAh! Okay. I found that one at least. Thanks!It was a little bit annoying to have to create an account to see it though. Like I said, I'm used to seeing all of everything in a single anonymously-accessible document. But at least the account is free, and the "required" fields like my phone number and mailing address will take anything non-blank.

author: AaronD
author_rank: Frequent Contributor
timestamp: October 07, 2021, 04:56:19 pm
content: Quote from: moffy on September 24, 2021, 01:13:01 amYes, please! I think I'll be okay with the other peripherals, but USB is still a bit much. I wrote my own HID-only stack from scratch for a PIC16F1454 and got it to pass the official USB tests on a dedicated Windows box, and that was enough for me! I'd rather drop in a working Audio2+HID stack if possible, and build my other stuff around it.Since you're using a dev board, it would also be good to know what off-chip resources it actually needs. I'm planning to use a custom PCB for both of my projects.

author: Zipdox
author_rank: Regular Contributor
timestamp: October 14, 2021, 04:24:49 pm
content: Open source hardware multichannel audio interface when?

author: BrianHG
author_rank: Super Contributor
timestamp: October 14, 2021, 04:43:45 pm
content: I've seen an 8 channel open source version for the Arrow DECA FPGA board. No MCU.https://github.com/hansfbaier/adat-usb2-audio-interfacehttps://github.com/hansfbaier/deca-usb2-audio-interfaceStill under development.

author: Zipdox
author_rank: Regular Contributor
timestamp: October 14, 2021, 05:50:26 pm
content: Quote from: BrianHG on October 14, 2021, 04:43:45 pmNot really complete audio interfaces. I'm sure it could be useful but you'd still need an ADAT interface.

author: ve7xen
author_rank: Super Contributor
timestamp: March 17, 2022, 09:00:40 pm
content: Quote from: AaronD on October 07, 2021, 04:56:19 pmI've been playing around with this dev board again in the past few days. Previously I had been playing with the NXP demo UAC2 code application, slightly modified to support 24/96 operation. I'd noticed very occasional audio glitches and low volume output, despite the output being hooked up to the onboard WM8609 DAC's in-built headphone amplifier. In the process of trying to debug this, thinking the code was mis-configuring the DAC, mis-justifying the I2S output, or hitting underruns or something of the sort, I found in the datasheet that the WM8609 can only operate at 96KHz if either the DAC or ADC is disabled. So it wasn't really suitable for my case, and this got me to bring out an off-board WM8740 DAC (with hardware jumpers for configuration). Over the past few hours of testing, I believe the problems were with the WM8609 implementation, not the USB-I2S code as I originally assumed.So at least as a 'simple' UAC2-I2S converter, the LPC5526/8 seems viable. It could also support up to 8 channels of I2S which is not bad, though the demo code is not set up for this and it would require some fiddling with the DMA. Not sure how it would do as a DSP, but it's a pretty fast micro with single-precision FPU so I would think it'd be fairly capable.Not quite ready to start my own hardware design yet (but probably soon, and the S variants seem to be available in low quantity right now, though a bit pricey), but the off-board requirements appear to be pretty light for this application. A 3.3V power supply, crystal, standard USB protection stuff should be all that's really required. Looks considerably less complicated than XMOS. Unlike many alternatives, this chip has the USB HS PHY on-chip which is quite convenient.I'm also toying with the idea of just implementing UAC2 myself. The more I look into this, the less scary it gets. Then something like a SAM3U starts looking appealing as quite a cheap option, it's the slowest / simplest / cheapest chip I found with a USB HS PHY onboard, though it's kind of hard to price things right now and LPC5526 seems to also be a similar price.

author: cncjerry
author_rank: Unknown
timestamp: March 19, 2022, 05:54:39 pm
content: Check out the dsp devices made by wondom or sure electronics. They support AD visualdsp and run stand alone or connected to a pc. I've used both their DSP boards, adau1701 is dirt cheap and very functional. I had an SDR SSB modulator running in it as well as a simple ADSR synth. The other with the 1400 (?) series chip needs an external DAC but I dont see that listed anymore. That board was much more capable. It needed an external DAC as mentioned. They sent it to me to test. I used a crystal semi DAC with that specialized interface, can't remember what it was called? Maybe just i2c? Can't remember, but I was surprised that the crystal board I had supported it on a connector. Wondom have amps with the 1701, blutooth, tons of simple examples, great support, etc. I've made custom variable crossovers with compression, expansion, filters etc in 30 minutes. The only challenge is getting their programmer to work the first time. If you test the board, you have to flip the little switch on it one way or the other...The other DSP I did was with AD SHARC and Blacfin boards like the 7603 based. The 7603 board supports VDSP too, might use the 1701 chip for audio. But if you are doing simple 2.1 stuff, these are all you need. Granted those general purpose chips might have higher latency but bad code could cause that on anything.As far as midi, it doesn't matter whether the device is coming over straight USB or thru a midi/USB converter like the Maudio devices as the PC libraries make that transparent. I've coded that up in an hour. there are trace modules, etc built into windows and mac's, don't know about linux but most likely there as well. I trace my Fantom and other synths that way using the mac midi trace and AU host that come with the mac development system and another program that makes internal midi busses for routing.Jerry

author: Marco
author_rank: Super Contributor
timestamp: March 19, 2022, 07:14:41 pm
content: Might be getting on the edge of just being too late, with Mini-DSP having a relatively affordable Sharc box now.Of course they haven't discovered that Lake Audio invented zero latency FFT based convolution decades ago, but baby steps.

author: AaronD
author_rank: Frequent Contributor
timestamp: March 20, 2022, 03:25:10 am
content: Quote from: cncjerry on March 19, 2022, 05:54:39 pmIt would make things a lot simpler if I could afford a 64-sample buffer in my end-goal project, that a lot of pre-made libraries seem to be built around. The stereo USB to 2.1 one can, and I think the USB Audio class has a bigger buffer than that anyway, but the problem then is how to get from USB into a pre-made library. The easy-to-use ADAU chips that I looked at didn't have USB, nor did I find any USB soundcard chips with raw I2S I/O.---My end-goal project is mic-to-speaker through some structural stuff that I can't get rid of. Essentially, that one needs to compliment the acoustic lowpass, as yet to be measured, and time-align to the speed of sound through that material, whatever it ends up being. Absolute distance is just a few inches, so I'd rough-guess around 10 samples total at 48kHz, including the converters.For that, I'm thinking to process 1 sample at a time, all the way through, then the next, etc., and then put an explicit delay in the middle that can be adjusted at runtime between 0 and 32 samples or something like that, along with all of the other coefficients.This one also needs USB Audio, not for normal use, but for a PC to record/monitor what the DSP is doing at each stage and to play back test signals. 8 channels of USB audio each direction for that. (it's a little bit more involved than what I said here, but you get the basic idea)And USB HID to read/write the coefficients through a PC app.

author: cncjerry
author_rank: Unknown
timestamp: March 23, 2022, 03:45:27 pm
content: In my notes above, replace VisualDSP with Sigma Studio. Sigma studio is the graphical block programming IDE by Analog Devices. Also, the 14xx series board was a 1452 and it supports TDM and I2S. Very capable product and easy to control.

author: AaronD
author_rank: Frequent Contributor
timestamp: March 23, 2022, 04:20:21 pm
content: Quote from: cncjerry on March 23, 2022, 03:45:27 pmYes, I've played with the SigmaSudio IDE and really like what I see there, and all that you said is true...with a block size of 64 samples. Thus, a minimum of that much latency. Not a problem for the USB sound card project, but an absolute deal-killer for the other one.I'd like to use the same system for both if at all possible, using the USB sound card as a primer of sorts for the other one, and not have to learn a new ecosystem again. If it ends up being a custom thing for both, as I'm pretty sure it will anyway for the other one, then that's fine.Also thinking about re-purposing the DMX512 lighting-control standard for real-time system control, 2 channels of which are DSP audio gains for some auxiliary functions. If my DSP chip is really a general-purpose MCU with a spare UART, then that's trivial. If it's a purpose-designed DSP chip, then the options become much more limited. Yes, I'd probably have to write my own DSP code for a general-purpose thing, instead of using a pre-debugged library, but I can do that.The math is publicly documented; I just have to implement it and double-check my implementation, and the sanity of my desired signal flow and the behavior of some known-oddball components that I'd have to fabricate/hack anyway if I were using the DSP IDE. (the nice thing about custom is that you can do exactly what you want; the bad thing about custom is that you have to do *everything*)And as I've said before, I have yet to see a purpose-designed DSP chip that includes a USB audio path. That part has always been external through a converter, which itself is almost if not more of an engineering challenge than the DSP is! So why not combine the USB and DSP functions into one chip? That pretty much requires a general-purpose MCU with USB, and then put the DSP code in that.

author: cncjerry
author_rank: Unknown
timestamp: March 23, 2022, 05:56:27 pm
content: you have me interested in reviving an old project. Check this board out. It has all my favorite components as I've coded STM32 boards, etc. I was working on a Eurorack modular synth general purpose module that depending on the loaded code, acted as oscillators, filters, ADSR, etc, etc.There is a learning curve with this and other boards like it as you have to access the 1452, program the STM32, etc.Search for ADAU1452 in eBay. Item number but many sellers: 383654346294ADAU1452 DSP Board Learning Board + AD1938 Audio CODEC Board 192KHz 24BitDescriptionADAU1452 DSP Board Learning Board + AD1938 Audio CODEC Board 192KHz 24Bit with Schematic DiagramsFeatures:- The new version of DSP board is equipped with MCU_STM32F103C8T6.- AD1938_CODEC decoder board added with a MCU for soft control, which can set the sampling rate and audio interface type.- On-board routines provide input or output IIS/SPDIF (optical fiber/coaxial selection) interface.- Factory setting is 192K_TDM format, OUT3 / OUT4 are pass-through interfaces, and OUT1/OUT2 is the output interface in stereo divide-by-2 mode.- S1 Button: Input selection. Blue light flashes for SPDIF input, red light for IN2 analog input, and green for IN1 analog input.- S2 Button: Preset EQ switch. A total of 10 EQ presets.- S3 Button: Super bass switch. Useful in electronic crossover subwoofer channel, it is turned on by default.- Three-color LEDs respectively display input channel and working status by flashing.- One STM32 MCU is reserved on the board, and the MCU can be selected to control DSP, and USB/UART/SPI port can be reserved. It can also be used as a MCU development board.- Potentiometer ADC collection for volume control, LED drive example.

author: SiliconWizard
author_rank: Super Contributor
timestamp: March 23, 2022, 06:20:48 pm
content: Just a quick follow-up about TinyUSB. I have now used it on both the RP2040 and iMXRT1062, in the first to implement a MIDI device, in the second to implement a MIDI device and a CDC interface.So far, I'm pleased with the experience. It's rather straigthforward to use and makes porting to a different MCU (as long as it's supported) pretty painless.Haven't tried implementing USB audio with it yet.

author: AaronD
author_rank: Frequent Contributor
timestamp: March 24, 2022, 02:20:39 am
content: Quote from: SiliconWizard on March 23, 2022, 06:20:48 pmGreat! That's encouraging. I picked up two RP2040's (RasPi Pico) about 2 months ago with the idea of using one as a programmer and the other to play with USB audio, probably becoming the USB stereo to 2.1 project. Then almost immediately after I got them, I fell into a different major project that took me halfway across the U.S. So I haven't touched them yet, and likely won't for another month or so.In the meantime, I'm still watching the forum in the evenings, and refining what I want these projects to do.Let us know how audio works. Especially if you get more than 2 channels each direction and more than 16 bits per sample (still standards-compliant/driverless on the PC side), and/or composite Audio+HID.(I'll use the HID part as a generic device to read/write the DSP coefficients with a custom HID-only app, but I doubt the USB stack would know that that's what it's doing. As far as the stack is concerned, it's just two unrelated functions mashed into a single device.)

author: AaronD
author_rank: Frequent Contributor
timestamp: April 22, 2022, 07:19:53 pm
content: Okay, I'm home now, so we'll see what I can come up with too.A quick and dirty calculation though, says that I probably can't get 8 channels each direction at 32-bit 48kHz on a Pi Pico (RP2040). It'll do Low Speed (1.5Mbps) and Full Speed (12Mbps), but not High Speed (480Mbps). 2 channels (stereo, one direction) of 16-bit 48kHz is 1.536Mbps, so stereo each direction in 32 bits is just slightly over half of the absolute maximum for that chip.A bit of googling suggests that that's common for a lot of MCU's with built-in USB: Low-Speed (1.5Mbps) and Full-Speed (12Mbps), but not High-Speed (480Mbps). There are a few MCU's that can drive a USB connector directly at High-Speed, but many. The options there seem to be an M3 core that works like you'd expect; or an 8-bit 8051 that only does the setup and some application-specific glue-logic, while the High-Speed USB data goes to an external chip (likely an FPGA, if you've chosen this over the M3 version).I guess the USB-sound-card-with-DSP project can still use the RP2040 that I have - I can still have 4 input channels for DSP-validation alongside the 2 output channels for playback, all at 32-bit 48kHz, plus HID to read/write coefficients - but the later one with 8 channels each direction will probably end up on an M3. So much for keeping the same platform throughout, but at least my DSP routines should still copy/paste at the source level. (it's just everything else that needs to rework: USB config, audio I/O, etc.)

author: AaronD
author_rank: Frequent Contributor
timestamp: April 22, 2022, 07:30:43 pm
content: Quote from: ve7xen on March 17, 2022, 09:00:40 pmHow's it coming? Did you decide to do it? Or do I get to?......when I get there.

author: ve7xen
author_rank: Super Contributor
timestamp: April 22, 2022, 09:32:06 pm
content: Quote from: AaronD on April 22, 2022, 07:30:43 pmIt ended up on the backburner for now, and since my dream was to make it platform agnostic, is probably a bit ambitious anyway. I'd be happy if I could get something usable on LPC5526 or SAM3U, my actual 'need' is really just a USB-I2S bridge. I've taken on way too many projects lately and the sun is starting to arrive, so it might be some time yet... I also noticed TinyUSB has an implementation of UAC2, but it seems kind of partially complete and TinyUSB has limited support for the necessary isochronous mode in its peripheral drivers, especially ones with HS (and lacks support in all the platforms I have to hand...). Might be worth a go if you have an STM32F4 to hand.I did find some time to experiment with the LPC55S28 board and NXP demo code some more, and while fiddling with the buffer and feedback parameters helps, the occasional glitches are still present; this was not solely due to the DAC implementation. I think that something is wrong with that code's timing feedback loop, but the code doesn't seem to align with what the USB spec says on the matter, and I don't quite grok how it's working in the time that I've had to look into it, the code is a bit of spaghetti.

author: AaronD
author_rank: Frequent Contributor
timestamp: April 22, 2022, 11:30:49 pm
content: Quote from: ve7xen on April 22, 2022, 09:32:06 pmI don't....yet? I'll focus on the Pico for now, since I have that already and it looks like I can get at least one project out of it. Then port the important stuff to something else later.

author: AaronD
author_rank: Frequent Contributor
timestamp: November 08, 2022, 07:51:55 pm
content: Quote from: ve7xen on April 22, 2022, 09:32:06 pmI've been playing with it for a while......and found pretty much the same thing.https://forums.raspberrypi.com/viewtopic.php?t=341735TinyUSB's maintainer said several months ago, that UAC2 was tricky and a work in progress.https://github.com/hathach/tinyusb/issues/1186#issuecomment-975449748https://github.com/hathach/tinyusb/discussions/796#discussioncomment-632928It still doesn't seem to have been fixed yet. Makes me wonder how in the world the example projects work, as posted on Instructables, etc., that use the TinyUSB audio driver. (Maybe they don't? And the example is simply how it's *supposed* to work in theory?) I've tried the examples that come with the TinyUSB library. They don't work either. Same problem.Quote from: ve7xen on April 22, 2022, 09:32:06 pmThat'd be just fine for my current project. Stereo USB to 2.1 DAC's, with DSP in between, trying (so far) to run on a RasPi Pico (RP2040). Get that working, then put my DSP code somewhere between the USB stack and the DAC(s).I seem to have everything working so far, except UAC2.

author: bob808
author_rank: Frequent Contributor
timestamp: November 09, 2022, 04:42:09 pm
content: I got an ADAU1452 and I was considering adding a USB interface to my project but dropped the idea. Once I set my filters I won't need to fiddle with it.For having a USB connection to the DSP chip you can add a CY7C68013A which could be used to emulate a USBi programmer. That would give you control via SigmaDSP but no audio data. You could add a separate USB-I2S converter for audio.In my case I designed LVDS drivers on the DSP PCB, and they move the data through a HDMI cable to a raspberry pi 4 shield that I designed. So I2S DIN/DOUT is managed by linux with a rt kernel. This way I have both sink/source available on the network with pulseaudio and JACK, so I can access them from both Windows and Linux machines.

author: AaronD
author_rank: Frequent Contributor
timestamp: November 09, 2022, 06:39:17 pm
content: Quote from: bob808 on November 09, 2022, 04:42:09 pmI was about to dismiss that as "not what I'm trying to do," but then remembered that my actual use for this particular project (not the later 8x8 one) is on a Raspberry Pi. So I do have I2S available directly from the Pi, which allows me to bypass UAC altogether! I even used it to test the temporary DAC board and snoop on the I2C config, that I then used to write my Pico I2S driver!So I guess now I need to have the Pico emulate that DAC over I2C, like the Pi driver expects, absorb the I2S stream from there, and feed the modified stream to the real DAC. That's certainly doable...if the hardware I2C didn't die when I start the PIO! (I'm using one of the PIO's as I2C to get around that) I'll still need USB HID to manage the DSP coefficients, but maybe that's more reliable?Only other problem is the mismatched channel count between my input and output. Stereo 2.0 in, 2.1 out. I could tie up both PIO's, one for stereo I2S with the Pi, and one for TDM with the DAC(s), but if the PIO had 2 more scratch registers, I would only need 1 of them and two of its state machines. As it is, I have to "pseudo-hard-code" the channel count and resolution into the PIO program itself. (load a constant, instead of copying a register that never changes) Pseudo- because the flash image has placeholders that the init code replaces with a calculated instruction, so that all the variations can be specified with #defines in a header file.Brainstorming:Maybe an I2S slave can use fewer PIO instructions? And piggy-backing on some existing signals can allow a TDM master to also use fewer instructions? Perhaps enough to fit both into one PIO? I thought briefly about combining everything into one big (32-instruction max) PIO program that does everything, but then the different bitrates made me think that it might be better to keep them separate and use a sort of Digital PLL to keep their sample / frame / L_R clocks in sync. (L_R has less of a meaning for multichannel TDM, but it still signals a new set of channel values)For why I'm not using an existing DSP, like the ADAU1452, I also want to learn how to write my own DSP code directly. Pretty much every off-the-shelf DSP that I've seen does things in buffers of 64 samples or so, which is not noticeable at all when that amount of latency is added to our perception (A/V sync, for example, when watching a video), but it *can* become a problem for the later 8x8 project, which is really a stereo mic feeding stereo earbuds that are close to each other through some structural material. The purpose of that is to undo the acoustic lowpass ("unmuffle", if you will), and the physical distance is small enough that individual samples can cause or fix a comb filter through the (rather wide) crossover region between acoustic and processed sound. So I think I need to write my own code to process the one absolute newest sample all the way through and shove it out NOW, while running at an otherwise unreasonably high sample rate, both to reduce the time between samples, and allow the converters to use less aggressive filters, which reduces the latency even more.I'm a live audio engineer, so I already know what processing I want in what order and why, so the educational aspect of playing with the blocks is already past. I need it to match the speed of sound over an unusually short distance, which according to some datasheets and some math, is pretty close to as instant as I can get it from one TDM stream to another. (essentially a 3-sample pipeline, if you're familiar with optimizing a CPU: receive from ADC, process, send to DAC) Hopefully, that turns out to be ever so slightly too fast, so I've designated a place in the DSP chain to put just a handful of samples' worth of delay.The high channel count, for a fundamentally stereo application, is because of a few auxiliary stereo sources and sinks (other hardware devices), and for testing / debugging / recording on a host PC. (tap off of several points along the chain, to see what all of them are doing at the same time) I'd be okay with something other than USB for that too, but I don't know of any other accessible standard that can serve 8 or more channels each direction with high resolution and high sample rate. (currently looking at 32-bit, 96kHz) And I still need to manage the DSP coefficients from that PC as well.But that's a future project that I really hoped to use the current stereo-to-2.1 one as a springboard for. It's looking increasingly like no. The data-transport tools just aren't quite there yet.

author: bob808
author_rank: Frequent Contributor
timestamp: November 09, 2022, 08:02:58 pm
content: The DSP project is in the works at the moment. I did use the RPi setup for a while now, but directly with a DAC. I2S still over HDMI, master clock on DAC board. I set everything 24/96 in my case, simpler and fits my needs, RPi resamples everything. So for me the RPi and DAC are slaves atm without DSP. I still haven't decided on the clocking setup for the DSP but most likely RPi will be slave.I used this driver:https://github.com/AkiyukiOkayasu/RaspberryPi_I2S_SlaveSeems to work fine for both DIN/DOUT. There's a link for the master version.I have no clue on the delay that the DSP will add. I'll see when I get to test it. For my setup I have two channels in and six channels out.

author: AaronD
author_rank: Frequent Contributor
timestamp: November 10, 2022, 12:08:52 am
content: Quote from: bob808 on November 09, 2022, 08:02:58 pmI'm pretty sure thatthat's"not what I'm trying to do!"Does look like an interesting project though. Using a Pi 3 (as tested by the maintainer) in place of where I have the Pico, to do pretty much the same job. But there's enough processing power in the Pi 3 that you might run a live FoH mixer for a small band! (even more in the 4)You're already using Jack, so you might also have a look atCarlato host an arbitrary set of processing plugins and nothing more,NonandArdourfor a more "soundboardy" look and feel, andLSPfor the plugins themselves. Can't comment on the measured latency of those either, but they're more than fast enough for the live work that I've done so far, on an x64 laptop running Ubuntu with off-the-shelf USB I/O.Non's maintainer is one of those guys whose social skills all went to writing software instead, so he's good at that, but most of the community doesn't like him, and some have forked his project to maintain themselves so that they don't have to deal with him, hence the banner on his site about "Free Software under attack!"I was part of an e-mail exchange with him and the community, where all of that became more than clear. Seems like good software though.For what I'm doing, the Pi 4 is simply a desktop replacement, with some interesting I/O that most desktops don't have, on a 40-pin header. Mouse, keyboard, monitor, speakers, networking, and used as a desktop PC. I also have a 3B+ with nothing to do (been bumped out of two different rigs now, both by 4's), but that seems like overkill, and I might have another use for it later, so I want to keep it available for that. Plus, the Pi 4 rig has limited space in a rather non-standard arrangement, so the questions of size, location, and interfacing are kind of important.I'm pretty sure I'll end up with a custom PCB at some point for the Pico; I just need to determine what all goes on it. So far, just the RP2040 chip and its supporting hardware, and maybe the DAC's. Or maybe the DAC's go on a different board so that the wiring in between can separate the digital and analog worlds? I have some other functions in mind too, to use the 2040's spare cycles after the audio part is done and working...

author: SiliconWizard
author_rank: Super Contributor
timestamp: December 25, 2022, 09:51:57 pm
content: Quote from: Marco on December 25, 2022, 09:42:30 pmI've used a Linux RT kernel on x86-based machines with success for that kind of stuff. (USB audio / ALSA access / very short sample buffers.)Problem is, there is no Linux RT available for the Pi that I know of. You can always try and build it yourself, but I don't know what kind of luck you'll have with that. Don't know if there's a full set of patches that'll make it run successfully on the Pi. And people not familiar with building the Linux kernel, forget about it.But if you actually can point us to an official Linux RT kernel for the Pi, that'd be cool.

author: Marco
author_rank: Super Contributor
timestamp: December 26, 2022, 06:50:20 pm
content: Yah, I saw the main Raspberry Pi kernel is a fork and preempt-rt is still a patch set too. So combining them is a bit of a headache.I thought preempt-rt was mainline by now, but I thought wrong.

author: SiliconWizard
author_rank: Super Contributor
timestamp: December 26, 2022, 06:58:20 pm
content: That is an "interesting" fact that practically no SBC (at least that I know of) has a supported preempt-rt Linux kernel. I don't know how hard the task is. Actually apart from being a user, I've never built a kernel for an ARM target myself, so I don't know the subtleties if any. For the x86(_64) platforms, preempt-rt is not too much of a problem.You'd expect that this would have triggered enough interest in combination with an SBC, but apparently not. Unless there is a real technical hurdle here. Weird.