Trying to find someone that can program audio effects (DSP? Impulse Response?) - Page 1
Forum section: electronics / general

author: K3mHtH
author_rank: Regular Contributor
timestamp: December 27, 2017, 12:24:27 am
content: Hey everyone,This is totally out of my wheel-house so I'm looking to hire a contractor that is capable of programming high quality DSP audio effects to run on dedicated hardware (they can dictate the hardware). I'm trying to do some interesting reverb-type effects, but I'd also like to do more complicated things like emulate or model vibrating strings and vibrating metal panels. Where do programmers for that kind of stuff generally hang out?I'm also finding a lot of companies that make audio VST plug-ins that would fit the bill as well, but I don't know if those programs can easily be run on a chip instead - if so I'd love to just licence an existing program.Hardware wise, it seems like some of the companies doing this well (Strymon - makes DSP guitar pedals) use the Analog Devices SHARC DSP processor.

author: iampoor
author_rank: Frequent Contributor
timestamp: December 27, 2017, 03:38:49 am
content: I have been asking the same question, and have been asking around the effects industry for some time about this. I am not aware of any "audio only" effects freelancers, outside of a few that work on very specific platforms (Spin FV-1). I am personally studying this problem in my free time, but dont have much real expierence yet. Good luck on your search.

author: iampoor
author_rank: Frequent Contributor
timestamp: December 27, 2017, 04:04:50 am
content: Quote from: blueskull on December 27, 2017, 04:00:00 amWow. Last time I talked to the Audio Weaver people you had to buy the software, an expensive support contract (per year), and purchase a minimum of 1000 licenses. Lets just say it wasnt cheap......

author: rhb
author_rank: Super Contributor
timestamp: December 27, 2017, 04:52:30 am
content: I'm a 64 year old seismic research scientist and programmer. I'm moderately familiar with audio effects, but prefer to play pure acoustic on my archtop guitar. However, I spent 30 years doing DSP for the oil industry. So if you can give me an intelligible description of what you want I can certainly do it. The only reason I'm "retired" is there is no work for old guys in the oil patch. I'd love to have a job just for the sake of the technical interaction. I miss being asked to wave my magic wand.Send me a PM if you're interested.

author: K3mHtH
author_rank: Regular Contributor
timestamp: December 27, 2017, 07:47:08 pm
content: Quote from: rhb on December 27, 2017, 04:52:30 amWill PM you more details now. Wow AudioWeaver looks pretty nice, thanks for the link.

author: Yansi
author_rank: Super Contributor
timestamp: December 27, 2017, 07:50:16 pm
content: Quote from: blueskull on December 27, 2017, 04:00:00 amSorry, but you are not going to do any kind of reverb on STM32 using just impulse response convolution, as the OP wants. That requires way more grunty DSP to do so.

author: Marco
author_rank: Super Contributor
timestamp: December 27, 2017, 08:19:54 pm
content: Lets do some really ridiculously rough calculations of how fast a processor would have to be able to do a 1024 point real-FFT to be able to do a 10 second reverb. Ignoring overhead from zero-latency FFT implementation, imperfect scaling, memory etc etc.For a 10 second reverb you basically have to be able to do a 10(seconds)*2(overlap)*48000 point FFT in 10(seconds)/2(forward+inverse)/2(stereo) seconds. So lets say a million point FFT in 2 sec. Lets nlogn that back to 1024 points since no benchmarks for million point FFTs are available, a*log(1e6)*1e6=2, a = 2/6e6. So the processor has to be able to do a 1024 point FFT in about 2/6e6*1000log(1000)=1ms.The STM32F7 is actually able to do a 1024 point FFT in 334us. Now that's not enough room for all the overhead and the STM32F7 generally lacks sufficient memory, but it does show that the processing requirements for FFT based convolution really aren't that high. The patents for how to do zero latency filtering with FFT have expired a long time ago ...

author: Yansi
author_rank: Super Contributor
timestamp: December 27, 2017, 08:32:32 pm
content: I do not think that memory is really your enemy here. There is PLENTY of memory and even MORE can be added on the ext. bus. (Dunno what you would need all that memory for)1024 point FFT in 334us? Doesn't sound right, but haven't done much FFT on Cortex M7s yet. Where did you get that figure and what CPU speed was it?Also 48kHz sample rate does not very much qualify for high quality (respecting the OP's requirements), nor does 32 let alone 16bit fixed point arithmetic in the FFT. Almost sure 1024pts FFT at respectable audio quality is not possible on such device. Do you have any data to support this? I would be interested! Thanks.

author: Marco
author_rank: Super Contributor
timestamp: December 27, 2017, 09:16:57 pm
content: A 10 second impulse response in 16 bit 48KHz is 1 MB and the zero latency FFT convolution needs a bunch of buffers of similar size in floating point, so you need quite a bit of memory.The FFT numbers came fromhere(page13). They seem to be for complex FFT though, rather than real-FFT and I overestimated the necessary speed by 2x (took overlap for overlap add into account twice). So the STM32F7 might actually be able to pull it off, with sufficient memory (and sufficiently fast memory).PS. the F7 can dual issue a floating point MAC and a memory op, the only thing it's really missing to be a "real" DSP is zero overhead loops.

author: Yansi
author_rank: Super Contributor
timestamp: December 27, 2017, 09:33:32 pm
content: Interesting. Have not noticed this appnote.Not sure how reverbs are implemented in reality (not even heard that they use FFT), so can not give any further expertise (as my DSP expertise ends at designing a bunch of stupid biquad filters), so really do not know how it would end up on STM32. The computational power in a Cortex M7 is respectable, but I think certainly not for high quality audio application.By the way, do you have any recommended online literature about real implementations of reverbs? Have tried to look something up in the past, but rather unsuccessfully.Thx, Y.

author: Marco
author_rank: Super Contributor
timestamp: December 27, 2017, 10:16:21 pm
content: Quote from: Yansi on December 27, 2017, 09:33:32 pmFor a long time Lake DSP heldthe patentfor using partitioned impulse responses to implement long convolution with low latency. So everyone else was kinda screwed in implementing low latency convolutions for long impulse responses. I haven't paid any attention to audio DSP in ages, but I see impulse response based reverb has gotten a lot more popular lately so I assume everyone is using the FFT approach nowadays.Lake DSP has a couple other more recent patents they seem to try to pretend are relevant, but the first one is all you need ... best to not even look at the others.QuoteNot reverbs in general, but I have googled some links on how to implement low latency convolution with impulse responses for you based on my ancient/outdated knowledge.Gardner's paper "Efficient Convolution Without Latency"is the classic reference (together with the above linked patent).This bloghas a really short example of how to implement it with Audio Toolkit (it skims over the details of how to use FFT to do convolution, but details on overlap-add/overlap-save FFT based convolution can be found through google). I think the open source LV2 pluginKlangFalterimplements it too, but I have not used it or looked too deeply into the source code.

author: rhb
author_rank: Super Contributor
timestamp: December 28, 2017, 01:46:56 am
content: I clearly need to do some reading, as this seems a bit convoluted if you'll excuse the pun. But my initial reaction is, "Why would you use an FFT to implement a reverb?" I've done it a lot on recorded data, but it doesn't make sense in a real time situation. Set a timer and add the echo to the output appropriately scaled at the proper time. How many reflectors do you want? Oh, you want your reflectors to move? Not a problem!As for the Lake DSP patent, all I can say is , "Gad! Someone has patented the wheel!" An Infinite Impulse Response (IIR) filter is just a tapped delay line. And an IIR is as long a convolutional filter as is possible. It's infinitely long.I suspect that the "invention" is the result of people not being aware of how to efficiently resample data. Something implemented by Dave Hale in CWP/SU *many* years ago. This is a chronic problem in DSP.I shall read the patent more closely, but the claims are on the surface outlandish. I've been doing this stuff for 30+ years and endured 2 semesters of Integral Transforms and a course in Linear Systems at UT Austin.Convolution in time is a multiplication in frequency. The opposite is also true. There's a bunch of twiddle to deal with, the series needs to be padded with an equal number of zeros to prevent wrap around and in specific instances there are little nastinesses with respect to phase and end effects. But all of this is well established though not necessarily well known by practitioners. I had my head handed to me on platter by arbitrary sample rate resampling by FFT. I thought it would be a 2 day task . Beat the crap out of me for 2 weeks until I remembered that the FFT is defined over the semi-closed interval + or - Pi.

author: iampoor
author_rank: Frequent Contributor
timestamp: December 28, 2017, 03:31:07 am
content: Quote from: Yansi on December 27, 2017, 09:33:32 pmKeith Barr (Founder of Alesis) developed a very simple, yet powerful reverb algorithm back in the 80s. I think this is the simplest way, without using Impulse responses. Delay lines and allpass filters. I would be amazing if a Cortex A4 or A7 wasnt powerful enough for this.http://www.spinsemi.com/knowledge_base/effects.html#ReverberationOne thing missing from the audio weaver for STM documentation (or maybe I missed it) is the requirements for external memory for long delay lines. I havent used the STM32 before, is adding SDRAM easy, or is it a huge pain? Okay enough thread hijacking...

author: Marco
author_rank: Super Contributor
timestamp: December 28, 2017, 03:41:28 am
content: Quote from: rhb on December 28, 2017, 01:46:56 amMostly to reuse measured/"raytraced" responses.QuoteA thousand with multiple reflections, for instance.QuoteBut not arbitrary, can't replay the exact response from a room with IIR (or correct for room response).QuotePartitioning the impulse response and implementing it with FFTs of increasing size (and a FIR filter for the first part if zero latency was required) was novel at the time of the patent. Gardner invented it independently at almost at the same time, but that's a separate issue.

author: coppice
author_rank: Super Contributor
timestamp: December 28, 2017, 04:01:15 am
content: Quote from: rhb on December 28, 2017, 01:46:56 amI think what they patented was novel for 1993, probably because it was just becoming economic to do this kind of thing in real time. It has elements of much earlier solutions, but I don't think their exact solution was used. They break a huge convolution into chunks. They convolve each chunk by a relatively small FFT->multiply->FFT with OLA. They then sum the results of each of the smaller convolutions, and get an output with the latency of the small FFTs. This seems an efficient, low noise approach to achieving reasonable latency.

author: rhb
author_rank: Super Contributor
timestamp: December 28, 2017, 04:13:34 pm
content: When that patent was issued I was working at the Mobil R&D lab in Farmers Branch having been laid off from ARCO R&D in 1991. The patent is an example of ignorance recreating what was well known in another field. It is a huge problem, especially in DSP. I've read far too many papers only to realize at the end it was just another wheel that had been around for a long time with a different name. The "Modified Discrete Fourier Transform" of claim 33 in the Lake patent is just the standard real to complex FFT which had been around since the 1950's, though not well known as it was being done by hand with a desk calculator. I had the privilege of having a few drinks with someone whose name I forget who was doing FFTs in the mid 1950's. Mention of it was made in print, but it never attracted any attention until after the Cooley-Tukey paper.An echo is a multiply-add operation, scale and sum. If the echo arrives in between time samples then it must be interpolated using a sinc(t) operator. That requires 8 multiply-add operations to reduce the error to a few percent.I typically use FFTs to implement convolutions because it is both faster and more flexible. But I've only worked on recorded data.When I was at UT Austin, I wrote a paper for a class which was a complete analysis of the reflection and transmission response of a plane layered half space with a free surface boundary with the source and receiver embedded at arbitrary locations in the medium. That coda is infinitely long. In the Z transform notation I used it has the form of a fraction. The denominator produces terms which continue forever. The problem is the geophysical equivalent of including echos from both sides of the wall and the other rooms in the building. I should note that more than one dimension becomes very difficult and is not amenable to a closed form expression as is the 1D case.The classic example is reverberation in a water layer or between two walls. This was first implemented in analog form using a tape recorder with feedback from the output to the input. It is far simpler to do by feeding the output of a delay line back into the input suitably scaled. This is precisely what tape, spring and plate reverbs do.An IIR filter is one in which the Z transform has terms in the denominator. An FIR filter only has terms in the numerator. The classic IIR example is:1/(1 + R*Z**a) which evaluates by polynomial division to 1 +R* Z**a + R**2*Z**2*a + R**3*Z**3*a......That's the denominator for reflection between a surface with a reflection coefficient of 1 and another surface with reflection coefficient of "R" with a delay of "a" for the propagation. The proper expression is more complex and having done this 30 years ago I should have to look it up to get it right. In this context there is no need.The point is this. A data stream passing through a DSP system need only have terms added to the input at the appropriate delays. If the reflecting surface is moving, then the delay changes at each bounce.Designing a filter to reproduce the room ambiance from a recording is certainly easily done in the frequency domain via an FFT. But it does not need to be implemented that way. The design of filters to remove reverberation effects was introduced by Norbert Wiener around 1940 and is called a "prediction error filter". The first example was done by his student, Enders Robinson, by hand over the course of a summer at MIT in the early 50's. My PhD supervisor at Austin's claim to fame was implementing a dereverberation filter using a magnetic drum and movable recording and playback heads in the late 50's. Millions of hours of computer time have been spent doing "predictive deconvolution" of seismic data.The compute is trivial now, but when I started with Amoco in 1982 we started getting 120 trace data back from the boats. All of a sudden the decon step in he processing was taking several weeks to complete. All this was being done on a large IBM mainframe in Chicago with an attached vector processor. These were tape to tape jobs, so each step was a single compute job. The predictive decon step needed over a week to run at normal priority. (This was a multi processing batch system) As a result, it would be aborted when they took the machine down for the Sunday maintenance period. The job would then be restarted at a higher priority, but would again fail to complete. So it would restart again. Eventually it would run at very high priority which allowed it to complete, but forced other jobs to abort and go through the same cycle. Once we found the cause we just broke the data into several pieces, ran them and then merged them after decon.Yes, I know. tl;dr

author: Yansi
author_rank: Super Contributor
timestamp: December 28, 2017, 04:19:18 pm
content: I did read

author: coppice
author_rank: Super Contributor
timestamp: December 28, 2017, 05:56:43 pm
content: Quote from: rhb on December 28, 2017, 04:13:34 pmThat line shows a fundamental misunderstanding of what a patent is. You can't patent basic techniques. You can only patent the application of techniques to solve a real world problem. Similar maths gets included in patents over and over again, as it is used to solve different problems. The basic maths used in that patent dates back to at least the 60s (at least I think that's where the full expression of the OLA technique was formulated), and I was implementing similar things in real time hardware in the 70s. I wasn't using those techniques to solve a real time live performance latency problem, though, so they were fair game for a new patent in 1993.