Descrete Fourier Transforms - Resolution vs sampling duration - Page 1
Forum section: electronics / projects

author: max_torque
author_rank: Super Contributor
timestamp: October 23, 2015, 02:13:07 pm
content: Ok, appolgies for the Maths in this one!If i have understood it correctly, when transforming a descrete signal from the Time Domain into the Frequency Domain, the Frequency resolution available from a DFT type operation is principally defined by the total sampling time, rather than the sampling frequency (which just sets the Nyquist limit etc).So, what do i do if i want to sample a non repetitive signal that only lasts for a short period of time, and yet i want high resolution ?Lets say, for the sake of argument, i have a non repetitive signal that lasts just 5ms, and contains frequency components up to, say 20Khz. I'm going to have to sample it at over 40Khz to meet the Nyquist requirement, however 5ms sampling is a 200Hz separation.Unless i'm missing something, there is no way to improve on this without doing something clever??More experienced DSPers please chime in

author: dom0
author_rank: Super Contributor
timestamp: October 23, 2015, 02:15:20 pm
content: One technique is repeating the signal. The issue with that approach however is spectral leakage at the signals "seams", so to speak, which can be somewhat alleviated by windowing the signal before repeating it.One can also insert zeroes between samples, but that's effectively just interpolating the added bins in the output between the bins of the original output.

author: c4757p
author_rank: Super Contributor
timestamp: October 23, 2015, 02:18:01 pm
content: Quote from: max_torque on October 23, 2015, 02:13:07 pmThink about it - how would you know? With your sample rate, a 20kHz signal and a 20.01kHz signal will look identical for the first thousand or so repetitions - the difference in period hasn't even accumulated to be worth one sample!

author: T3sl4co1l
author_rank: Super Contributor
timestamp: October 23, 2015, 04:18:18 pm
content: Or perhaps more acutely, Fs gives the frequency scale (where the bins in the result are simply ratios 1/2^n to (2^n - 1) / 2^n of Fs, assuming a 2^n sized FFT is used).And yes, the frequency goes beyond Fs/2; for a real input signal, the upper half is always exactly symmetrical with the lower half (more precisely: Hermitian, I believe?). This is a consequence of conservation of information: the two halves together conserve phase and amplitude information. (It just so happens that real signals don't contain complex values, so half the input data is effectively zero.)As for resolving transients better, if you can trigger on them, you can do what a scope does and zoom in on it. If they're repeating, you can accumulate a very detailed picture over time (binning the samples into a statistical study over time, or averaging them out to a smoother figure). A DFT on such a sample will give relatively widely spaced bins, given Fs, because you're only resolving a small burst rather than the full time between transients. But that's precisely the computational/sampling savings you were asking for, so this should be understandable.Tim

author: Karel
author_rank: Super Contributor
timestamp: October 23, 2015, 04:28:27 pm
content: DFT resolution = Fs / DFTblocksize.For example, if the samplefrequency is 1000Hz, and the DFT uses a blocksize of 2000 samples, the resolution (frequency width of a bin) will be 0.5Hz.For a given samplerate and blocksize, repeating and averaging will lower the noise. It will not increase resolution.

author: max_torque
author_rank: Super Contributor
timestamp: October 23, 2015, 09:52:07 pm
content: Quote from: Karel on October 23, 2015, 04:28:27 pmIndeed, but to get that 0.5Hz resolution, you have had to sample for 2 sec! (0.5 = 1/2)If my signal "chirps" for just 4ms then it cannot be sampled for 2sec!One thing i thought about was to sample the same signal with a dual ADC at different frequencies, where the sampling frequencies are chosen to give interleaved frequency bins?

author: rfeecs
author_rank: Frequent Contributor
timestamp: October 23, 2015, 10:08:44 pm
content: Quote from: max_torque on October 23, 2015, 09:52:07 pmWhat happens if you just sample for 4mS and fill in the rest of the two seconds worth of samples with zeros?

author: max_torque
author_rank: Super Contributor
timestamp: October 23, 2015, 10:21:40 pm
content: Interesting question!As the Sine/Cosine coefficients are multiplied by the input sample value and then integrated, surely adding zeros, just pads out the bins? ie, the zero terms don't change the total integrated bin values (because Sine(x) x 0 = 0)?

author: T3sl4co1l
author_rank: Super Contributor
timestamp: October 23, 2015, 10:55:33 pm
content: Quote from: rfeecs on October 23, 2015, 10:08:44 pmIf you sample for 4ms and convert that (without padding), you get the same thing, zoomed in, in a sense.Padding with zeroes just dilutes the same information that's there (i.e., making all the values smaller, which isn't a problem for floating point but may be a problem for fixed point).Which leads to...Quote from: max_torque on October 23, 2015, 10:21:40 pmChanged versus what? It's certainly not the same as just the sample (unpadded), because much of the low frequency content over the (virtual or real) 2 s sample period cannot possibly be nonzero (aside from a DC offset, the most the sampled (nonzero) period can affect is frequencies of 1/2 wavelength). And it can't be the same as the fully sampled condition, which fills in those zeroes with, well, we haven't beentoldwhat the signalis, but presumably it's just dead line noise inbetween. I guess...In any case, the minimum sample rate for a single-shot event is Fs/2 = BW, plus practical margin. If there's a certain aspect about the transient (such as envelope shape rather than carrier frequency) you want to focus on, you could do some RF magic (downconversion), which is also equivalent to ET sampling.Tim

author: rfeecs
author_rank: Frequent Contributor
timestamp: October 23, 2015, 10:56:44 pm
content: Quote from: max_torque on October 23, 2015, 10:21:40 pmBut you have more samples, so each nonzero sample gets summed over more sinusoids at more frequencies that are closer spaced.

author: mark03
author_rank: Frequent Contributor
timestamp: October 23, 2015, 11:05:16 pm
content: @rfeecs has the correct answer.The best way to think of the DFT / FFT is as a periodic sampling of the DTFT (discrete-time fourier transform). The DTFT X(f) = sum_n(x_n * exp(-j*2*pi*f*n)) and is a continuous function of frequency. The DFT / FFT is a sampling of this continuous function at N points evenly spaced in frequency from 0 to 2*pi*(N-1)/N. If you would like those samples more closely spaced, you simply zero pad the input to the desired length. There is nothing to be gained by tiling the signal.Note that there are fundamental relations here which cannot be avoided. If your signal is truly time-bounded ("finite support" I think the theoreticians would say), then in the frequency domain it will have energy at all frequencies out to infinity. This implies that *any* discrete-time sampling will result in at least some aliasing. The closest you can get to finite support in both domains is a gaussian, whose transform you may recall is another gaussian... and of course both stretch out to infinite time and frequency, although in practice they decay so quickly that no one cares.

author: rfeecs
author_rank: Frequent Contributor
timestamp: October 23, 2015, 11:11:06 pm
content: I can think of it this way:Say you have a just a single 20 kHz tone that you pulse for 4mS.The spectrum of that looks like a sin(x)/x shape centered at 20kHz.The DFT actually gives the spectrum of a periodic waveform that repeats with a period that is equal to the length of the total sampling interval.The effect of this on the spectrum is to create a comb of frequencies that are spaced with frequency 1/(total sampling interval). Each frequency ends up as a point in my resulting transform.So if I make the interval longer, I get a tighter spaced comb of frequencies. So I will see a higher resolution for my sin(x)/x shape.It sounds counter-intuitive. I'm not adding any more information by padding with zeros...

author: JacquesBBB
author_rank: Frequent Contributor
timestamp: October 24, 2015, 02:40:13 am
content: Quote from: max_torque on October 23, 2015, 02:13:07 pmYou can largely improve on this with a technique used in particule accelerator tuning, that is called frequency analysis. Search google for "frequency map analysis".The idea is not to make fft, but search for the max of the continuous spectrum. For example over a finite interval of length T, for a pure periodic signal of frequency nu, the continuous spectrum amplitude will bef(x)=Sin(T(nu-x))/(T(nu-x))The max will be obtain for X=nu, exactly, independently of T.In practice, this will work as soon as you have a few periods of your signal, say at least 10. The limitation you will have will be given by the noise and the accuracy of the digitalisation.Better also to sample at 80kz for a 20kz signal, or at least 50khz.The question is not the high frequencies but the low ones. What are the main frequencies of your signal ?Not the harmonics .

author: SteveLy
author_rank: Regular Contributor
timestamp: October 24, 2015, 03:31:56 am
content: Quick and easy brute force way is to pad your signal with zeros before performing the DFT. Note that a DFT always gives the spectrum of your signal repeating indefinitely. That is, it assumes periodic boundary conditions in time. Zero-padding also helps minimise artifacts arising from that. [loosely speaking]

author: max_torque
author_rank: Super Contributor
timestamp: October 24, 2015, 11:28:05 am
content: I've just remembered why i hated maths so much at school!! (a long time ago now)So, i think i've got it, in English:We take an input signal, sampled at a known discrete interval, and this signal is "deconstructed" into the corresponding amplitudes of a frequency span of pure Sines, where the frequency of those Sines is assigned to each "bin". For each input signal, we take the real and imaginary components, weighted for the phase of each pure Sine bin, and add it on (integrate it) to that bin. At the end of the process, we should be left with all the energy of the original signal split down into our frequency bins. (assuming we have an integer number of input signal phases sampled).The Sine/Cosine weightings are calculated from the phase difference between the input signal and that of the pure Sines, as long as the input signal has a constant sample interval. So, frequency components of the input signal with the same periodicity as each pure sine get through to the appropriate bin. At the end, in order to correctly balance the energies between the original signal and the integrated totals in the bins, we divide each bin by the number of samples.So, for a pure +-1 amplitude sine input at say 1Khz, the 1Khz bin should end up with the value of 1 in it, and all other bins exactly 0.So, if we pad the incoming signal with leading or trailing zero's and recalculate the Sine/Cosine co-efficients as necessary (with a higher granularity, we can "zoom" into the signal frequency span in the bins. The side effect, is to artificially reduce the true magnitude of the input signal, because we are padding with zero's. This means that when we divide each integrated bin total, we will get too low a value, in comparison with the original, non padded signal total energy. As we know how much padding we are doing, then we can correct the output amplitude by reducing the division, to get back to the correct energy total.i think!

author: max_torque
author_rank: Super Contributor
timestamp: October 24, 2015, 02:13:12 pm
content: So,I fired up excel and wrote some VB to perform a stepwise DFT on a known input signal. As i can use a harmonically pure "input" signal of known amplitude to the function i can tell if i'm calculating the DFT correctly.If we take a phase accurate (over the sampling period) Sin wave input, without zero padding, we get the following:Which looks good. The Total spectrum energy matches that of the input signal, and we have no leakage. (input signal freq chosen to fall exactly on centre of a bin)If i "pad out" the same signal with the same number of zero's, then we double our DFT resolution. However, we need to correct the output amplitudes to reflect the known amount of zero padding. Doing that we get the following:Now we have significant leakage into the the adjacent bins.

author: SteveLy
author_rank: Regular Contributor
timestamp: October 24, 2015, 02:35:55 pm
content: The "leakage" comes from the fact that you have truncated a perfect sinusoid. Such a signal requires all those other frequencies (technically, the convolution of the DFT of a rectangular window and a pure harmonic). But if your signal is truly limited in time, i.e., it starts with zero before some point and ends at zero after, then zero padding will just explicitly utilise that fact.Try e.g. windowing the sinusoidal with a cosine^2 or cosine^4 function so it goes smoothly to zero at the ends. Also, it's not realistic that your sinusoidal perfectly "fits" inside the unpadded DFT domain: try one that has a period that is not an integer divisor of the unpadded interval, e.g. sin(sqrt(2)*8*pi*t/N). [t = discrete time index: 0,1,2...; N = number of samples in unpadded array]If you try to figure the frequency of such a signal from an unpadded DFT, you won't be able to easily read it off a plot of the DFT because it will fall between DFT samples (but you could do it in principle with appropriate mathematical analysis). And if you don't tame the signal by appropriate windowing you will get "leakage" even without padding (because the DFT assumes ongoing periodicity).But if you smoothly window (or even if not and don't mind the extra "leakage"/side-lobes) and zero pad, say by a factor of 4 or more, and do some rudimentary interpolation, you will get a very precise peak at the frequency of 4*sqrt(2) (in unpadded DFT units for the above example). The more you zero pad the more easy it will be to graphically pinpoint the exact frequency of the signal.If you wanted to do this for a demanding high-performance application, you would not go nuts with the zero-padding but do trickier but more efficient mathematical analysis on the unpadded DFT. But for quick and easy implementation zero-padding works a treat.

author: max_torque
author_rank: Super Contributor
timestamp: October 24, 2015, 03:07:37 pm
content: Quote from: SteveLy on October 24, 2015, 02:35:55 pmCould you expand on that^^ a bit more? Sounds interesting!As you say, for "perfect sinusoidal" inputs, we get a zero output when that signals frequency falls exactly between the bin centre frequencies!ie:In effect, the DFT is completely blind at all the points of overlap (assuming perfect phase / pure frequency of the input signal)

author: SteveLy
author_rank: Regular Contributor
timestamp: October 24, 2015, 03:30:19 pm
content: Quote from: max_torque on October 24, 2015, 03:07:37 pmI don't think I can here, at least not to the extent that it would be useful. It's a bit involved mathematically. It involves proper windowing and further processing on the DFT to extract information you're after. It's usually a fitting/optimisation procedure. Unfortunately I do not have specific references. I learnt the theory at uni and through my work as a physicist and developed my own tools. The basic theory is just properties of discrete and continuous Fourier transforms and how the two relate. The convolution theorem is also a big one, very useful.There are approximate methods that involve limited or no zero-padding and some basic interpolation. They compromise precision for ease of implementation but for many applications they are good enough and you don't need fancy maths. I have not used these much but if you search I'm sure you'll find good recipes.QuoteNo, it's not blind at all. You will get a broadened peak if the frequency falls between DFT sample points. You definitely won't get a zero output. The DFT always contains all the information the original signal contained. Something might have gone wrong with Excel (not the best tool for this job, but I appreciate that it may be the most convenient one to hand). I'll try to show you an example but maybe not tonight (I should be in bed asleep, it's 2:30am here).

author: JacquesBBB
author_rank: Frequent Contributor
timestamp: October 24, 2015, 03:50:09 pm
content: Quote from: JacquesBBB on October 24, 2015, 02:40:13 am@max_torqueA) You have not answered my question : what is the main frequency of you signal ? is it 20 khz, or if 20 khz is just one of the harmonics, what is the main frequency ?The way to handle the signal depends on this main frequency.B) You say you have 5 ms of signal. Is your signal a pulse of 5 ms of signal of frequency ?? , or the signal is stationnary and you get only 5ms of data.C) How important is the accuracy of the determination of the frequency. What is the precision you would like, ideally, to obtain ?With proper windowing and the method I mentioned to you above, you can attain incredible frequency resolution, even with a record over a very short time, much better than what would be obtain by direct FFT (2pi/T).

author: max_torque
author_rank: Super Contributor
timestamp: October 24, 2015, 04:02:47 pm
content: Quote from: SteveLy on October 24, 2015, 03:30:19 pmThanks you Steve, Your (and other posters) help is appreciated! (I'm a long way from a maths guru)Turns out i had the imaginary calc part still commented out of my code, oops, hence the zero's! (who could have imagined leaving out the imaginary bit eh??)

author: max_torque
author_rank: Super Contributor
timestamp: October 24, 2015, 04:12:49 pm
content: Quote from: JacquesBBB on October 24, 2015, 03:50:09 pmI have a signal, which is continuous in the time domain, however, it contains "frequency" information that i am interested in and "Noise" that i am not interested it. The signal and the noise has a range of between roughly 5 Khz and 20Khz, and the exact frequencies i want to monitor for amplitude are spread in this range.The short acquisition time dependency comes because i am already windowing the continuous time domain signal into short "chunks" when i know the signal to noise ratio is the highest (well, actually, i know the noise is the lowest). In effect i am grabbing the signal in small bursts when the noise (which is in the same frequency band as the signal, and can be significantly larger than the signal) is low.Having done this, what i then want to do is to transform that time domain signal into the frequency domain, with the best resolution possible (ideally around 100Hz wide bins). Doing this allows the "user" to identify the fundamental and harmonic(approx 3 of) components of the signal, and then the system will look specifically for the amplitude of those components. ie initial sweeps are wideband, then to minimise processor load and data reduction, a few critical sampling frequencies will be chosen.

author: Marco
author_rank: Super Contributor
timestamp: October 24, 2015, 04:55:31 pm
content: AFAICS for a single 5 msec chunk you will always have to make assumptions to find a peak at 100 Hz or better resolution. If you can assume sure there is only a couple of major frequencies and they are further than 200 Hz apart then a single padded chunk can give you all the resolution you need. Otherwise the information simply won't be there.Couldn't you combine two or more 5 msec chunks and align them on 10 msec boundaries? Lets say the first chunk is at X and the second chunk is at X+28 msec. You take the first chunk, insert 3 msec of zeros, insert the second chunk and pad it with zeros again to whatever length needed for the FFT. This would line up the frequency components and actually have enough information for 100 Hz bins to be meaningful without assumptions I think.

author: SteveLy
author_rank: Regular Contributor
timestamp: October 24, 2015, 05:30:50 pm
content: That is a nice challenging problem you have there, max_torque. Things going for you are that the frequency components you are looking for are only a few, well separated and harmonically related. The thing going against you is that you have a lot of noise. You say as much as signal at times: at those times you won't get any useful data! The 5ms capture interval is adequate for the frequency range you're interested in.You will want to sample the signal at a rate 80kHz or preferably higher. (Technically 40kHz sampling rate is the Nyquist minimum you need to reach 20kHz but bit of breathing space up top is good.) Let's say you sample at 100kHz and capture 512 sample points to get a 5.12ms interval. A DFT of that will give you approx 200Hz frequency bins (195.3Hz = 1/5.12ms to be precise). To up that to 100Hz, zero pad your signal to 1024 samples.This is how I'd go about the task for a first attempt (and this may be sufficient):Window your signal. Try, e.g., sin(pi*t/N)^2 or sin(pi*t/N)^4, where N is the size of the unpadded signal array and t is the discrete time index. The latter is a good approximation to a Gaussian and will give you minimal side lobes/"leakage". But try other windows as well (including rectangular, i.e., no window). There is a trade-off between minimising side lobes and how broad your peaks get. But since you have a lot of noise and only a few, well separated harmonic components in the signal, aggressive Gaussian windowing will likely be suitable (because it cuts the noise by smoothing the spectrum).Pad your windowed data to 2x or 4x the size (or 8x etc, see what works).Calculate the DFT power spectrum.Interpolate the power spectrum with cubic splines and look for the global maximum of the interpolation function (above the 5kHz cutoff). (This will improve your frequency precision by an order of magnitude or more above the padded DFT resolution.) Look for peaks near the harmonics as well.Repeat for multiple samples and check for consistency. Do some averaging, basic stats and error analysis. Tweak the parameters as needed.You might like to increase the capture interval to 10ms and window it down to an effective 5ms with even more aggressive (narrower) Gaussian windowing. If you use a Gaussian window make sure it's periodic (does not have kinks at the ends) and is properly shifted/scaled (so it goes to zero at the ends).PS. If/when you have sample data, upload it. I'll have a crack at it.

author: max_torque
author_rank: Super Contributor
timestamp: October 24, 2015, 06:27:48 pm
content: Hmm, more interesting points to think about and digest!Currently this is targeted to run on a STM32F4, which is plenty fast enough to sample the input and do the necessary sums after each sample (using a lookup Sin table to remove that calculation overhead (we know the sample interval, and how many samples we are taking, so we can pre-caclulate that phase information))I'll need to design a suitable analogue AC coupled front end to avoid aliasing etc, and possibly with a variable gain (because the signal+noise amplitude also varies, but that's another story....) and then i'll need to look into the ADC resolution i need as well (onboard ADCs on the STM are 12b)All good fun

author: mark03
author_rank: Frequent Contributor
timestamp: October 26, 2015, 02:50:52 pm
content: Quote from: max_torque on October 24, 2015, 04:12:49 pmIt's worth mentioning that the DFT may not be the best tool for this task, although it is probably the best way to get started. If you already have a model of your signal, which consists of noise plus a discrete number of sinusoids, then other techniques with whimsical names like MUSIC and ESPRIT (google it) are better. The catch, IIRC, is that your model needs to be accurate, i.e. you need to know a priori how many sinusoidal components are in there waiting to be identified.