Fast Power Grid Frequency Estimation - Page 1
Forum section: electronics / projects

author: ali_asadzadeh
author_rank: Super Contributor
timestamp: March 01, 2021, 11:35:47 am
content: Hi,I have read the attached paper.I want to prove it using a numerical example, I have problem how to calculate the hp(n) values,Let's suppose that I have a sample signal which I have sampled with an ADC and for simplicity I will use Matlab to make the signal. (the signal has a 50.1HZ frequency) and I want to estimate it with this paper approach,Code:fs = 12800; % Sampling frequency (samples per second)dt = 1/fs; % seconds per sampleStopTime = 0.02; % 20 mili seconds or a 1 full cycle of 50Hz ACt = (0:dt:StopTime-dt)'; % secondsF = 50.1; % Sine wave frequency (hertz)X = sin(2*pi*F*t); % Generate sine waveacording to paper it seems that I should cascade some moveing avrage filters like this codeCode:M = 256;h = ones(1,M)./M;y1 = filter(h,1,X);y2 = filter(h,1,y1);y3 = filter(h,1,y2);Then I calculated the convolution of the X and y3 to determine the frequencyCode:F = conv2(X, y3);And now getting frequency from two samples of the F, like thisCode:DF = ((F(1) - F(256)) * 256 * 50 )/ (2*pi*255)So estimated f should be like thisCode:Fest = 50 + DF;Which is 24.0013 and it’s far from the 50.1 answer according to the paper results.So the question is that am I doing it in the right way? Because I do not get some parts of the paper.Any help would be highly appreciated

author: ali_asadzadeh
author_rank: Super Contributor
timestamp: March 04, 2021, 08:04:43 am
content: I need some help, please help me find what am I doing wrong?

author: ali_asadzadeh
author_rank: Super Contributor
timestamp: March 06, 2021, 07:39:33 am
content: I may find some hints about why I'm getting wrong results, the question is how to generate a complex sin wave in matlab?

author: rstofer
author_rank: Super Contributor
timestamp: March 06, 2021, 02:25:19 pm
content: Here's a simple example of a fundamental along with the 3rd and 5th harmonic.Code:t = 0:0.001:2*pi;y = 0.636620 * sin(t) + 0.212207 * sin(3*t) + 0.17324 * sin(5*t);plot(t,y)It approximates a square wave. The coefficients have been scaled for some reason I fail to recall. Nevertheless, the shape is right.

author: ali_asadzadeh
author_rank: Super Contributor
timestamp: March 07, 2021, 06:46:24 am
content: Thanks rstofer, do you have any Idea what's wrong with my code in matlab? How should I evaluate the results of the paper in matlab?

author: ali_asadzadeh
author_rank: Super Contributor
timestamp: March 07, 2021, 07:07:46 am
content: So, Now I have done a little bit more, Now I'm generating a complex reference sine wave, The things that I have done are as followCode:clearclc%ADC sin wave, it represent the data that I have captured with the ADC%and want to estimate the frequencyADCsin = dsp.SineWave(1,50.1);ADCsin.ComplexOutput = false;ADCsin.SampleRate = 12800;ADCsin.SamplesPerFrame = 256;ADC = ADCsin();plot(ADC);grid onhold on%ref sin wave with 50Hz complex valueRefsin = dsp.SineWave(1,50);Refsin.ComplexOutput = true;Refsin.SampleRate = 12800;Refsin.SamplesPerFrame = 256;ref = Refsin();plot(imag(ref))%calculating xsxs = ref .* ADC;%calculate the CAFM = 256;h = ones(1,M)./M;y1 = filter(h,1,ref);y2 = filter(h,1,y1);y3 = filter(h,1,y2);%doing the actual final convolutionF = conv2(xs, y3);%calculating the final resultFest = 50 - (angle(F(4)) - angle(F(3)) * 256*50)/(2*pi*90)the answer is clearly wrong! so what I have done wrong?

author: penfold
author_rank: Frequent Contributor
timestamp: March 07, 2021, 08:46:59 am
content: Is your application of the filter correct? Do you get the same result as formally defining it by repeated convolutions of h_avg as the paper does, theres a part of me saying that the resulting length of the filter affects the result.. do you get a similar result from a single application of the filter() function as you do with two applications or is it entirely different?

author: ali_asadzadeh
author_rank: Super Contributor
timestamp: March 07, 2021, 01:28:52 pm
content: I get different answers, you can test the code in Matlab and give feedback if you find something, thanks.

author: nctnico
author_rank: Super Contributor
timestamp: March 07, 2021, 01:57:24 pm
content: I'm not quite sure whether this is such a good method. From a quick glance: What it basically does is demodulating the frequency offset into a near DC offset which is prone to being influenced by harmonics without being able to filter these properly.Regarding the wrong result: I'm missing the arg() from the final calculation in your code. I have not dug too deep into the paper but is seems it is using the angles to determine the frequency offset.

author: ali_asadzadeh
author_rank: Super Contributor
timestamp: March 07, 2021, 03:09:22 pm
content: QuoteIsn't the arg means the angle? or am I missing something?

author: penfold
author_rank: Frequent Contributor
timestamp: March 07, 2021, 03:12:12 pm
content: Quote from: ali_asadzadeh on March 07, 2021, 01:28:52 pmI'm not running Matlab (I'm only on gnu octave) so your revised code doesn't play well without me installing dsp toolbox etcWhen you say different answers, do you mean different answers from convolved vs successive filter applications or different for each filter order.Have you tried using the graph of F = conv2(xs, y3) to see how its affecting the results? I noticed there sign change with the original (incorrect) code which suggests that there's something very wrong with the filter application.

author: ali_asadzadeh
author_rank: Super Contributor
timestamp: March 07, 2021, 03:24:21 pm
content: QuoteIf you have an octave code that works, Maybe I can test it hear in MATLAB too. and I mean from different results if I use y1,y2, or y3 for convolution, besides the paper says it has about mHz accuracy and none of the answers are even close

author: nctnico
author_rank: Super Contributor
timestamp: March 07, 2021, 03:29:01 pm
content: Quote from: ali_asadzadeh on March 07, 2021, 03:09:22 pmYou are right. I overlooked that. Angle() should do the same as arg() (Why didn't they call it arg() ?)I took a look at the paper again. I think you need to carefully go through each step and check whether it is right. Note that the paper isn't the final version so it may contain errors or is missing a step. I think the CAF filter may need more research on how this should be implemented. An easier path could be to find the final version or a paper which describes the same method in a more clear way.

author: gf
author_rank: Super Contributor
timestamp: March 07, 2021, 08:22:22 pm
content: Try this:Code:M = 256;N = 256*M;fs = 50*M;t = [0:N-1]/fs;d_t = t(2);% reference signalref = exp(-1i*2*pi*50*t);% frequency sweep 50-52 Hzf = 50 + 2*[0:N-1]/N;% input signalsig = sin(2*pi*f.*t);% down-convertbbsig = sig.*ref;% 3rd oder moving average filterbbsig = conv(bbsig, ones(1,M)/M);bbsig = conv(bbsig, ones(1,M)/M);bbsig = conv(bbsig, ones(1,M)/M);% remove settling of filterbbsig = bbsig(4*M:end-4*M);d_angle = diff(angle(bbsig));% unwrap angle wrap-around from 2*pi -> 0d_angle(d_angle<-pi) += 2*pi;d_angle(d_angle>pi) -= 2*pi;% still unclear why this is off by a factor of 2 ???% d_f = d_angle/(2*pi*d_t);d_f = d_angle/(2*pi*d_t) / 2;plot(50+d_f);ylim([45 55])title("frequency vs time")grid onIMO the key is that the algorithm is supposed to operate on a cotinuous stream of samples, not on a single block of size M. I simulate this stream by a much larger block and discard leading and trailing samples after filtering which are garbage until the filter has settled. The calculated frequency difference is still off by a factor of 2. I did not find the mistake yet. So I added an extra division by 2. Apparentyl that works, but that's just a hack (I'm not happy if things work with a hack, and I don't understand why).

author: ali_asadzadeh
author_rank: Super Contributor
timestamp: March 08, 2021, 06:57:47 am
content: Thanks for the code gf,this part of code does not work in matlabCode:d_angle(d_angle<-pi) += 2*pi;d_angle(d_angle>pi) -= 2*pi;what did you want to do? so I can redoe it in matlab

author: gf
author_rank: Super Contributor
timestamp: March 08, 2021, 06:07:53 pm
content: Quote from: ali_asadzadeh on March 08, 2021, 06:57:47 amThe code works in Octave. I guess the operators += and -= are possibly not understood by Matlab. They have the same meaning as in C, i.e. A+=B is a short form for A=A+B.

author: gf
author_rank: Super Contributor
timestamp: March 08, 2021, 09:07:21 pm
content: Quote from: gf on March 07, 2021, 08:22:22 pmDid some more investigations. The previously assumed need for an extra division by 2 turned turned out to be definitively wrong (which is a good sign - it would not fit with my phi=onega*t worldview). So I removed it again. For constant frequencies the result looks actually correct now. It rather seems to be the transient response which looks pretty strange (see plot, orange is actual frequency, blue is measured frequency). After the transition, it returns to the correct value, though. Strange

author: gf
author_rank: Super Contributor
timestamp: March 08, 2021, 10:34:12 pm
content: Found the problem now. My simulated input signalCode:sig = sin(2*pi*f.*t);contains phase discontinuities. The method does not like them, but they are unlikely in a real power grid. Replacing it with a correct, phase-continuous chirpCode:f(1) = 0; sig = sin(cumsum(2*pi*f*d_t));solves the problem and the output looks like expected nowActually I find this an interesting way to implement a kind of streaming hilbert transformerEDIT: Attached is a plot of the (simulated) measured frequency, starting at 51Hz, stepping down do 49Hz, ramping up to 51Hz again, and staying at 51Hz.Risetime of step response is about 500 samples, i.e. approx. two 50Hz-cycles or ~40ms.Here is the latest version of my Octave code:Code:M = 256;N = 256*M;fs = 50*M;t = [0:N-1]/fs;d_t = t(2);% reference signalref = exp(-1i*2*pi*50*t);% simulated frequency profile:% start at 51Hz, step down to 49Hz, ramp up to 51Hz% and remain at 51Hzf = [repmat(51,1,N/4) repmat(49,1,N/4) repmat(51,1,N/2)];f(N/2:N/2+16384-1) = 49 + 2*[0:16384-1]/16384;% create phase-continuous chirp signal for the frequency profilephi = 2*pi*f.*d_t; phi(1) = 0;sig = sin(cumsum(phi));% down-convertbbsig = sig.*ref;% 3rd oder moving average filterbbsig = conv(bbsig, ones(1,M)/M);bbsig = conv(bbsig, ones(1,M)/M);bbsig = conv(bbsig, ones(1,M)/M);% discard settling of filterbbsig = bbsig(3*M:end-3*M);d_angle = diff(angle(bbsig));% unwrap angle wrap-around from 2*pi -> 0d_angle(d_angle<-pi) += 2*pi;d_angle(d_angle>pi) -= 2*pi;% calculate instantaneous frequencyd_f = d_angle/(2*pi*d_t);plot(50+d_f);ylim([45 55])title("frequency vs time")grid on

author: ali_asadzadeh
author_rank: Super Contributor
timestamp: March 09, 2021, 06:55:34 am
content: Dear gfHi,Thanks for the feedback ,would you please resend your new complete code, also would you please use a Complete code for += and -= so it can be done in matlab too, so I can test it here too.

author: gf
author_rank: Super Contributor
timestamp: March 09, 2021, 08:19:18 am
content: The complete code of my simulation was already attached to the previous message. It is not more than that. A streaming DSP implementation would of course look different, processing one sample at a time when it arrives, and not operating on blocked data. The Octave assignment operators are documentedhere. I leave the trivial conversion to Matlab code as an exercise for the "Digital Expert from 8-bits to 64-bits". Octave syntax isvery similarto Matlab anyway. In order to understand the algorithm described in the paper you need to understand the concepts ofAnaltic SignalsandInstantaneous Phase and Frequency, how to generate an analytic signal from a real signal, and a few DSP basics. One property of the chosen SMA filter is that is has zeros at N*50 Hz, cancelling harmonics at N*50Hz (and still suppressing N*(50Hz+df) pretty well, for small df). With a filter order of only 2, there is still noticable ripple from the -(100Hz+df) mixing product left in the filtered signal, so order 2 is not enough. With an order of 3-4, suppression is already pretty good, up to frequency deviations of a few Hz. You could even go higher, but drawback of a high order is that it reduces the response time - so it's a trade-off.

author: ali_asadzadeh
author_rank: Super Contributor
timestamp: March 09, 2021, 03:13:37 pm
content: thanks, I'm a little bit confused, can we do it on a 256 samples only, where is the estimated frequency in your code? my final goal is to fully understand this approach and try it on an STM32

author: gf
author_rank: Super Contributor
timestamp: March 09, 2021, 06:48:09 pm
content: The algorithm per se does not require any particular blocks, but can be reazied as a DSP which works on an continuous stream of samples, captured at a fixed sampling rate. For each sample you retrieve from the ADC and feed into the DSP engine, you can get one frequency estimate out.OTOH, the length of each of the 3-4 cascaded moving average filter stages must indeed be 20ms (i.e. exactly one period of the nominal mains frequency). For a sampling rate of 12800 Sa/s, each filter stage needs to have 256 taps then. The condition also implies that the sampling rate must be an integral multiple of the nominal mains frequency, too.The simulation is not implemented as a streaming DSP, but rather in an "Octave/Matlab-friendly" way. It creates 64k samples (~5 seconds of) of simulated input data with a given, simulated frequency profile (in the variablef), and applies the steps of the algorithm to the whole vector in parallel (as it is usual practice in Matlab/Octave). The simulated input signal is the vectorsig, and the vector of estimated frequencies is50+d_f, which is eventually plotted at the end (-> one estimated frequency per sample).

author: ali_asadzadeh
author_rank: Super Contributor
timestamp: May 07, 2021, 09:08:51 am
content: I managed to edit the matlab code, now it's working as expected,Code:clearclc%ADC sin wave, it represent the data that I have captured with the ADC%and want to estimate the frequencyADCsin = dsp.SineWave(1,50.124);ADCsin.ComplexOutput = false;ADCsin.SampleRate = 12800;ADCsin.SamplesPerFrame = 768;ADC = ADCsin();plot(ADC);grid onhold on%ref sin wave with 50Hz complex valueRefsin = dsp.SineWave(1,50);Refsin.ComplexOutput = true;Refsin.SampleRate = 12800;Refsin.SamplesPerFrame = 768;ref = Refsin();plot(imag(ref))%calculating xsxs = ref .* ADC;%calculate the CAFM = 256;h = ones(1,M)./M;y1 = filter(h,1,xs);y2 = filter(h,1,y1);y3 = filter(h,1,y2);%calculating the final resultFest = 50 - (angle(y3(end)) - angle(y3(end-1))) * 12800/(2*pi)Now I want to know if it's possible to make the algorithm simpler, is there a way to combine all the 3 filters to a pre-calculated fixed stream to only do the convolution one time,I mean is there a way to combine all these 3 lines of code to a one time filter?Code:%calculate the CAFM = 256;h = ones(1,M)./M;y1 = filter(h,1,xs);y2 = filter(h,1,y1);y3 = filter(h,1,y2);So that we do the convolution only one time?

author: gf
author_rank: Super Contributor
timestamp: May 07, 2021, 12:54:32 pm
content: Quote from: ali_asadzadeh on May 07, 2021, 09:08:51 amThe combinedd filter kernel is h3=conv(h,conv(h,h)).But there is no computational saving at the end, since h3 is (almost) 3x as long as h then.For a µC implementation, a runnung average filter has the advantage that you can calculate it with only one addition and one subtraction per sample (and storage space for keeping the last M samples). For three filters in series, it is still only 3 additions and 3 subtractions per sample then. OTOH, convolution with h3 were (3*M-2) multiplications and (3*M-2) additions per sample, i.e. much more expensive, particularly if M is large.

author: ali_asadzadeh
author_rank: Super Contributor
timestamp: May 07, 2021, 07:26:21 pm
content: Thanks gf, is there some clever way of combining the first convolution into the 3 stage filter, and get a vector so, all the work could be done with a single run of conv?I mean combining the ref signal with the 3 stage filters.

author: gf
author_rank: Super Contributor
timestamp: May 07, 2021, 09:01:59 pm
content: Quote from: ali_asadzadeh on May 07, 2021, 07:26:21 pmNot normally. The multiplication with the referrence signal is not a convolution.A special case applies to your program, though. If you areonlyinterested in y3(end) and y3(end-1), then you can omit the calculation of y3(1:end-2). In this case the convolution degrades and could be replaced by two dot product calculations, and the multiplication with the reference signal could be merged into one term of each dot product. But as I said, that's a special case, applying only to your program as is. The algorith from the paper is rather suposed to be fed with an infinite continuous streams of samples, and to calculatae a frequency estimate foreach sample, and not just once every 768thsample.