Your thoughts on averaging methods - Page 1
Forum section: electronics / rf-microwave

author: EE-digger
author_rank: Frequent Contributor
timestamp: July 29, 2024, 03:48:53 pm
content: Sounds like it belongs in metrology but you guys are likely closer to it.Whether on a VNA or Spectrum analyzer, when you are averaging, I expect positive and negative spikes above nominal to reduce as something like the square root of N, the number of samples.Gonna remain nameless for now but I have one instrument which does this perfectly. Positive spikes are suppressed more because of the vertical log scale, but while the average is running, the noise "band" continues to shrink and is roughly centered on where the ultimate noise floor would be.On another instrument, if I run an average of 7, it goes kerchunk, kerchunk, kerchunk and the noise drops rapidly. If I run 32 or 64, you can barely see the noise reduction live. It eventually gets down but seems wrong. And the negative spikes aren't reduced "much".Whether 7 averages or 64, you should see the same rate of reduction as you watch it. Would you agree?Ok, it's a Siglent product. In SA mode, averaging looks proper but in VNA mode, you get the latter behavior which just looks wrong.

author: joeqsmith
author_rank: Super Contributor
timestamp: July 29, 2024, 04:00:59 pm
content: Demonstration of averaging using my own software. Obviously this is what I would expect, or I would change it. Now is it right??

author: EE-digger
author_rank: Frequent Contributor
timestamp: July 29, 2024, 05:18:12 pm
content: It seems to be pretty good but I need to watch a few more times. Watching in the 4:20 to 5:30 range, watching the salmon. The yellow looks like max hold maybe? The violet is min hold maybe?HP had some good discussions, possibly back in the 8753C days, which I need to dig up, or possibly their DMMs. I have a PNA and it looks like your salmon trace ... pretty much.added: for the most part, your salmon seems to converge on what it should if the sample count were infinite.

author: EE-digger
author_rank: Frequent Contributor
timestamp: July 29, 2024, 05:40:05 pm
content: From HP/Agilent E4445A spectrum analyzer manual:In continuous sweep,the specified number of averages is taken, then the averaging continues, with each new sweep averagedin with a weight of [ 1/average number ] and the old average reduced by multiplying it by [ (Average Number -1) / Average Number ].What I'm thinking isbad averaging on the VNA appears to follow this. But I had a new thought. The SA and VNA are looking so different because the magnitudes of the noise of each is quite different.Will need to run some tests and compare.

author: joeqsmith
author_rank: Super Contributor
timestamp: July 29, 2024, 06:06:04 pm
content: Quote from: EE-digger on July 29, 2024, 05:18:12 pmYes, min/max was enabled to give viewer a clear view of the peak vs the average.QuoteIf there is something you would like to see, I can try to run it for you. In the video, I just show the noise floor of the VNA over some range but can certainly take an actual measurement and show the reduction. If you would rather see the PNA, that's fine as well. This software supports it as well (really, that's the main use of the software).

author: EE-digger
author_rank: Frequent Contributor
timestamp: July 29, 2024, 06:17:21 pm
content: Quote from: joeqsmith on July 29, 2024, 06:06:04 pmThanks for the offer and I'll keep that in mind.I think I may have found the problem and they have problems in other functions for "similar" reasons.I set up the SA function, then the VNA function, creating 20dB of on screen noise for each.With 100 averages, the SA, within about 5 averages, has already knocked the noise way down and "in place" vertically.With 100 averages, the VNA very slowly travels down vertically, the entire noise band does this. I'm suspicious that they are missing the 2nd half of the weighting equations, the reduction by [ average number -1 / average number ]. This may explain why the entire noise band travels downward and with very little reduction in p-p noise as it goes.Do you use the equation like HP?

author: joeqsmith
author_rank: Super Contributor
timestamp: July 29, 2024, 06:46:36 pm
content: describing min/maxhttps://youtu.be/M60Qalyx-r0?t=109describing the round robin LIFO averagehttps://youtu.be/M60Qalyx-r0?t=424I am not doing something like taking the (old * (num_avgs-1) + new) / num_avgs for the new data, but certainly could if I needed the resources.If you wanted, I could set the same sweep range, number of points, number of averages and run it looking at the floor. The problem is getting data off the LiteVNA takes some time. So if you wanted to do an A/B compare, keep the number of samples in the sweep range small. 401 or so should be fine.

author: EE-digger
author_rank: Frequent Contributor
timestamp: July 30, 2024, 01:10:39 pm
content: After more testing, all I can tell is that averaging for this VNA is broken.For averaging of 7, noise is rapidly reduced, then the reduction drops off and probably changes to some sort of moving window.For averaging of 100, noise reduction can barely be seen and at 100 averages, is NOT AS GOOD AS 7 AVERAGES !The spectrum analyzer side is good. Either 7 or 100 will rapidly reduce noise up front, then the 100 continues to reduce it further, as one would expect.Anyway, this is going onto a list that's going to Siglent soon.Thanks for your help and input. Just curious though, are you saying that your software uses the LIFO based method currently?

author: joeqsmith
author_rank: Super Contributor
timestamp: July 30, 2024, 02:35:49 pm
content: For every factor of two averages, the SNR will improve 3dB. S21 was measured with the LiteVNA64 (latest HW revision). Port 2 was terminated to 50 ohms and an IFBW of 4kHz. No calibration was performed as we are just looking at a relative change. Shown with none, 10 and 100 averages. We see a reduction of 10dB per step as expected. I take it that is not what you are seeing (you never actually present any data).Yes, I use a LIFO which is currently limited to 100 samples. I could switch to a moving calculation beyond some number of averages, or just always use that method. With the PC, I wasn't too concerned about the resources but changing would require little effort if it became a problem.Just for fun, showing the return loss with an open and load attached sweeping to 9GHz with the same settings. No averaging is shown and the unit was calibrated using the low cost short and open. The load is a sorted Mini-Circuits ANNE terminator. The LiteVNA was sold as a 6.3GHz device and harmonics are not used. Dislord provided firmware that supports harmonics beyond the 6.3GHz limit. You can clearly see where it switched. I have been experimenting in the 8-12GHz region with the LiteVNA but it required a hardware conversion to get there. For the $120 we spent for these, they are an excellent value!!

author: joeqsmith
author_rank: Super Contributor
timestamp: July 30, 2024, 03:09:02 pm
content: I seldom use additional filtering or averaging beyond the IFBW. I mentioned the harmonic mode for example with the LiteVNA and showed the return loss. Here is that same setup attached to a homemade microwave high Q cavity that is tuned to 9.05GHz. Averages was set to 100 but you can see it has only acquired the first 80. That cavitiy is going to have a much better return loss than the 2.5dB measured. It should be more in the order of 60-70dB. Averaging obviously isn't going to get me there.The Narda plots is showing the return loss looking at a Narda waveguide terminator over a frequency range of 8-12GHz. The dark red is showing my current poor man's frequency converter with the LiteVNA64. Rather than maybe 5dB, we can now measure 70dB or so and can even get useful data up to about 12.5GHz with this setup.There's no substitute for good hardware.

author: EE-digger
author_rank: Frequent Contributor
timestamp: July 30, 2024, 03:19:35 pm
content: I apologize for lack of data. My equipment and computers are in different spaces at the moment.As you did, this composite give the best picture of how bad the problem is.As the filename says, red(orange) is 1 sample, yellow is 10 and cyan is 100.Also, the p-p magnitude of the noise "band" is never reduced. For purely asynchronous noise, the band should be reduced to a line at infinite averages.

author: joeqsmith
author_rank: Super Contributor
timestamp: July 30, 2024, 04:06:58 pm
content: You show roughly 20dB for 100 average, which is expected.QuoteObviously we are looking at log scales to view those details. If I plot that same data I previously showed for the LiteVNA64 on a linear scale of course we see what you describe. I now suspect we have more a new user problem than a Siglent firmware problem. I wouldn't be too quick to point out their flaws until you have a firm grasp of the basics. That, or I am really missing your point, which is entirely possible!!

author: RoGeorge
author_rank: Super Contributor
timestamp: July 30, 2024, 04:27:17 pm
content: Quote from: EE-digger on July 30, 2024, 03:19:35 pmThe noise amplitude should be smaller and smoother when averaging many samples. I suspect what you see is an artifact caused by how the averaging is done internally, combined with how a very wide bandwidth is crammed in the width of the screen, but I'm not very sure.Can you try the same 1/10/100 averaging, but for a much narrower frequency sweep, for example only a kHz or so instead of a 4.5GHz sweep?My guess is the pp noise should start to behave as expected when the frequency sweep is not that wide, and the RMS noise should then decrease a lot with the number of averaged samples.

author: EE-digger
author_rank: Frequent Contributor
timestamp: July 30, 2024, 04:39:49 pm
content: Quote from: joeqsmith on July 30, 2024, 04:06:58 pmNot new user by a long shot. Won't go into details here.This plot is from an Agilent 8357A PNA. top/gold = 1 av. mid/magenta = 10av. bottom/tan = 100av.The p-p noise band is not reduced much and I believe RoGeorge is correct that the noise statistic comes into play. Don't have time to repeat right now.

author: EE-digger
author_rank: Frequent Contributor
timestamp: July 30, 2024, 04:42:29 pm
content: joeqsmith - I think you may have misread my Siglent plot. Unlike the PNA, the Siglent, from top to bottom, is 1,100, 10. 10 averages is much better than 100. Also, I just corrected the PNA file name to reflect the top to bottom.

author: joeqsmith
author_rank: Super Contributor
timestamp: July 30, 2024, 05:23:54 pm
content: Quote from: EE-digger on July 30, 2024, 04:42:29 pmYes, I misinterpreted the graph. It was not clearly marked, and I assumed blue was 10 and yellow was 100 averages. What you describe is not at all expected. The p-p*** I wouldn't have expected to change enough on a log scale to give you what you are asking.I wonder if you plot the noise vs averages on a graph, what you would see. Does their software have such a feature or have you written any custom software for it yet that would make this an easy task?I have that same old PNA relic that we could use as a sanity check as well if you would like me to look at something with it.***QuoteJust to be clear as my post may have been a bit confusing. I am suggesting what I have shown has exactly the same P-P noise but its the visual effect when looking at the log scale that is making it look like there is little difference. 20dB is a pretty substantial difference in noise level IMO.Just the background of what was mentioned:https://helpfiles.keysight.com/csg/N52xxB/S2_Opt/Trce_Noise.htmhttps://testworld.com/wp-content/uploads/understanding-and-improving-network-analyzer-dynamic-range.pdf

author: joeqsmith
author_rank: Super Contributor
timestamp: July 31, 2024, 12:18:24 am
content: Shown comparing the rolling average with the LIFO with 100 averages. Basically no difference as expected.

author: joeqsmith
author_rank: Super Contributor
timestamp: July 31, 2024, 12:23:43 am
content: Shown with the running average off, 10, 100 and 1000 averages.

author: joeqsmith
author_rank: Super Contributor
timestamp: July 31, 2024, 06:02:42 pm
content: Article on smoothing and averaging.https://www.microwaves101.com/encyclopedias/smoothing-is-cheating

author: Kleinstein
author_rank: Super Contributor
timestamp: July 31, 2024, 06:28:28 pm
content: When it comes to averaging there is a fundamental difference between the SA and VA mode. In VA mode the signal is a complex / vector result and averaging is relatively easy and no fundamental problem, as the result of interest in coherent with the excitation. Averaging can improve the SNR. There is however still the limit from the calibration / unwanted couplings and spurious signals.In SA mode averaging is a different thing as the result is power for a given BW per bin. Averaging changes the result to average power and not the one instant power. There can still be some sense in averaging, especially to get rid of the down peaks, but don't expect that much gain in background or SNR.So not 10 dB gained from 10 fold averaging.

author: EE-digger
author_rank: Frequent Contributor
timestamp: August 07, 2024, 02:58:13 pm
content: Quote from: Kleinstein on July 31, 2024, 06:28:28 pmSorry for the lengthy absence.You are indeed correct for the SA mode and that is what's observed.For the VNA mode, I've repeated testing a number of times now and my final summary (I have to stop beating the dead horse)is this:When selecting 10, 100, 1000 or more averages on the PNA, the noise always drops by XdB for the first N averages. For instance, on 1000 averages, the noise within the first 10 has dropped the same as a selection of 10 averages would have provided. And visually, those first 10 averages are always at the same rate ... fast. On this unit, those first 10, or N become VERY SLOW so that at 100 or 1000 averages, you can barely watch (or see) the improvement in real time.On this unit, I finally discovered that if I walk away for a long time, like maybe 10 times the 100 average time total, the noise floor does eventually come down to what's expected (compared to a competitor or lower end from same vendor).And back to Joeqsmith- yeah, it's a relic but a darned good one. The Asian units can't compare on speed, accuracy or range. But then again, it had a $65,000 price tag in 2001 (or so). They also can't compete on weight. That's why it's going away, before I need a hernia repair.

author: joeqsmith
author_rank: Super Contributor
timestamp: August 07, 2024, 04:30:26 pm
content: Quote from: EE-digger on July 30, 2024, 04:42:29 pmQuoteQuoteI am confused. Your Siglent VNA with average enabled is less noisy with 7 than 100, unless you allow the 100 to run 10X longer or 10000 averages?QuoteI've owned mine for a few years now. I knew the shortcomings before buying it and was willing to overlook them. I doubt I could come close to the performance/price with anything new out of China. That seems to be true with the scopes as well.

author: EE-digger
author_rank: Frequent Contributor
timestamp: August 07, 2024, 04:55:07 pm
content: QuoteQuoteI am confused. Your Siglent VNA with average enabled is less noisy with 7 than 100, unless you allow the 100 to run 10X longer or 10000 averages?QuoteI've owned mine for a few years now. I knew the shortcomings before buying it and was willing to overlook them. I doubt I could come close to the performance/price with anything new out of China. That seems to be true with the scopes as well.Have a look at my earlier reply #10. The three traces are for 1,100, then 10 from top to bottom. Each trace was stopped when it reached full count.If you let the 100 count trace continue for many more full count periods, the noise comes down to where it should be ... similar vertical distance from 1 to 10 as from 10 to 100. But this takes a handful of minutes.This has been shared with the mfgr, along with some math errors.---I picked up a 1200 x 800 , 10" LCD module to play with on the PNA. The results were stunning. Bright, sharp and even four windows looked great. But I gave up on the PNA so that effort ended. The new VNA is pretty decent and I have no choice but to keep it. And it weighs around 12lbs.

author: joeqsmith
author_rank: Super Contributor
timestamp: August 07, 2024, 05:28:01 pm
content: When you hear back from Siglent, please provide us with an update.