The "double difference method" - any pointers? - Page 1
Forum section: electronics / general

author: Cerebus
author_rank: Super Contributor
timestamp: February 15, 2016, 07:55:39 pm
content: I was reading a voltage reference related paper the other day that included a casual, throw away, reference to the"double difference method"in the specific context of using said method to unambiguously identify which of a collection of voltage references was producing noisy or otherwise anomalous data. All in the further context of not having an absolute reference to work from. i.e. They were developing voltage references that were, in some fashion,beyondthe references they already had available.I've tried searching for a"double difference method"that could be the one in question but have come to a dead end. I've encountered mentions of using a similarly named method for finding the epicentre of earthquakes and in the social sciences - both sources of highly noisy data - but I haven't found an unambiguous enough description to either know if I've found the right thing or to apply the method from that discipline to the original question.Has anyone got useful pointers to a clear description of this method - either specifically in the context of "which voltage reference is lying" or more generally as a statistical method of unambiguously identifying outliers in noisy data?The volt nuts home-brewing voltage references can hopefully immediately see how interesting and useful this could be.Ian

author: AndyC_772
author_rank: Super Contributor
timestamp: February 15, 2016, 08:07:27 pm
content: I guess that if you have three references, called A, B and C, and measure each of them in turn against one of the others, then you might get results along the lines of:A vs B = noisyA vs C = quietB vs C = noisyIn this case, A and C are both quiet, so the comparison between the two yields a quiet result too. But, since B is noisy, measuring it with respect to either of the two others yields a noisy result.I'm not sure whether that type of comparison would actually warrant a name as such, but it seems like the obvious way to identify which is the outlier. Moreover, it seems fairly apparent that if the three references were 'quiet', 'noisy' and 'noisier', then the set of comparisons between them would show different degrees of noisiness, from which the relative merit of each reference could be readily determined.

author: Cerebus
author_rank: Super Contributor
timestamp: February 15, 2016, 08:50:40 pm
content: Quote from: AndyC_772 on February 15, 2016, 08:07:27 pmThe problem is thatallvoltage references are noisy and drift, the good ones and the bad ones. So, at any moment a good voltage reference can have wandered off positive from the 'right' value for a bit, another good reference can have wandered off negative and the crappy bad one just happens at that moment to have the 'right' value - how do you tell which is which. The answer is, obviously, long term statistics. But how do you tell - and here comes the magic word - unambiguously in the short term?I found the paper in question again. Here's the throwaway reference:QuoteThe hint I have, from the other uses I found, is that 'difference' here is used in the same sense as 'differential'.

author: Cerebus
author_rank: Super Contributor
timestamp: February 15, 2016, 11:11:39 pm
content: Quote from: DiligentMinds.com on February 15, 2016, 10:28:52 pmNo, it's definitely "double difference" in the quote; and the apparently related statistical material I could hunt down (seismology, social science) is what provided the hint of "differential" but that's in the "derivative" sense (e.g. dv/dt or d2v/dt2) not the "differential pair" or "differential measurement" sense.Hopefully someone will come along soon who recognises the term. I'm fairly confident it won't be a social scientist but there might be a seismologist kicking around.

author: quantumvolt
author_rank: Frequent Contributor
timestamp: February 16, 2016, 02:29:52 am
content: Quote from: Cerebus on February 15, 2016, 08:50:40 pmAs you quote some 'scientific' paper here - wouldn't it be scientific of you to name the source. Could help you and be useful for others ...Anyway, here's your stuffhttps://en.wikipedia.org/wiki/Difference_in_differencesThis qoute fromhttp://web.worldbank.org/WBSITE/EXTERNAL/TOPICS/EXTPOVERTY/EXTISPMA/0,,contentMDK:20188244~menuPK:384339~pagePK:148956~piPK:216618~theSitePK:384329,00.html#differencegives a hint on how to use the method:'Double difference or difference-in-differences methodsAn estimation method to be used with both experimental and non-experimental designDouble difference or difference-in-differences methods compare a treatment and a comparison group (first difference) before and after the intervention (second difference). This method can be applied in both experimental and quasi-experimental designs and requires baseline and follow-up data from the same treatment and control group.A baseline survey is conducted for the outcome indicators for an untreated comparison group as well as the treatment group before the intervention followed by a follow-up survey of the same sampled observations as the baseline survey after the intervention. If the sampled observations tend to differ in the follow-up survey from the baseline survey, then they should be from the same geographic clusters or strata in terms of some other variable.The mean difference between the “after” and “before” values of the outcome indicators for each of the treatment and comparison groups is calculated followed by the difference between these two mean differences. The second difference (that is, the difference in difference) is the estimate of the impact of the program (A special case of double differences is “reflexive comparison” that only compares the treatment group before and after the intervention).'

author: awallin
author_rank: Frequent Contributor
timestamp: February 16, 2016, 06:00:48 am
content: Quote from: Cerebus on February 15, 2016, 11:11:39 pmTry searching for 'three cornered hat' or 'N-cornered hat' - quite common method in oscillator/clock measurements.It works well if the compared standards are roughly equal in performance and fulfil the other assumptions of the method (i.e. noise is not correlated between the standards and they look like a stationary noise process)http://www.wriley.com/3-CornHat.htm

author: Cerebus
author_rank: Super Contributor
timestamp: February 16, 2016, 12:01:20 pm
content: Quote from: quantumvolt on February 16, 2016, 02:29:52 amIn English, putting 'scientific' in quotes like that implies sarcasm which I hope was not your intent. As there's a Thai flag next to your name I'll make the assumption that you're not a native English speaker and give you the benefit of the doubt. If you are a native English speaker then the attitude isn't helpful to anybody and may I suggest that you go forth...QuoteActually the word would be "rigorous" not "scientific".I tried to do everybody a favour and actually searched all and read most of the whole Metrology category before diving in with questions. The paper is Spreadbury which appears well known enough amongst the cognoscenti here and hence isn't in itself worth re-visitingper se. Quoting a reference would just have been so much extra clutter to detract from the actual question.QuoteThat's exactly representative of the kind of thing I found that didn't quite jibe with the original context. There's no"intervention"in our case to compare for and I can't quite seem to form a hypothetical"intervention"(like"God just make reference 3 especially noisy") to form a hypothesis from that fits the method. Or if that is the right kind of hypothesis I can't just quite see how to apply it here.An aside:Statistics, as of itself, doesn't make my head hurt but most people's attempts to teach it, explain it or explain how to apply it do. As the bits I do understand seem quite straight forward and simple - for example, the central limit theorem - I have a strong suspicion that most of the people I've got this feeling from don't really understand it properly themselves. The worrying thing about this feeling of suspicion is that I get it from the people who most need to properly understand the subject - University lecturers tasked with teaching it, medical researchers, social scientists, polititical scientists etc. etc.To keep quantumvolt happy, herewith a complete bibtex citation for the paper in question:

author: Cerebus
author_rank: Super Contributor
timestamp: February 16, 2016, 12:25:26 pm
content: Quote from: awallin on February 16, 2016, 06:00:48 amI don'tknowif that's what the original author  was referring to by another name but it certainly makes a lot sense in this context and looks like a useful approach to solving the problem. I have some suspicion that it is the same and that Spreadbury was playing fast and loose with the termsdifferenceandvariance.It's the throw-away use of the term "double difference method" without a citation or further explanation that lead me to believe that this was a well known method that I just, for whatever reason, had never heard of before.Thanks, that was helpful.

author: alanambrose
author_rank: Frequent Contributor
timestamp: February 18, 2016, 04:46:30 pm
content: I'm also interested in n-cornered hat re volt refs - I think this might be key to characterising drift.A.

author: Cerebus
author_rank: Super Contributor
timestamp: February 19, 2016, 05:24:33 pm
content: I think I've got my answer, at least for practical purposes, in the three-cornered hat method. I'll probably come back and revisit this here in a while (think months not weeks - I need to build the stuff I want to measure first) with some working code that I'll try and keep as general as possible.Thanks for everyone's comments.Ian

author: alanambrose
author_rank: Frequent Contributor
timestamp: March 01, 2016, 08:29:02 pm
content: Ah, I just noticed, there's some good discussion here which talks about comparing references:https://www.eevblog.com/forum/testgear/fluke-732b-repair/25/A.