Making do with what you got/ Keysight 34465 & Genuine 34470 Voltage Reference - Page 1
Forum section: electronics / general

author: View[+]Finder
author_rank: Unknown
timestamp: January 25, 2021, 07:45:39 pm
content: Hello All,ViewFinder here.This post is a 'spin-off' of the "Buyers Beware . . ." series which addresses the quality of the China-origin version of the voltage reference board that is used in the Keysight 34470 DMM. Because the 34465 and 34470 have the same main board, someone thought "Why not?" and . . . . Now I want to find out if it works.My reasons for buying a reference from Keysight rather than the China knock-off stemmed primarily from having a reliable baseline comparison installed in a 34465 as I knew there would be interest on this forum. And also curiosity, sloth (didn't want to DYI), lust (Keysight delivered in less than a week), maybe getting better performance from the meter. So my focus was on limiting the effect of temperature on the voltage measurements while determine whether the replacement of the reference was beneficial.Here's the link to all the details of the China clone, what folks have done to bring it up to the level of the Keysight and some photos of the construction needed to mount the board properly and manage thermal influences:https://www.eevblog.com/forum/metrology/buyers-beware-misleading-34470a-reference-modules-(clones)-on-ebayI set out to document a base case for comparison--some of which will be found in the link above--more has been done and will be reported here.

author: View[+]Finder
author_rank: Unknown
timestamp: January 25, 2021, 08:52:27 pm
content: Testing the Keysight 34465 ('65) with the 34470 ('70) reference.Any serious testing of precision measurement devices must always include "under what conditions?" in the reporting. I have included that in the post referenced above and will in the near future provide a better description of the "Making Do with What You Got" working environment (MyLab and also MyShop). As you might imagine, I strive to do the best I can, etc.In this case, the testing was done in a room (MyLab) dedicated to electronics generally between 21C and 29C depending somewhat on the season--in California, 38 degrees North Lat--right now it is 22C in the Lab and will probably get up to 25C later. The testing shown on the attachment was done at night on the 20th, 21st, 24th of January. The protocol was to run autocal on the meters first, then collect 1000 observations timed 30 seconds apart by a Python script. Meters were accessed over GPIB (3458) and TCPIP (34465) and internal temperature and voltage were captured without delay. The source of the voltage (the DUT) was a PDVS2 (Ian Johnston #NotSponsored) set at 300mV and was connected directly to the 3458 meter and and to the 34465. Both meters were set at NPLC=10. The data from the three nights were appended in a dataset for plotting,The blue plot is based on data from the 3458 and the orange plot from the 34465. From the data, the 34465 seems to run about 6C over ambient versus about 8C over ambient for the 3458. The 3458 is located above the 34465 (10cm airspace between). The fan and filter are in place on the 3458 and direct air inward; the fan on the 34465--contrary to reports elsewhere on the forum--exhausts air from the enclosure, make-up air coming from numerous vents its the sides.Comments and suggestion are encouraged . . .

author: Kleinstein
author_rank: Super Contributor
timestamp: January 25, 2021, 10:09:04 pm
content: Using the merters internal temperature is a bit confusing - the temperature that really matters is likely the temperature of the external 300 mV reference. Both mters can have a different delay in the internal temperature.So the more logical way to plot the date would be to have the time for the x axis. Chances are there would be quite come correlation, as the likely moving part ist the 300 mV source.Anyway 300 mV is an odd test point - it is not large enough to see much of the reference effect, more like mainly ADC noise and a little effect from the references.The logical tests would be at 0 V for the ADC and amplifier and at some 7 V (~ raw reference voltage) or 10 V (near maximum of the range). With a rather noisy expernal reference one may have to look at the difference or ratio of the 2 meters.

author: View[+]Finder
author_rank: Unknown
timestamp: January 25, 2021, 10:53:40 pm
content: Quote from: Kleinstein on January 25, 2021, 10:09:04 pmGood point on the temperature of the reference, I can put a thermistor on it easily and the 6500 has scanning capabilities. I was more interested in getting synchronous measurements and the internal temperatures were available with no extra effort. (I'm lazy sometimes)I have plots of all data over the time of the collection (generally from 22:00 to 06:00) I did not capture a time stamp in the python program, but looking at the data sequentially will be easy, I can post that right away.The choice of 300mV was the result of having run some other unrelated testing and the PDVS2 was well established and it was bedtime for me. (lazy again)By "ratio" do you mean M1/M2 or something else?

author: View[+]Finder
author_rank: Unknown
timestamp: January 25, 2021, 11:21:28 pm
content: New plots: Temperature and Voltage over eight hours

author: Kleinstein
author_rank: Super Contributor
timestamp: January 25, 2021, 11:52:37 pm
content: As expected most of the scattering in the voltage is correlated. So it's the variations in the external reference. So essentially no information about the reference in the "34465".I have not checked how bad the PDVS2 did perform, but it's not good enough to check the new reference in the DMM. For a test I would more suggest a low noise refernence - so liek LTZ1000 based or better.A possible test source could be a carefully shielded and carefully handled 9 V battery in an isolated box - if at constant temperature and not moved much (e.g. no earthquake) it can give a short time low noise source. The battery type / history can make a difference.

author: View[+]Finder
author_rank: Unknown
timestamp: January 26, 2021, 12:07:41 am
content: Brilliant! Here I am fussing around with battery power for a reference and not considering a batteryasthe reference. Considering discharge curves, what about a 12.4V LiPo It should show less voltage drop. I'll get started on this right now!Thanks!!UPDATE: test in progress . . .Done. Short run--just 20 minutes--just to check basic function. 12.4V on LiPo battery insulated in metal box, ambient 23.8C

author: Kleinstein
author_rank: Super Contributor
timestamp: January 26, 2021, 11:46:34 am
content: Are the 12.4 V still inside the 10 V prime ranges. This is at least very much at the upper boarder. It makes no sense to use a different range.The noise looks rather bad for both meters. With so much noise it is no good. At least the 3458 should be better than this, even at 1 PLC.

author: View[+]Finder
author_rank: Unknown
timestamp: January 26, 2021, 07:43:33 pm
content: New data:Going down to be in the 10VDC range is no problem at all and will be a good comparison with the plot below. I can get on that later. Probably around 8.4VDC will work. Same protocol.UPDATE: New plots with 8.34VDC LiPo battery DUT (reference)

author: View[+]Finder
author_rank: Unknown
timestamp: February 04, 2021, 07:37:19 pm
content: Attached are plots showing "Serial Correlation of Voltage Readings from an LTZ1000 Reference (HP3458 A9)" as measured by an HP3458 and for comparison, a KS34465 with a genuine Keysight 34470 reference. The testing comprised about 14 hours of data collection.More info here:https://www.eevblog.com/forum/metrology/making-do-with-what-you-got-an-hp-3458-opt002-reference-with-no-heater/msg3448248/#msg3448248

author: View[+]Finder
author_rank: Unknown
timestamp: February 13, 2021, 01:27:12 am
content: HELP, Is there something wrong with my (modified) KS34465?Using an HP3458A A9 board as the DUT in the post above, the KS34465 (with the reference from the KS34470) measured voltage that looks 'as expected' on the plots.Contrast that with the two plots attached. Both were captured from the same 10 V reference (not the A9). The first one was by an HP3458 and the second by the KS34465.Would someone please help me figure out why the plot points are lined up like a military formation? My guess is that it is related to the ADC in the meter, because I'm seeing the same 'feature' with another reference. I certainly hope there is an explanation.Thanks . . .

author: TiN
author_rank: Super Contributor
timestamp: February 13, 2021, 02:05:59 am
content: They are lined up because of resolution cap of 34465. But your ppm plots make no sense to me, why it's always a straight line?Attach your RAW data points (CSV or DSV or whatever log file where values from DMM are recorded) so I can take a look?

author: Kleinstein
author_rank: Super Contributor
timestamp: February 13, 2021, 10:07:44 am
content: The plots of only the short time difference looks at the more higher frequency noise of the references and the DMMs. This is usually the least problem and of very limited interest. It may have some sense with the broken reference without the heater, to suppress the long time drift.For the references it is usually the slower effects and long time drift that really matter - so just that what the plots are hiding.

author: View[+]Finder
author_rank: Unknown
timestamp: February 13, 2021, 03:39:04 pm
content: Quote from: Kleinstein on February 13, 2021, 10:07:44 amThe 'reference DUT' Keysight A9 board is working with a replacement LTZ1000 and OpAmp (precision socketed). Heater is working as it should based on the data from the PSU.

author: View[+]Finder
author_rank: Unknown
timestamp: February 13, 2021, 04:44:41 pm
content: Quote from: TiN on February 13, 2021, 02:05:59 amRaw data files attached. The primary purpose of the testing was to evaluate the 34465 with the Keysight 34470 reference replacement. Testing was done over 'minutes and hours' rather than 'days and weeks' because if the reference replacement didn't work in the short term, long term wouldn't matter.I've included the PVDS reference data for comparison.The statistic of interest for cross-comparison was the standard deviation of the "((Reading(+1)/Reading)-1)*10^6" expressed as PPM where 'Reading' is the voltage as measured by the 3458, etc. The 'by 3458' data was NPLC=100 at 5 second intervals. The 34465 was NPLC=15 (max).

author: Kleinstein
author_rank: Super Contributor
timestamp: February 13, 2021, 05:00:52 pm
content: It is kind of Ok to check the short time scale too, but this is more like the simple part and usually no real need to plot those data. This are more like dataset were the main information is in the RMS value. The RMS value of the "((Reading(+1)/Reading)-1)*10^6 " function would be close to the initial point of the Allan deviation curve. The more intrresting part of this would be however more like the values at 1 minute to a few hours and ideally even longer.Chances are most of us are better in looking at such data in the time domain directly.

author: View[+]Finder
author_rank: Unknown
timestamp: February 13, 2021, 05:56:40 pm
content: Changing the time between readings from +1 to +10 or +100 is a simple task and I have done it for several tests. Another approach is to use the formula for adjusting standard deviation (variance) for differences in the periodicity of the data. Note that the analysis in progress is focused on the variance of the voltage, not its value. The stability of references over time is dependent upon environmental factors many of which are on my list of things to fix.

author: Kleinstein
author_rank: Super Contributor
timestamp: February 13, 2021, 06:25:18 pm
content: The Allan deviation curve is essentially what you get when you look at the relative noise with different delays. This would be a good way to look at the results, though the direct plot over time also tells a lot.

author: TiN
author_rank: Super Contributor
timestamp: February 13, 2021, 06:44:45 pm
content: Why there are multiple columns in datasets? One hint for future - always add actual timestamps and environment conditions (BME280 I2C sensor and arduino or Raspberry Pi are cheap), this will help to reveal correlations to changes.

author: View[+]Finder
author_rank: Unknown
timestamp: February 13, 2021, 07:14:20 pm
content: Exactly so.Before there was 'Allan deviation' there was 'first difference of logarithms' which is what was used before calculators could divide. Subtracting logX from logY is the same as X/Y and far easier than 'long division' (like anyone still does that).My work experience with calculating investment performance--rate of return over time--was originally based on 'first differences' and some of the old-timers still used that term. The formula:((Reading(+1)/Reading)-1)*10^2 shown as percent was the common expression, 'reading' in that case being the value of the asset. Over the course of my work the measurement interval changed from 'annual' to 'quarterly' to 'monthly.' In recent times the frequency of data capture is driven by the routing of fiber cable.((Reading(+1)/Reading)-1)*10^6 as PPM is the basis for my analysis of voltmeters' performance. If the imbedded interval is 5sec (with a fudge for the NPLC=100), then a 'lag+10' is about a minute and 'lag+100' is about an hour and a half. A run of 10,000 readings should yield over sixteen hourly measurements.Beyond that, we're in the realm of temperature variation and other environmental factors--all of which need to be addressed once the 34465 (modified) is declared fit for duty.

author: TiN
author_rank: Super Contributor
timestamp: February 13, 2021, 07:46:27 pm
content: Sorry, it does not make any sense to me, don't have any financial education here.I've cooked quick ppm plot using your RAW data with PDVS 10V outputs.Excel-file with results.In graphical chart it reveals this:Which is in line to what LM399 reference would do in non-stable conditions with direct DMM measurement. No dead on straight linesOrange plot is difference between 34465A and 3458A (longer column data with 10001 samples) in ppms.

author: View[+]Finder
author_rank: Unknown
timestamp: February 13, 2021, 08:04:57 pm
content: Quote from: TiN on February 13, 2021, 06:44:45 pmThere are multiple columns because there are multiple runs, mostly on different days, and there are three DUT's and two meters.Sorry, but this forum does not accept .CSV files and I had to ZIP them. I did not look at the ZIP file I sent, so if they are mixed up tell me what you want me to do and where to post.There are timestamps for my Keithley meters as a matter of course when extracting from their web interface, Keysight used to provide this feature but no longer does, HP3458 requires programming to add timestamp to Python script. I understand then desire for specificity as to environment, however the timestamp can be approximated from the starting time and the measurement interval, and also as explained in my prior posts.Yes! Temperature. The Keithley 6510 is a DAQ and I have cards for it to collect temperature from multiple areas in my lab. I also have a NIST traceable thermometer and hydrometer on my desk that I check during taking measurements. Unless it is a really hot or cold day, my lab has had a regular cycle and there is data on it.My immediate concern is whether my enhancement to the 34465 is working more like a 34470. Then will come the questions about stability over time. From the data at hand, there are no "correlations due to changes" and that is fine for now. If there had been autocorrelation, it would have shown up in the residuals from the regressions (the little charts on top of the charts in prior posts), likewise the shape of the histograms of readings (available) look "normal" (or close to it). The "straight line" regression and even distribution of plot points over the observations is good news. It is likely that longer intervals will show the effect of other factors and that will be the subject of further analysis.

author: View[+]Finder
author_rank: Unknown
timestamp: February 13, 2021, 08:30:28 pm
content: Hello Illya,Your plots are correct in what they measure, as are mine. You are measuring deviation from a norm (the median) and I am measuring deviation from a prior value. Your plot show the effect of serial correlation and that is to be expected when plotting thedifference from a value and another value. If you are trying to examine serial relationships of values, that is the right way to do it.In the investment performance field, the comparison is often against a norm, however the statistic is not the 'value' of the investment but the 'rate of change' in the value over some period of time. This is because there are problems when the residuals of a regression are correlated, as they would be in the 'value' data.I don't know if this is a problem as well in the analysis of voltage measurements, but I will look into it today.

author: Kleinstein
author_rank: Super Contributor
timestamp: February 13, 2021, 10:03:04 pm
content: We are back at the problem using the LM399 based reference to judge the quality of tha LTZ1000 based ref. this just does not work that easy. All one can say from the curves is that the modified ref in the 34465 is not much worst than a LM399, maybe a little better. Using different PLC setting and thus difference contributions of ADC noise add to the problem. I would at least use something like the average of 10 conversions at 10 PLC for the 34465 - so the same what the 3458 does internally for 100 PLC. It may be even better to use 20 x 5 PLC or 50x 2 PLC, though the difference is likely small.One could still use the PDVS reference, if the 2 meter are measuring in parallel. In this case it would make sense to look at the difference or ratio to get an indirect comparison of the 34465 meter with the reference of the 3458. This may give a first crude idea if the upgraded 34465 is at least reasonable. It is still only a weak test, no really stringent. For a better test it would need a LTZ1000 or betterreference. A parallel reading from the 3458 may be used to check the external ref somewhat, as the correlations can give an idea which of the 3 involved refs is likely to blame. The works often OK for jump like events. With the 3458 running in parallel one may even get away if the external reference has some weakness, like higher drift or some popcorn like jumps. So a JRV type ref from a JFET and resistors may partially work.

author: View[+]Finder
author_rank: Unknown
timestamp: February 14, 2021, 12:34:56 am
content: We've kinda gotten off the track here. I've attached plots from my prior post for clarity. The green plot is from an HP A9 board DUT measured by an HP 3458 DMM; the orange plot is the same DUT measured by my 34465 DMM modified with the 34470 Keysight reference. The plots are 'reading versus reading' like Allan deviation, not 'reading versus median' as TIN has added.There is no disagreement that measurements taken over an extended period of time is the cornerstone of metrology. What is important to me right now is whether my 34465 is better than it was with the LM399 that was replaced. Eliminating the environmental effects by keeping the time between readings short (5 to 10 seconds), puts the focus on how the 34465 is behaving in comparison to its grandfather, the 3458 when measuring the best DUT available to me, that being an HP-A9 board.Based on every recent test of three levels of meters (6.5, 7.5? and 8.5) measuring DUT's based on LT1021, LM399, REF102CP and LTZ1000 references, the 34465 comes in about where one might expect for a 7.5 digit meter. The 3458 is better and the Keithley 6500's are not as good on the specific test described above.Just in case anyone is interested, my general impression of the DUT's is as above from left to right

author: TiN
author_rank: Super Contributor
timestamp: February 14, 2021, 12:40:19 am
content: Comparing one reading to next reading is showing DNL and does not reflect anything else much.To make sure than reference modification is good (or better) than original factory condition one must test and verify much more than just DNL figure or short-term noise.Perhaps testing INL and 1 month stability would be more unveiling than just few hours or a day of noise logs. You could use PDVS as source and test INL of modded 34465, using 3458A as reference (if your 3458A is good and stable).

author: Kleinstein
author_rank: Super Contributor
timestamp: February 14, 2021, 09:28:51 am
content: I would not expect very much effect of the changed reference on the INL. It would still be good to check if one has the means. INL at the ppm level can be tricky and there is allways a chance to find a way for am error to come up in an unexpected way. For the INL one has to expect the unpected.The short time noise is only a minor issue an it look good. Here it would be even more surpricing to see a large negative effect. At least the multislope 4 implementation in the 34410 as a more direct predecessor has at least some higher frequency filtering for the reference. So there should be little effect of the reference noise on the short time noise and usually the LTZ1000 ref is also lower noise on the shorter time scale.The real issues to test with the new reference is longer time scale noise. For checking long time drift it could help to allread start with a few data points under well defined conditions. Here the A9 board with the newly changed chip is probably not yet a very stable ref. point. It would be more like the 3458 meter with a more aged internal ref.. It would still make sense to use the new A9 board as external reference so one can also record the drift of that board relativ the the 3458.There are some plots off Allan deviation available for 34465 meters with the original 399 reference. The Allan deviation plot also includes the short time part, so one could to a direct comparision. The DMM noise comparison thread has quite a few such data. For comparable data I would suggest to use 10 PLC contineous readings.https://www.eevblog.com/forum/metrology/dmm-adc-noise-comparison-testing-project/msg1316500/#msg1316500

author: View[+]Finder
author_rank: Unknown
timestamp: February 14, 2021, 08:32:04 pm
content: Quote from: TiN on February 14, 2021, 12:40:19 amhttps://www.maximintegrated.com/en/design/technical-documents/tutorials/2/283.htmlGotta love the internet: all about DNL and INL. Yeah, differential nonlinearity was exactly what i was trying to measure (except I called it "volatility risk" as old habits are hard to break). Testing both for DNL on a periodic basis and a longitudinal test of INL over at least a month are excellent suggestions. Long-term comparisons of risk and return were the bread and butter of my old day job.As much as I respect Ian Johnston's work, the TI REF102CP based device looks better as a DUT. But why not use the Keysight A9 board?

author: TiN
author_rank: Super Contributor
timestamp: February 14, 2021, 11:26:20 pm
content: REF102CP and A9 board are useless to measure INL because they provide only one fixed point.You need variable source to sweep over the DMMs range.

author: Kleinstein
author_rank: Super Contributor
timestamp: February 15, 2021, 09:04:30 am
content: The linearity testing article from Maxim is interesting, but more like suitable for lower resolution and relatively fast ADCs. With a DMM linearity testing would likely need a different technique. Anyway the INL/DNL should be the least important part to test.A point to check is the ACAL behavior - if there is significant settling after ACAL, like with some 34470, this could effect other measurements. If this depends on the cal data (as the "fix" from Keysight suggests) the effect after the reference change could be even larger.The main parts are lower frequency noise (e.g. via Allan variance test, both with a short and a low noise 7 or 10 V source) and a little the long term drift. I would not expect much positive effect of the reference on the noise with a short, but this test is relatively easy and one never knows - thinks can always get worse. With no direct data from the before state, the comparison is likely limited.The long term drift only needs kind of some 30 minutes data sets in well warmed up state every few days or weeks. So it can be done in parallel to other tests and it can help to start early. Here your best ref. is likely the 3458 meter - the actual voltage source(s) would be more like tested in parallel. Even the renewed A9 board would likely show some initial drift and the initial drift rate may be informative independent from the rest.

author: View[+]Finder
author_rank: Unknown
timestamp: February 16, 2021, 12:04:50 am
content: Quote from: TiN on February 14, 2021, 11:26:20 pmI have an HP3245 in very good condition (even better if i could find an HV board for it) and it was born to sweep. It was calibrated to my HP3458 in both DC and AC just over a year ago. As you know it only covers +/-10VDC (without the HV board). There is also a Keithley 230 that I think has a sweep function. What are the parameters and what would be the testing protocol? I assume that the 3458 and the 34465 would measure and use a Python script to control the process over GPIB. Probably better to put the sweep in the Python, I have done this already for testing power sensors.