How reliable are accredited calibration laboratories - Page 1
Forum section: electronics / general

author: CrazyTiger
author_rank: Regular Contributor
timestamp: June 21, 2021, 01:23:59 am
content: Hi,I am sharing my recent experience sending two of my repaired signal generator for calibration. This lab is a well know international company. They were ANAB accredited for ISO17025 and I checked with ANAB, they were indeed accredited for the scope of my calibration. I was charged a premium price for the calibration. Upon completion of the calibration I was furnished with 18 pages of calibration report, and this where the horror began.1. The standard they used for calibration: the power sensor starts 10 MHz, but they measured power down to 300 kHz with that sensor and it passed the test, must be a miracle sensor.2. I have two pages of testing of LF Output of the signal generator, they did frequency response test, accuracy of output voltage, frequency and so on, here is the kicker both my unit don't have LF output.3. Another pages was for pulse modulation, my unit does not have pulse modulation option either.4. SSB phase noise test, they used a spectrum analyzer with phase noise worse then my sig. gen, and yet the test pass superbly, though the limit they stated is wrong.At first i was thinking, maybe they used a wrong template, till i got the other certificate, same horror show and the result stated is different indicating they must be using a random number generatorIn this case my question is how is ANAB certifying all this labs, are they also just giving out accreditation.How many people have encountered this kind of laboratories?Basically the are just churning out fake certificates

author: DuPe
author_rank: Regular Contributor
timestamp: June 21, 2021, 09:25:45 am
content: What is their (cal lab) Statement to your findings?

author: Henrik_V
author_rank: Regular Contributor
timestamp: June 21, 2021, 11:32:50 am
content: First would be to contact the lab.Next escalation steps IAAC and finally ILAC...In the US you have 4 organisations for accreditation (according to IAAC), nice for the labs, but for metrological quality ??So the best a customer can do is to ask for a quote for your type of instrument including the typical/expected* uncertainty, references and methods.*) sometimes they could not be reached, but the lab should answer the question why.Since I am a expert assessor myself, I can only answer your question for Germany and in the area of acceleration....Would like to see follow-ups

author: HighVoltage
author_rank: Super Contributor
timestamp: June 21, 2021, 12:02:46 pm
content: If you have found already so many mistakes, I would not be surprised, if you will find more by looking in to their reference equipment. This seems to be very sloppy.Some companies just need a calibration certificate and they do not care about the numbers shown and they are happy that they have a valid certificate to fulfill their obligation to ISO requirements.I just had a similar experience with an expensive vacuum gauge that was reading far off.The gauge showed -960 mbar when in reality it should be reading -999 mbarIt was sent in for calibration with adjustment.Adjustment was not done!And a perfectly looking calibration certificate came back with the gauge, showing all values within its specifications.And this was a well known calibration lab in Germany.The invoice stated adjustment and total price was several hundred Euros.

author: Psi
author_rank: Super Contributor
timestamp: June 21, 2021, 12:36:45 pm
content: Quote from: HighVoltage on June 21, 2021, 12:02:46 pmSad but true.

author: binary01
author_rank: Contributor
timestamp: June 21, 2021, 01:10:09 pm
content: Yes, I agree with others; like many commercial industries, there are good and bad operators and accreditation doesn't seem to always protect you, unfortunately.I think it is a problem in the industry, with some labs trading on neivity of there customers with respect to metrology, issuing poor reports and a new calibration sticker.I have certainly had problems with larger well-known accredited labs, and the response when raising it with them doesn't always fill me with confidence. Sometimes there are legitimate mistakes, sometimes systematic errors or lack of care.However, there are definitely good and thorough accredited labs around, if you do some digging (I know because I work in one 🙂, could be a biased opinion)

author: mendip_discovery
author_rank: Super Contributor
timestamp: June 22, 2021, 04:43:16 pm
content: As someone who runs a 17025 accredited lab, I can say that is not ok and you should complain. The accreditation body should be informed if the lab doesn't rectify it to your satisfaction. I know some ISO9001 labs do some stuff that isn't quite right, I worked at one. But in the UK with UKAS we have to be very well behaved as they are on it if stuff isn't right.When being audited recently my auditor spotted an issue on an external cert, I spoke to the lab they said they can make a statement, was told by my auditor if they do that their auditor will take a look at the worksheet...he is their auditor.

author: bdunham7
author_rank: Super Contributor
timestamp: June 22, 2021, 11:20:57 pm
content: Quote from: CrazyTiger on June 21, 2021, 01:23:59 amI've heard of labs that may pencil whip a calibration, usually where it is a no-data or OOT-only data certificate and report. I'd presume that they at least check that the meter works or comes from a client that wouldn't send a broken meter in for calibration. I think this probably happens where the customer just needs the paperwork and the cal tech or lab knows it. The certificates are often very brief and have no real information on it. I'd call that a 'good enough for who it's for' job.Then there's the lab that is underequipped and tries to get by with what they have. We'll call that the 'doing the best we can' job. This can range from questionable to downright useless. In some cases the data may be fake-ish or overly optimistic, as frequently seen in eBay sales.However I have not yet run across a supposedly reputable lab with the balls to do what happened to your stuff, for which the only explanation does seem to be that they fabricated or recycled the entire thing. I'd have a conversation with them and see how it goes--their attitude will tell you a lot. Keep in mind that it is sometimes permissible to characterize test standards and instruments beyond their primary advertised specifications, so your first and fourth issues could theoretically have a valid explanation.There have been a few cal-lab discussions in the forums in the past, I think one of them was someone with a similar issue in your country, but I can't find it right now. It's not a rare issue at all. I'd stick with an OEM A-brand calibration lab if you can.

author: Tony_G
author_rank: Super Contributor
timestamp: June 23, 2021, 12:13:53 am
content: Quote from: bdunham7 on June 22, 2021, 11:20:57 pmI have to completely agree with this statement - Once you go outside the OEM cal labs then it really is about finding a 3rd party one that does good work - There isn't anything inherently wrong with 3rd party cal labs but the quality can be so variant that you might need to try a few to find one or speak to other people in your industry to see who they use and their thoughts - If someone called me up and asked about it I'd be happy to share even if they were a competitor as everyone benefits from good cal labs.TonyG

author: TimNJ
author_rank: Super Contributor
timestamp: June 23, 2021, 03:17:38 am
content: Quote from: bdunham7 on June 22, 2021, 11:20:57 pmOur cal lab sent a guy to our office to calibrate our stuff over the course of three days. He left his stuff in the open after he left for the day. Mainly out of curiosity, I just wanted to see what kind of cool metrology stuff they had. Didn't touch anything, but I found no evidence of an inductance standard...and we have something like 3 or 4 high accuracy LCR meters. No decade inductor either. But the LCR meters got a sticker!!Now...maybe there was some more crates of equipment in the truck, but I don't think so. And maybe newer meters don't need an inductance standard ? I don't know...but most of the older meters put them in the calibration procedure.

author: bdunham7
author_rank: Super Contributor
timestamp: June 23, 2021, 04:08:04 am
content: Quote from: TimNJ on June 23, 2021, 03:17:38 amI had an LCR meter here for service, being resold by a surplus dealer. After I patched it back together, I considered having it calibrated. The procedure requires an OEM calibration kit with 4 characterized resistors, an open and a short fixture and that's it. So no inductors, per se and the machine basically calibrates itself at your command.In this case, the kit costs $4Khttps://www.ietlabs.com/pdf/Datasheets/7000-09.pdf] https://www.ietlabs.com/pdf/Datasheets/7000-09.pdf and the job takes a total of 90 minutes on the bench, not including warmup and acclimatization, but only about 10 minutes of tech time. I ended up sending it out 'as-is' (it seemed to work pretty well) because the two labs I contacted couldn't answer basic questions about the process, i.e. "do you have this kit at your location". One eventually, perhaps accidentally, admitted that they would ship the unit to their "Massachusetts location", which happens to be where the OEM is. The way that thing was put together it could barely survive a car ride in my cushy SUV, let along 3000 miles of UPS.

author: TimNJ
author_rank: Super Contributor
timestamp: June 23, 2021, 02:15:03 pm
content: Quote from: bdunham7 on June 23, 2021, 04:08:04 amI figured as much. I know the new meters have simplified calibration, a lot taken care in software/firmware I guess.

author: mendip_discovery
author_rank: Super Contributor
timestamp: June 23, 2021, 05:18:22 pm
content: Quote from: TimNJ on June 23, 2021, 03:17:38 amSome Multi-Functional Calibrators come with stuff inside them for calibration. So don't be fooled by the state of what is on the desk.Remember Calibration is not adjustment. Manufacturers have muddied the waters over the years and created confusion for the consumers over what calibration is and they feel that if its been calibrated its been adjusted. A lab should only adjust if its going out of specification and then they should give you before and after adjustment readings.QuoteThe consumer should be making efforts to choose a lab with a good uncertainty, there is no point in getting your equipment calibrated by a lab with a 1% uncertainty when your kit has a manufacturers spec of 20ppm otherwise any of the readings they have given you could be either in or out.In response to bdunham7 talking about needing 4k of kit, would a good Decade box not cover that, looking at the specs and procedure it looks easy enough to DIY. My portable calibrator even has a simulated resistance mode that may have worked.

author: Kleinstein
author_rank: Super Contributor
timestamp: June 23, 2021, 05:49:50 pm
content: Simulated resistance is usually only good for DC, not so much for calibrating a RLC bridge, that likely needs a limit on the parasitic inductance / capacitance.One may get away with a set of resistors in DIY boxes, if they are properly characterized. Already a decade box may be a problem from parasitics.

author: bdunham7
author_rank: Super Contributor
timestamp: June 23, 2021, 05:56:30 pm
content: Quote from: mendip_discovery on June 23, 2021, 05:18:22 pmNot likely. The resistor are characterized in some way (they refer to Q) and the characterization constants are marked on the units and must be entered into the machine during calibration. A decade box isn't going to cut it. But I'm sure there is a cal lab manager somewhere that would tell their tech to just use what they have....

author: TimNJ
author_rank: Super Contributor
timestamp: June 23, 2021, 06:55:48 pm
content: Out of curiosity, I checked a few "digi-bridge" style LCR meters from the last 20-30 years. Seems they need the precision resistor set only...I guess this is acceptable because the overall topology is a Whetstone bridge?I have an old HP 4332A LCR meter at home. It requires two standard inductors, two standard capacitors, and two standard resistors to calibrate (properly). Per my understanding, it does not use a bridge, but has a in-phase/quadrature detector, sort of like a VNA, but just for two test frequencies. Maybe that explains it.Is bridge the prevailing LCR meter topology these days? What does the IET/ DER DE-5000 use?Not to get too crazy off-topic here...

author: bdunham7
author_rank: Super Contributor
timestamp: June 23, 2021, 08:10:23 pm
content: Quote from: TimNJ on June 23, 2021, 06:55:48 pmNone of the ones I've seen use an actual old-school bridge with an adjustable resistor, but there are a variety of systems that are in use. The QuadTech one I linked sends out a signal, that can be fixed or swept, through a resistor and then monitors the signal across the DUT and across the resistor--apparently this is called the I-V method. I think it is functionally the cousin of a VNA. Usually the handheld ones have fixed frequencies and do not load-adjust the bias voltage. Fancier ones let you add DC bias and can amplify the bias to maintain a constant voltage across the DUT.There are more bridge-like models as well, but AFAIK most LCR meters of either type can have the calibration constants determined with just resistors, just like you calibrate a nanoVNA with an open, a short and a 50R load. With an LCR meter, you really have to calibrate it with the fixture or leads that you are going to use. My BK 886, for example, has you run a CAL OPEN and CAL SHORT (with a fixture) any time you want, which typically would be every time you turn it on or change the leads. I can understand that this is how you set the constants, but this seems to be conditioned on the meter working properly. It baffles me that there isn't a performance test requirement with L, C and R standards.

author: Kleinstein
author_rank: Super Contributor
timestamp: June 24, 2021, 07:31:50 am
content: When using digital signal analysis there is little need to do an extra calibraion for 90 deg. phase shift. So a good resistor with low parasitics is good enough for the calibration to 0 phase shift. A test with an actual resistor or capacitor from time to time would be good. More to check for defects than just drift of the internal reference resistor. Many defects are expected to be detectable in the open measurement. If in doubt, one is free to have a small box with a few reference elements with measured values close to the RLC meter and check them from time to time.

author: Oldtestgear
author_rank: Regular Contributor
timestamp: June 24, 2021, 10:46:57 am
content: I bought a Solartron 7081 off eBay a few years ago that had a current calibration sticker on it. Out of curiousity I asked the qualifying facility what a re-calibration would cost. The Sales Office quoted a low price very quickly & I was doubtful whether they could do the job to my satisfaction. Anything better than I could do (using a calibrated Keysight 34470A + Datron calbrators) would be great. After checking their UKAS accredittation it was obvious that they could not perform any calibration to better than 6.5 digits. When I queried their abiity to do the job, they suddenly stopped chasing me for an order number.I assume that the original customer just needed a piece of paper to satisfy an audit trail, especially as the 7081 had an offest on a couple of ranges. I know the offest could be the reason it ended up on eBay but it is a stable & very usable instrument that I will re-calibrate ( to a limited extent) myself "one day".

author: Villain
author_rank: Regular Contributor
timestamp: June 24, 2021, 02:49:16 pm
content: Out of experience i can say that UKAS takes a lot more caution and care auditing and granting accreditations compared to for example ANAB and A2LA.Although every lab - even accredited can fake as much as they want really (at least until they get busted). I would stick to reputable companies even if price is higher, except you know someone who is working in a smaller cal lab and you can be certain that the quality is good.

author: Teti
author_rank: Regular Contributor
timestamp: June 26, 2021, 07:04:26 pm
content: Quote from: shodan@micron on June 21, 2021, 12:48:59 pmHi Shodan, love your voltnut YT channel!Sorry for OT

author: CrazyTiger
author_rank: Regular Contributor
timestamp: June 27, 2021, 01:55:31 am
content: Update:After sending a long complain email to the calibration provider, i also kept their Principal in US in loop. The company admitted that something went wrong in their process and they have recalled both the unit for re-calibration. They will be barring the cost for both transportation and calibration. Lets see how it goes

author: bdunham7
author_rank: Super Contributor
timestamp: June 27, 2021, 03:46:56 am
content: Quote from: CrazyTiger on June 27, 2021, 01:55:31 amOh dear! Reading between the lines, perhaps an A-brand OEM lab still wasn't good enough? If so, at least they have a reputation to protect.

author: Psi
author_rank: Super Contributor
timestamp: June 27, 2021, 12:21:43 pm
content: Quote from: CrazyTiger on June 27, 2021, 01:55:31 amSay that you expect a detailed engineering level explanation about how this situation occurred and that you will not accept a meaningless statement from their PR department.

author: HighVoltage
author_rank: Super Contributor
timestamp: June 27, 2021, 03:28:00 pm
content: I would ask for a detailed8D-Report.If they know what that is, it will be easy for them to prepare one.If they don't know what it is, they have to figure it out.If you want, you can black out their company details and post this 8D report here for us.This will be very interesting.

author: Psi
author_rank: Super Contributor
timestamp: June 28, 2021, 08:33:13 am
content: Never heard of a corrective action report called an 8d Report, but maybe it's different for medical.

author: mzzj
author_rank: Super Contributor
timestamp: June 30, 2021, 07:03:35 pm
content: Quote from: CrazyTiger on June 21, 2021, 01:23:59 amI work in a 17025 lab and my gut feeling would be that US-based labs are often struggling with 17025 requirements but I have never seen anything as bad or blatantly wrong as yours.Lots of smaller labs do quite a bit of manual "wrestling" with the calibration certificates, especially with products that are new to them or low volume and that often leads to all kind of stupid errors.Big players that do concentrate on their own products (like Keysight) have much more standardized workflow and automatic templates for all of their products.I would be asking totally new calibration AND corrective action report and/or see their complaints procedure..