Keithley 2450 SourceMeter Repair - Fatal System Error PC:000990A8 - Page 1
Forum section: electronics / repair

author: NyquistNoise
author_rank: Contributor
timestamp: June 14, 2025, 07:49:43 pm
content: Hi all,I've been attempting to repair a "new" Keithley 2450 SMU to no avail.Largely this SMU works, all Source & Measure functions run as intended on all ranges, GPIB, USB, Ethernet is all functional etcHowever, after some amount of time (could be minutes or hours...), the SMU locks up completely with the following error:FATAL SYSTEM ERRORThe instrument has encountered a fatal, unrecoverable error.Power cycle this instrument.If the issue persists, contact us at tek.com/keithley.PC: 000990A8 Main Rev: 1.7.12bVector: 00000003 Slot:0, Slot Rev:1.7.12bI've had a look at all supplies that I can with a datalogger up until the error, none seem to drift or act odd, all clocks appear present on the two Freescale processors, as well as the FPGA even after it has crashed. The USB and Ethernet interface bomb out on error and the screen/buttons are all unresponsive. A firmware update did not seem to fix this, and unfortunately there appears to be 0 documentation on error codes, so I'm in the dark...Does anyone know what this error might mean? This feels very much like a comms kind of error. There are some isolators present on the analog board, which appear to carry back and forward data over the PCI-E type connector, and their content changes when in error state, but not sure if this is the chicken or the egg.Edit: Images were 0bytes.

author: coppercone2
author_rank: Super Contributor
timestamp: June 14, 2025, 08:09:00 pm
content: Not keithley but I had at least 2 that i remember bad USB cable incidents that would crash a keysight equipment along with a corresponding error in the program that keysight has. I assume if you were looking for that thing you would use a error data tester. Now its not know if its just the cable, or the cable along with degraded connectors (on PC and TE), or even a bad cable shield or ground loop type problem occurring... however its been gone for 2 years since the cable was replaced. And this is under warentee cal equipment not some ebay purchaseits quite a expensive piece of equipment and it also could be fairly repeatedly crashed by certain bluetooth devices connected to it as a DUT , enough to lock out the front panel and give garbage readings requiring a equipment reboot. The EMC performance of these equipment is greatly exadurated. Crashes with the PC seem common enough (random).Anyway I suggest two thingsa) inspect the comm connectors with a magnifier and light to see if its OK, it may need cleaning (always a good starting point)1) replace comm cable + different USB ports2) try it on different computers. These modern TE software interfaces are always having problems with windows to the point of a dedicated liaison between KS and company, its possible its not totally HW related but actually having to do with patches, order in which patches are applied, stuff like that (see what happens if you have 10+ of the same expenisve thing being used regularly by a lab by different ppl). also corporate security systems interfere with it (computer protection SW). Different computers also have different USB ports that might fare better.3) something with ground, check to make sure its in the same outlet or w/e. My mains was tested and it was fine so I never went this far (and I believe its supposed to be isolated somewhat), in that lab it was not wired by the bosses nephew supposed apprentice electrician and they paid professionals well to wire it up

author: Miek
author_rank: Regular Contributor
timestamp: June 14, 2025, 09:06:42 pm
content: That style of error (printing the program counter and a vector number) makes me think it has hit some hard fault in the processor like trying to execute an invalid instruction, accessing an invalid memory address, etc. - so, I'd suspect corrupt program memory or bad ram.

author: NyquistNoise
author_rank: Contributor
timestamp: June 15, 2025, 10:36:51 am
content: I'm not suspecting comms from the USB front personally, as this still bombs out with just Mains in. Good point about firmware though, I might try downgrading to an early version and "stepping" through each one to see if that sorts it.

author: NyquistNoise
author_rank: Contributor
timestamp: June 15, 2025, 10:39:44 am
content: @MiekI think you're on to something there, it didn't occur to me that PC is of course program counter 🤦‍♂️ This does occasionally happen at different PC counter values so RAM sounds quite likely. There appear to be two copies of the MPU and RAM on the digital board, so maybe this is Source+Measure separately.No easy way to tell which one is at fault, RAM swap will be a PITA as its BGA.I'll post a better teardown soon

author: coppercone2
author_rank: Super Contributor
timestamp: June 15, 2025, 07:41:08 pm
content: Oh I thought it was crashing connected to a PC with the program turned onIDK if I would be screwing around with firmware if you suspect bad digital hardware or ram it might totally brick the system. That is like exactly how it gets bricked IMO when people do something with bad ramI had bad ram on the PC a few times, I would not be surprised if modern test EQ that is almost PC has similar problems. Bad ram = failed OS install. When PC's act up I usually go after the RAM because its pretty cheap now and has a bad reputation. For PC trouble shooting having spare ram is like having a spare power supply now, its basically essential IMO. A program like memtest might catch it but its just almost not worth the trouble when you have to run the program for 3 days to try to diagnose $200 of ram. Easier to whack new ram in there first try and just see if it malfunctions with normal use lol

author: coppercone2
author_rank: Super Contributor
timestamp: June 15, 2025, 07:56:54 pm
content: And did you consider heat and solder joints or cracked caps or vias, the keithleys take a while to warm up, in fact their accuracy seems to be shit unless you let it warm up on the low ranges. They warm up alot. My recent keithley 2001 repair was a matter of carpet bombing vias (and then relacing a switch)I noticed at workkeithley SMU = connect to weird test fixtureweird test fixture = odd shape, impractical wiring and connection pointsposition of meter for test = some where it is likely to take mechanical damage. infact its often placed backwards and on top of something so you can access the rear panel specials. IMO they should move some of those things or duplicate to the front panel

author: JoeyDeng
author_rank: Regular Contributor
timestamp: July 17, 2025, 03:54:50 pm
content: I got a similar "Fatal System Error PC" error on my Keithley multimeter. then I upgrade the system and everything get worse, it sometimes works fine sometimes reading overload during voltage measurement. I end up giving it to my friend. he tells me its the problem of the acquisition trigger and fixed it.

author: voltsandjolts
author_rank: Unknown
timestamp: July 18, 2025, 12:26:02 pm
content: Quote from: JoeyDeng on July 17, 2025, 03:54:50 pmAny details on that fix, to help others?

author: NyquistNoise
author_rank: Contributor
timestamp: July 19, 2025, 07:38:30 am
content: An update on the SMU:Everything about this fault seemed to point towards the main digital board. I have narrowed it down to this one specifically, there is a 90 degree PCIe type connector that joins the analog circuitry to the digital board. This was removed, so readings read "NaN" and the fault still occurred.Firmware updates also did not fix this issue at all, so as a stab in the dark, I've swapped the RAM using a BGA rework machine for both MCUs.This did not fix the issue, if anything, it fails in less time now. The RAM was a china "eBay special" so it is possible they are factory rejects and are already faulty (bad sectors etc)?All other functions still work, until this error occursI also contacted Keithley (Tektronix), who have refused to provide any further details on the fault, and would rather I send it to Germany to be fixed.I'd be inclined to swap the MCUs, however, I believe there is some level of factory programming in these/bootloader so it's not easy...Anyone got any ideas? I've done a partial teardown with pics, so will upload them

author: NyquistNoise
author_rank: Contributor
timestamp: July 19, 2025, 08:30:03 am
content: Analog board

author: coppercone2
author_rank: Super Contributor
timestamp: July 19, 2025, 09:51:14 am
content: maybe it has a bad via or SJ that you disturbed during soldering. My k2001 was malfunctioning, so when I worked on it and did a repair, it got alot worse, then I soldered everything within 3 inches of the 'repair' that made it work less and it was fixed

author: voltsandjolts
author_rank: Unknown
timestamp: July 19, 2025, 10:01:53 am
content: Quote from: NyquistNoise on July 19, 2025, 07:38:30 amRepair at Keithley would likely be a board replacement, either analog or digital (or both!) ...so $1000sHave you tried pointing a thermal camera at the top board?

author: smaultre
author_rank: Regular Contributor
timestamp: July 22, 2025, 09:52:41 pm
content: It's to hard to troubleshoot these, maybe power supply, fault isolators, maybe CPUs, maybe flash. Check the total operational time so you can partially predict components degradation.

author: voltsandjolts
author_rank: Unknown
timestamp: August 02, 2025, 01:17:33 pm
content: OP has narrowed the fault down, because it still occurs when the analog board is disconnected - so it's not the problem.This leaves the digital board, and the front panel board shown here (assuming same as 7510 dmm);youtu.be/PLhyBDveVxI?feature=shared&t=85It might be worth getting a quote for a new front panel board.Up for a few hundred bucks gamble? Hmm, if they sell it, might say send in for repair only, like KS BS.Also, the program counter value is quite low at 0x000990A8 which is in code space.Vector 3 is the hard fault vector, assuming an arm processor.So this could well be an mcu hardfault on internal flash memory read error. Might have ecc.Pity it doesn't tell you which processor, front panel or digital board!