I2C fault injection - Page 1
Forum section: electronics / general

author: tom66
author_rank: Super Contributor
timestamp: October 17, 2024, 01:59:13 pm
content: I've got a bug on a product where about 1 in 100k to 1 in 1 million requests to an I2C device on my bus fail.This is of course very difficult to reproduce. It seems to be a result on some analog signal variation, perhaps a NAK interpreted as an ACK or a glitch filter violation.I would be interested to know if anyone has any suggestions for a device that can inject I2C errors into an active bus. Injecting a NAK could be difficult but by inserting the device in line with the bus it might be possible to make the microcontroller see something that isn't there, or generate obviously bad responses based on a pattern.Could also try things like enabling clock stretching, stuck SDA, etc. and ensure that my system can recover from all of these fault conditions.

author: AndyC_772
author_rank: Super Contributor
timestamp: October 17, 2024, 02:15:47 pm
content: I think I'd approach this from the other direction. It might be good design to have a system that can recover from bus errors, but unless there's some specific design feature that makes errors inevitable (eg. long cables that pick up noise, or multiple asynchronous masters that might clash), you should expect I2C to be error free.Instead I'd concentrate on trying to generate a trigger condition for your scope in the event of an error. Put some debug into your code which says something along the lines of "if the value read over I2C is out of range, then pulse this GPIO", then set your scope to record the I2C interface and trigger off that pulse.It might then show you something unexpected that's happening on 'bad' accesses - maybe a particularly slow clock edge or an invalid logic level. Or you might discover that the transaction is perfectly OK and the bug is in the CPU after all (eg. an unfortunately timed interrupt). Maybe the slave device you're accessing reports garbage if it's read at exactly the wrong time (eg. when an update is taking place).

author: SiliconWizard
author_rank: Super Contributor
timestamp: October 17, 2024, 02:23:23 pm
content: Yes I'd also focus on being robust to occasional errors, but being able to reproduce problems is also important.I would probably design a tool that would act as an I2C slave device that randomly gives bad replies (unexpected NAKs or other errors). Could be done on a small MCU as long as it supports I2C slave.

author: eutectique
author_rank: Frequent Contributor
timestamp: October 17, 2024, 02:36:14 pm
content: So, what exactly constitutes a bug or a fail? Is it reading wrong data from a device? Or the the device is stuck and no further transfers are possible?The usual recovery from the latter condition is switching host SCL and SDA into GPIO output mode and issuing a series of pulses to simulate the transfer. This will unblock the device.Also, a good way to create electrically-noisy environment is to fire the gas piezo lighter (the kitchen variety) near your I2C wires. Improves the chances from 1:1M to 1:1.

author: SiliconWizard
author_rank: Super Contributor
timestamp: October 17, 2024, 03:16:57 pm
content: Quote from: eutectique on October 17, 2024, 02:36:14 pmYes that's the poor man's EMI generator many of us have used. (Note: that tends to trigger some hilarity from non-EEs, you've been warned.)Also works as a poor man's ESD gun.