SBC random hanging, power supply issues? - Page 1
Forum section: electronics / projects

author: davegravy
author_rank: Regular Contributor
timestamp: January 28, 2025, 11:26:02 pm
content: I have alinux SBCthat I designed a cape for. The cape includes power supply (including solar charge controller), sensors and LTE module. I have a couple prototypes that have been outside ( in Pelican case) during cold (~-10C) weather for stability testing. All has been well except for one unit that hung after 5 weeks and the second that hung after 8... no errors in syslog just a random freeze until I manually rebooted. One of them no longer has a working console UART RX pin, even though nothing was connected to it (suppose I could have ESD'd it accidently during my inspection).I am not extremely experienced, so my first suspicion is my own circuitry. Perhaps power supply noise? I'm supplying power to the SBC via its USB-C port. I have abuck converterthat steps down 12V LFP battery to 5V. I know these converters can be noisy and I just copied the design that TI Webench gave me (see attached), so I'm hopeful I just need to improve filtering because I didn't pay much attention to it.I have an old analog Phillips CRT 20MS/s scope (with a manual I can't decode) and I can't seem to figure out how to set a single-shot trigger level on it. I can see the 5V pin on the SBC has about 50mV pk-pk typically with some occasional transients to 100mV. The SBC doesn't list a supply ripple/noise requirement, what's a reasonable target?Am I right that I should really use a modern scope for this that has working level trigger and/or some kind of max/min stats function, in case there is a rare large transient that's too fast to see? What's a reasonable scope to purchase for this task? Would the Rigol DHO804 be up to it? Is 70MHz fast enough?Any other thoughts / suggestions welcome.

author: thm_w
author_rank: Super Contributor
timestamp: January 28, 2025, 11:36:01 pm
content: Yes something like DHO804 or SDS800X is more appropriate for this task.But I don't think the noise on the 5V is going to be a major issue as the board will have its own regulators as well for 1.8V etc. for the cpu core.Was there any evidence of moisture or corrosion on the board at all?Is this a hobby project or a professional project?Is there any way to use the CPU built in watchdog, if you have not enabled it already?With hardware damage I'm worried about possible ESD/lightning either hitting the LTE antenna (if you have external one) or maybe the input to the solar charge controller. Though those seem unrelated to the UART section at all.Are you sure thats failed and not just some software setting?

author: davegravy
author_rank: Regular Contributor
timestamp: January 29, 2025, 12:06:07 am
content: Quote from: thm_w on January 28, 2025, 11:36:01 pmHmm, not sure what else to look into. There's a JTAG port so I could wait for it to hang again and then see if I can get clues from looking at processor state?Quote from: thm_w on January 28, 2025, 11:36:01 pmNone. Also it's been well below 0C for a few weeks leading up to the latest hang, so any condensation would have frozen in place a while ago.Quote from: thm_w on January 28, 2025, 11:36:01 pmProfessional. It's a device that will be an internal tool for us, so not quite the same polish level as a publicly sold product. The project budget/resources are the lower end of what you'd probably find typically in industry.Quote from: thm_w on January 28, 2025, 11:36:01 pmI believe so, and it's on my list. The systems needs to record data continuously without gaps for reboot so solving the hang is preferred.Quote from: thm_w on January 28, 2025, 11:36:01 pmLightning is pretty rare this time of year but who knows. I'm confident RX is fried because there's console output on that UART during boot but input at the login prompt isn't echoed as it normally is, and the config hasn't been touched. The other unit that hung has all its IO in tact.

author: thm_w
author_rank: Super Contributor
timestamp: January 29, 2025, 12:12:56 am
content: Config can become corrupted. Are you certain that battery voltage never dipped below cutoff at any time?If this is pro project don't screw around with a CRT scope, get proper equipment.Replicating the failure may be difficult and you may have to end up relying on detailed logs to trace back what caused it.The watchdog would not be a solution, but would let the device resume operating and maybe upload its logs or report its current state that might give you a hint as to the issue. Usually its possible to tell what reset source was, so, you can log the watchdog reset on startup and know when and how often they happen.

author: davegravy
author_rank: Regular Contributor
timestamp: January 29, 2025, 12:25:06 am
content: Quote from: thm_w on January 29, 2025, 12:12:56 amFair point. Battery was 13.5V at the instant before, according to coulomb counter it was at 95%Quote from: thm_w on January 29, 2025, 12:12:56 amI did a deep dive into logs and can't find anything out of the ordinary... they just stop cold at the point of the hang, that lead me to suspect hardware.

author: xvr
author_rank: Frequent Contributor
timestamp: January 30, 2025, 12:47:15 pm
content: May be SBC itself?From DS:QuoteDid you use battery?

author: davegravy
author_rank: Regular Contributor
timestamp: January 30, 2025, 02:10:49 pm
content: Quote from: xvr on January 30, 2025, 12:47:15 pmyes, but replaced with one rated to -30C

author: davegravy
author_rank: Regular Contributor
timestamp: January 31, 2025, 07:54:47 pm
content: Quote from: thm_w on January 29, 2025, 12:12:56 amOK, I scoped the 1.8V rail, set a trigger at 1.91V and got the attached within about 30s of waiting.In theATSAMA5D27 datasheet (pg 67)it mentions acceptable range for nominal 1.8V VDDIODDR to be 1.7-1.9V.Mode was set to Peak. Are these transients too short to worry about? Is my scoping method too conservative?

author: thm_w
author_rank: Super Contributor
timestamp: January 31, 2025, 09:59:49 pm
content: That does seem like a lot of ripple, but, are you probing with a short spring lead?Can try with and without a load on the cpu if that matters.I assume the 5V output was completely clean then?It looks like you skimped a bit on the input capacitance (webench recommended ~20uF), and the output inductance is a bit high (webench recommended 1.5uH), if its 5V 3A o/p. But those shouldn't be a major issue.