Masters of ARM, pull in - Page 1
Forum section: electronics / general

author: JVR
author_rank: Regular Contributor
timestamp: November 18, 2014, 08:15:59 am
content: I have a small problem.I have a system running on a Freescale (spit) Kinektis K60 Cortex M4, with the RTX RTOS.Now the unit resets every few minutes, I've traced this using the SRSH register to aReset caused by core LOCKUP eventAnd that is where I am stumped, Freescale datasheets are shit, and I cannot find a reason in there as to what would cause a Core Lockup, anyone have any idea's of what I can look at?Regards,JVR

author: Whales
author_rank: Super Contributor
timestamp: November 18, 2014, 09:50:02 am
content: What have you tried so far to debug it?Treat it like any old error, eg a loop explosion or memory implosion. It might even be a hardware issue (ie unstable power -> glitches)

author: hans
author_rank: Super Contributor
timestamp: November 18, 2014, 11:01:23 am
content: Always remember for ARM that the CPU core is from an external party to Freescale as well, so you should search ARM sites first for better info.Lockup Cortex m4:http://infocenter.arm.com/help/topic/com.arm.doc.dui0553a/BGBFCHGC.htmlIt highly suggests that you have had a hardfault in your program, (unless you have a Non-maskable interrupt going). That can be a memory access issue (unaligned pointer, null pointer, stack problem etc.). There are probably some registers further that will describe the source of the hardfault issue.

author: dannyf
author_rank: Super Contributor
timestamp: November 18, 2014, 11:36:44 am
content: Sounds like a core error. I would try to find out the equivalent of it in the ARM manual and go from there. There are a bunch of fault status register and PC at time of the fault. The book by Yiu shows a set of approaches to debug a hard fault.Hard faults are either stupid errors (using a peripheral without routing clocks to it) or quite unusual - memory errors for example. So if I were you, i would try to rule out other causes first.

author: Brutte
author_rank: Frequent Contributor
timestamp: November 18, 2014, 12:44:40 pm
content: Quote from: JVR on November 18, 2014, 08:15:59 amDid you consider RTFRMing?First you should read the manual and then whinge on Freescale who has nothing to do with the core.QuoteLockup is a well defined event that is gracefully serviced by the core.Everything is explained in TFRM.In the opening postI have attached the view of part of core resources that are helpful in finding what causes those and other "unexpected" events. And of course stack analysis (unwinding it) is a must.

author: westfw
author_rank: Super Contributor
timestamp: November 18, 2014, 01:22:44 pm
content: Sounds like a fault in the hard fault handler.You can trivially create this sort of error by having even event handler addresses (ARM rather than Thumb) in the vector table, but I wouldn't expect something like that from a packaged RTOS...

author: nctnico
author_rank: Super Contributor
timestamp: November 18, 2014, 02:36:26 pm
content: For debugging purposes it is a good idea to have a hardfault handler which dumps the program counter and a small stack trace to the serial port. It will tell where the program crashed exactly and some clues from where a function was called.

author: Brutte
author_rank: Frequent Contributor
timestamp: November 18, 2014, 03:27:12 pm
content: Quote from: nctnico on November 18, 2014, 02:36:26 pmJust connect the debugger and put a program breakpoint in 0x0000000C. No need to write anything when/if you have a debugger at hand.By the way, lockup event won't raise HardFault handler anyway so HardFault handler won't help.

author: Sal Ammoniac
author_rank: Super Contributor
timestamp: November 18, 2014, 06:51:44 pm
content: Does it actually reset or just encounter a fault (like a hard fault or a bus fault)?If it's an actual reset, and it's happening every few minutes, perhaps you have the watchdog turned on and it's timing out and resetting the processor.

author: westfw
author_rank: Super Contributor
timestamp: November 18, 2014, 11:34:31 pm
content: QuoteDoes that work? 0xC is a vector location, not an address that is executed. I'm not sure how the breakpoint and other debug logic treats the vectors...QuoteThe ARM docs says that lockup happens if there's a fault in the HardFault handler. You PROBABLY want to catch the first hardfault and fix that, before the lockup happens.I don't know anything about RTX, but it's pretty common for an RTOS to relocate the vector table to RAM, so that handlers can be installed dynamically. That has two immediate implications:1) All the advice about 0x0C may not be useful.2) check to see if the vectors in the table are still valid; if it's in RAM, something could have trashed it, even if it was initially set up perfectly.

author: nctnico
author_rank: Super Contributor
timestamp: November 19, 2014, 12:52:50 am
content: A Cortex Mx device has several fault handler interrupts. What they do by default depends on what they are programmed to do. Some get stuck in a loop so the watchdog (if enabled) resets the device. Another option is to have the code continue or reset the device from the fault handler interrupt. BTW: using a debugger may be fine during development but the ability to get some sensible data from the serial port without the debugger attached & armed all the time has proven very useful. It can even help a field service engineer to report a bit more beyond 'the device isn't working' in case of problems.

author: Jeroen3
author_rank: Super Contributor
timestamp: November 19, 2014, 06:46:44 am
content: Continuing from a hardfault is very difficult, I'd advise against.You can get a register dump and the address of where the hardfault occured. Ditch this to the debug uart.You can also detect in which thread the fault occured and maybe not start that thread the next time.ARM cores know if the debugger is attached, so you can have code pass a breakpoint in the hardfault handler. (A breakpoint instruction without debugger resets the chip)It is common to have a routine at the start of your program checking the reset source (there are registers for that). If you encounter a lot of watchdog errors, or brownouts, you might want to switch to an alternate program flow. A safemode for example.

author: Brutte
author_rank: Frequent Contributor
timestamp: November 19, 2014, 08:38:46 am
content: Quote from: westfw on November 18, 2014, 11:34:31 pmYou are right, hint was not precise. Rather "put an address of a program breakpoint (+T) at 0xC". Mind any valid address would do if you tick a vector catch option. Once a uC halts there, the debugger does all the messy job automagically, reading registers, call stack etc.QuoteNeither do I but in such RAM case I would have used an MPU and secure all the handlers from any runtime modifications.

author: JVR
author_rank: Regular Contributor
timestamp: December 10, 2014, 08:27:25 am
content: So after loads of head scratching and a shitload of help from Joseph Yiu's Definitive Guide to Cortex as to how the exception pipeline works, it appears that a Core Lockup happens when there is an error while handling a Hardfault. Armed with this knowledge and a POS Ulink Pro that bombs out every 5 minutes, I eventually traced it to a message box that was trying to access memory @ 0x25A55525, ergo, nonexistent. It turned out to be another task completely that would wait for a GPS signal before filling a buffer, and this buffer would then overrun. The delay in acquiring GPS is what caused the randomness in the time for the failure to happen. Fixed the buffer code and its running happily.

author: jerry507
author_rank: Regular Contributor
timestamp: December 11, 2014, 08:13:58 pm
content: In the future make sure you enable your other handlers. That way if you get a memory management fault, you'll actually get that fault. The problem often happens that you get a lesser fault, but since you don't have them specifically enabled, they escalate to hard faults. If you're doing something in that handler and things have gone to hell, you'll generate a hard fault from that code. Now you've got a lockup event which has destroyed most of the evidence.