did i finally find the need for a better debugger? - Page 1
Forum section: electronics / general

author: JPortici
author_rank: Super Contributor
timestamp: November 23, 2016, 07:44:52 pm
content: Today i wasted many work hours tracing an issue (which is not solved yet but a workaround was found, need more testing though)BACKSTORY:I am in the process of rewriting my SENT TX emulation library for dsPIC because i needed it to be more flexible (and *faster*)and while i left it to test in asensor w/ SENT -> SENT in -> dspic33ev256gm106 -> SwSENT out -> scopescenario, it kept hanging at random times. After much looking i found out that the dedicated timer had the period register set to zero.. and of course the corresponding dedicated interrupt wouldn't be fired until a manual reboot.The part of the code which changes the dedicated timer period register wasn't modified so probably something else in the program is messing with me (is a pointer screwing around somewhere it isn't allowed, even if boundary checks are implemented? or a nested interrupt modifying accumulators that aren't saved? are nested interrupts actually enabled? i don't remember at the moment. i'm exausted.)and for now i implemented a workaround with a do-while loop that keeps rolling while PR is zero at every operation that modifies it, okay for the moment so i know that the micro is hanging there and i haven't solved the issue.* a simillar problem was found doing multiplications which seems to return zero at random times even if the operands are both different than zero.problem is, even if i stop the debugger when new PRx is zero (and it can never be so it's always a fault condition) i only can look at the call stack (pickit3) and while it can be useful at least for tracing back trap causes, having a list of at least the last 5-10-15 instructions could be a tremendous help.

author: RogerRowland
author_rank: Regular Contributor
timestamp: November 24, 2016, 06:25:39 am
content: Doesn't a data breakpoint get you what you need?http://microchip.wikidot.com/mplabx:set-data-breakpoint

author: JPortici
author_rank: Super Contributor
timestamp: November 24, 2016, 07:06:51 am
content: but does it?even if it stops when (in this case) PRx = 0 as far as i know i can't see the last N instructions excecuted (which is NOT looking at the disassembly file i believe)

author: RogerRowland
author_rank: Regular Contributor
timestamp: November 24, 2016, 08:07:29 am
content: Well, if it breaks at the instruction that wrote zero to the register, presumably that's your bug - bad pointer or whatever, per your earlier postulations. No need to see any further back is there?

author: JPortici
author_rank: Super Contributor
timestamp: November 24, 2016, 08:29:16 am
content: well the issue at hand seems to be a multiplication returning zero as one of the operand is a positive constant and the other is a variable always greater than zero (both are small enough to ensure that the result is at most 16 bit wide, no overflow) which means one of the accumulators was zero, as the operands were verified to be in the correct ranges. manually forcing the routine to proceed clears the problem for an indefinite numbers of cycles, having the instruction trace history would tell me exactly what happened with that multiplication

author: RogerRowland
author_rank: Regular Contributor
timestamp: November 24, 2016, 08:37:25 am
content: Quote from: JPortici on November 24, 2016, 08:29:16 amSounds intriguing. Any chance you can post a snippet of the relevant code? More eyeballs on it could be useful.

author: nctnico
author_rank: Super Contributor
timestamp: November 24, 2016, 11:35:18 am
content: How about checking the output and use a printf (output to serial port) to print the input variables when the result is outside the range? If you format the output sensible then you could collect logging data you can read into Excel and perform some analysis on it.

author: enz
author_rank: Regular Contributor
timestamp: November 24, 2016, 11:39:46 am
content: Check your stack.Weird problems like this often happen due to a stack overflow.

author: JPortici
author_rank: Super Contributor
timestamp: November 24, 2016, 11:57:08 am
content: after i put the failsafe do while loop for PRx the funny stuff happened hereCode:unsigned int SwSENT2_Ticks,SwSENT2_Tick, SwSENT2_Nibbles;...for (i=0;i<8;i++) {SwSENT2_Ticks = SwSENT2_Tick * (SwSENT2_Nibbles + 12);}...(i is local variable)Nibbles come from a previous passage where it's masked with 0xF so it's limited to 15. Tick is not too large, depends on Fp and a couple of prescalers... it around 50 in this specific program i think so zero is not an effect of overflow.XC16, Opt 0the disassembly pointed to the fact that the compiler was using almost only W0 and W1 which is asking for trouble. Entering an interrupt meanwhile could mess things up. it actually had in other projects where i relied too much on assembly.I too have tought about a stack overflow especially because i have 600 bytes or so left of ram but when things like this happened something spectacular like an address error trap orReserved Trap 7was called (btw what the hell is that? neither support guy or users in the forum would tell me)using printf? i wish. no pins left for second uart and present uart is already used by another tool (develop unit is attached to a testbench)Anyway, looking at the disassembly made me think about a stack problem so i went immediately to disable nested interrupt and see if things changed. they did! about three hours without SwSENT ever locking up, definetly a big improvement. Removing the lockup/forcing the start of a new transmission is just a matter of setting a bit so i prepared some kind of a watchdog too. will be added once i'm sure i've solved this one(but my OP was intended to be more about "what do i do when the basic debugger is not enough help? oh i wish i could have X")

author: RogerRowland
author_rank: Regular Contributor
timestamp: November 24, 2016, 02:06:40 pm
content: Quote from: JPortici on November 24, 2016, 11:57:08 amYes, understood and I concur. Embedded stuff is my hobby, I've been 40 years writing software on mainframes, minis and PCs, and the debugging tools on desktops rather spoil you when you try to do similar stuff with MPLAB X and a PICKit 3 (as do I).The cost of something "better" like an ICD 3 or REAL ICE is a touch too much for a hobby, so I tend to fall back on the old methods I used before desktop debuggers were a thing - frequent function/module testing during development, extensive logging, etc. (which sometimes causes more problems, especially with time-critical code) - but also just toggling an spare output pin is a great help to trace the flow - I use different numbers of toggles to checkpoint sections of code and also use it for measuring function time to see if it meets expectations (often I've cocked up a timer setting or an interrupt priority or something and things run much slower than planned).In your case, multiple nested interrupts and high risk of stack overflow gives you the worst of everything - those types of issues on desktops (i.e. multi-threaded, memory intensive tasks) are also difficult to debug because the cause is often far removed - in code lines and time - from the observed error. Having said that, most of the PICs I've used (8-bit and 32-bit) allow you to trap stack issues with a forced reset and a status register on restart that at least tells you it happened.