This code shouldn't work, yet it does - how? - Page 1
Forum section: electronics / general

author: HwAoRrDk
author_rank: Super Contributor
timestamp: January 04, 2019, 01:58:34 pm
content: So there I was, reviewing a piece of code I wrote a while ago, when I noticed that there was a bug in it. "Wait a minute", says I, "how is this code working? It shouldn't be!".The code in question, which is for an ATmega32M1:Code:typedef enum {CAN_BUS_SPEED_100,CAN_BUS_SPEED_125,CAN_BUS_SPEED_200,CAN_BUS_SPEED_250,CAN_BUS_SPEED_500} can_bus_speed_enum_t;typedef struct {can_bus_speed_enum_t bus_speed;struct {uint8_t canbt1;uint8_t canbt2;uint8_t canbt3;} reg_vals;} can_bt_reg_t;typedef struct {struct {can_bus_speed_enum_t bus_speed;} can;/* other members omitted for brevity */} config_t;config_t config = {{CAN_BUS_SPEED_500}};static const can_bt_reg_t can_bus_speed_bt_conf[] PROGMEM = {{ CAN_BUS_SPEED_100, { 0x08, 0x0C, 0x37 } },{ CAN_BUS_SPEED_125, { 0x06, 0x0C, 0x37 } },{ CAN_BUS_SPEED_200, { 0x02, 0x0E, 0x4B } },{ CAN_BUS_SPEED_250, { 0x02, 0x0C, 0x37 } },{ CAN_BUS_SPEED_500, { 0x00, 0x0C, 0x36 } }};void init_can() {for(size_t i = 0; i < (sizeof(can_bus_speed_bt_conf) / sizeof(can_bus_speed_bt_conf)); i++) {if(config.can.bus_speed == can_bus_speed_bt_conf.bus_speed) {CANBT1 = pgm_read_byte(&can_bus_speed_bt_conf.reg_vals.canbt1);CANBT2 = pgm_read_byte(&can_bus_speed_bt_conf.reg_vals.canbt2);CANBT3 = pgm_read_byte(&can_bus_speed_bt_conf.reg_vals.canbt3);break;}}if(CANBT1 == 0 && CANBT2 == 0 && CANBT3 == 0) return;/* rest of CAN peripheral initialisation... */}int main(void) {init_can();/* etc... */}Spotted the bug? It's in this line withininit_can():Code:if(config.can.bus_speed == can_bus_speed_bt_conf.bus_speed) {Yes, I forgot to surround the access ofcan_bus_speed_bt_conf[​i].bus_speedwith apgm_read_byte()call, necessitated due to that variable being declared as PROGMEM. So, it shouldn't work properly, because without that, the comparison will not be against the correct value. What I think should happen is that some random value - whatever resides at the same address in RAM as where the variable actually lives in flash - will be used instead, and the comparison will fail at every loop iteration; this will leave the CANBTn registers uninitialised, and so the later test for that (registers having the default value of zero) will be true, and the function will exit prematurely, leaving the CAN peripheral unconfigured and still in the standby state.However, the bizarre thing is,this code works!In my testing so far, the code appears to successfully set up the correct register values (or, at least,somekind of workable values) for 500kbps operation, configures the rest of the CAN peripheral, and I am receiving stuff from the bus perfectly fine. I just don't understand how...The only reason I can think of for this to work is by accident or happenstance: that when it readscan_bus_speed_bt_conf[​i].bus_speedit's looking at a RAM area with a value of 0x04, which matches theCAN_BUS_SPEED_500enum member's value. If the value in memory was something else that happened to match any other enum value, yes it would initialise the registers, but for the wrong CAN bus speed, and I wouldn't have a working system - which isn't the case.So, why does this code work? How can a PROGMEM variable accessed without using apgm_read_*function yield the correct value still?

author: andersm
author_rank: Super Contributor
timestamp: January 04, 2019, 02:31:32 pm
content: GCC has supportednamed address spacesfor some years now. If your environment is set up to use that support when available (ie. the PROGMEM macro resolves to __flash, and not the progmem attribute), then the pgm_read accessors aren't needed.For the ultimate truth, examine the disassembly.

author: HwAoRrDk
author_rank: Super Contributor
timestamp: January 04, 2019, 03:11:40 pm
content: Quote from: andersm on January 04, 2019, 02:31:32 pmThePROGMEMmacro resolves to__attribute__((__progmem__)), so the named address spaces stuff definitely doesn't apply in this case.I hadn't actually heard of the named address spaces feature, seems interesting, so thanks for pointing that out.Quote from: andersm on January 04, 2019, 02:31:32 pmI don't really know what I'm doing with regard to AVR assembler, but if I check what the compiler generated, what I can see is that there are only three instances of theLPMinstruction in the entirecan_init()function, which corresponds to the three instances ofpgm_read_byte()I was actually using. So I would guess it is safe to assume that the code can't be accessing flash in the aforementioned offending line of code.

author: HwAoRrDk
author_rank: Super Contributor
timestamp: January 04, 2019, 03:37:25 pm
content: I thought I would try and get some more detailed idea of what was going on with the code during execution by running it in a debugger. But I don't have an ICE, so the Atmel Studio simulator will have to do. But there is no simulator for the ATmega32M1 (), so what I did instead was copy the relevant sections of code into a temporary project targeting an ATmega328P, and replace the CAN register references (CANBT1, etc.) with some volatile uint8_t variables.When I stepped through the code, it did appear that without a call topgm_read_byte, it was accessing RAM. The structure of memory in this temp test code was such thatcan_bus_speed_bt_confhad an address of 0x00A8, and looking at that region showed it contained a run of zeros longer than the size of the variable; thus, the resulting value read got interpreted asCAN_BUS_SPEED_100, the first enum member, on every loop iteration.So, is my theory about the code accidentally working correct? By a matching value just happening to be in the appropriate location in RAM? I don't see any other way.Is there any way of finding out, without the aid of a debugger, whether this is what's happening?

author: ale500
author_rank: Frequent Contributor
timestamp: January 04, 2019, 03:54:45 pm
content: Atmel studio used to support a simulator, did you try that ?You can disassemble the object files using objdump -d name_of_the_object

author: ajb
author_rank: Super Contributor
timestamp: January 04, 2019, 05:42:33 pm
content: Quote from: HwAoRrDk on January 04, 2019, 03:37:25 pmSeems like that's exactly what's happening. So your title is a bit inaccurate: the code just happens to produce correct results for certain cases, but this not at all the same as actually working!QuoteSimulate it, as you are doing, or step through the disassembly manually and compare to the map file.Side note: since your CAN config data is fixed, you could avoid the loop entirely by simply using your enum to index into the config array. Since this is an init function it probably has almost no impact on real performance, but it would be a more elegant solution. Your method would be preferable if you were indexing on a non-contiguous value (like looking up a numeric baud rate in a list of fixed options, for example), but in the code shown there's no reason to not make the enum values contiguous and use them as indices. (Unless otherwise defined, enum values will be contiguous starting from zero per the C spec, but whenever the exact value is important it's always a good idea to declare them explicitly to make your intentions clear.)

author: SiliconWizard
author_rank: Super Contributor
timestamp: January 04, 2019, 05:49:44 pm
content: If the compiler is GCC-based, use the '-S' option to generate assembly for this C source file, and see what it actually compiles to.

author: T3sl4co1l
author_rank: Super Contributor
timestamp: January 04, 2019, 05:52:39 pm
content: It appears the for() loop is static, i.e., wholly known at compile time. Is it being unrolled or hard-wired?I don't know offhand if GCC is quite this smart/aggressive, depending on optimization level.Speaking of, you didn't mention what optimization level if any you're using.But in any case, the given code should indeed access RAM, and coincidences can happen. Just because it's a bug doesn't mean it's an automatic crash!Tim

author: cv007
author_rank: Frequent Contributor
timestamp: January 04, 2019, 07:18:35 pm
content: ./avr-objdump -d -j .data my.elfwill show the data section and initial values (which could change by the time the can init runs)./avr-objdump -t my.elfwill show all symbols, where you should be able to find the 'can_bus_speed_bt_conf' symbol and its address(and there are most likely another dozen variations that can get the info)probably have to work out the array offsets to bus_speed first, then see what variables in .data match up- or maybe you end up in the bss section and are looking at 0's so get the first array matched

author: richardman
author_rank: Frequent Contributor
timestamp: January 05, 2019, 12:48:23 am
content: Unlikely to work "by accident".I don't know GCCAVR for sure, but with our own AVR compiler, if a named object has the __flash attribute, it will generate the LPM instructions. I would imagine that a later release of GCC might have done the same. I know the original version didn't, but it's not a difficult feature to add.

author: HwAoRrDk
author_rank: Super Contributor
timestamp: January 05, 2019, 10:07:07 am
content: Quote from: ajb on January 04, 2019, 05:42:33 pmI did actually start out with the code like that, but I specifically changed it to how it is now because I wanted to make it more explicit - you know, so it's obvious that "these are the register values for CAN_BUS_SPEED_500", etc. - rather than leaving it implied or only noted with a comment. Like you say, it's only within initialisation code - it only ever runs at start-up and waking from sleep - so I don't particularly care that its not the most efficient or condensed code.Quote from: T3sl4co1l on January 04, 2019, 05:52:39 pmIt appears not to be unrolled. As I said earlier, the disassembly of the function only contains threeLPMinstructions (matching the threepgm_read_byte()calls), whereas if it was unrolled I would expect 15 (3x5). What impact would an unrolled loop have on the situation?Quote from: T3sl4co1l on January 04, 2019, 05:52:39 pmAs far as I recall, Atmel Studio defaults to -O1 for Debug level compilation, which is what I was using. I believe it's -Os (or maybe -O2?) for the Release configuration, but I've so far yet to compile this code with that.Quote from: cv007 on January 04, 2019, 07:18:35 pmI had a look at this, but I've practically no idea what I'm looking at (or for).The latter command includes this line of output with reference to that variable:Code:0000009c l O .text00000014 can_bus_speed_bt_confI'm clueless about what the output of the former is representing.By the way, Atmel Studio by default outputs with compilation not only an objdump disassembler listing (-S output, I believe) but also a .map file with a ton of information in it, if that is any help.

author: westfw
author_rank: Super Contributor
timestamp: January 05, 2019, 10:16:08 am
content: Likely an accident.However, avr-gcc WILL highly optimize access to PROGMEM structures at compile time if you forget the pgm_get_xxx() call.It's an occasionally cause of apparently mysterious bugs where, ieCode:Serial.print(foo.menu); // print top-level menu value - works fine.Serial.print(foo[readDecimal()].menu); // doesn't work, even when 0 is entered(where foo is some array in PROGMEM.)You can actually use this to implement "digitalWriteFast()" in the Arduino World, for boards that do link-time optimization, since the "pin" to port/bit tables are constant arrays in progmem:Code:// Compiles to SBI/CBI when possible!static inline __attribute__((always_inline)) void _dwfast(int pin, int val) {uint8_t mask = digital_pin_to_bit_mask_PGM;uint8_t port = digital_pin_to_port_PGM;volatile uint8_t *outport = (volatile uint8_t*)port_to_output_PGM;if (digital_pin_to_bit_mask_PGM == 0x99) // Never trueLTO_Not_Enabled();if (val) {*outport |= mask;} else {*outport &= ~mask;}}

author: T3sl4co1l
author_rank: Super Contributor
timestamp: January 05, 2019, 01:51:08 pm
content: Quote from: HwAoRrDk on January 05, 2019, 10:07:07 amNo, I mean unrolled as in, instead of doing a loop that does known things each time, it justgoes and does those things!There would be no LPM because that value is also known at compile time.In this case, it's looping over an enum, and the comparison target is static, so it really only needs to ask one thing:Code:if(config.can.bus_speed == CAN_BUS_SPEED_500) /* true */ { ... }I may be making more assumptions about config.can.bus_speed than the compiler would actually take, but this process is entirely reasonable if so.So to really understand what's going on, learn the instruction set and follow the execution path. Failing that, you might look at the listing for different optimization levels and see which statements are broken up the most.Tim

author: EEVblog
author_rank: Unknown
timestamp: January 05, 2019, 02:29:09 pm
content: Quote from: HwAoRrDk on January 04, 2019, 01:58:34 pmPower-on default register values perhaps?

author: cv007
author_rank: Frequent Contributor
timestamp: January 05, 2019, 02:47:29 pm
content: You can work your way through this simple example to find where vars are located-Quoteand you can see in the end, the compiler did not incorrectly get the wrong data for the mypgm var as it used the known value instead of (incorrectly) loading from its address (which is actually a .text address).Doesn't sound like this is happening to you, but you would have to actually see the disassembled code to know (I wouldn't take those present lpm instructions as proof of anything, I would want to figure out exactly what is happening).What I am trying to suggest, is that you can know what vars are at what address, and even without debugging you could see which vars are being accessed incorrectly if the compiler was trying to read a var in progmem without the progmem read function (reading from data). You could then deduce what is actually being loaded incorrectly (although the data vars could change before that, you would know which vars to look for in your code and could figure out if they change prior to being accessed).QuoteAre all 0's for these 3 registers, which could probably be an explanation as the first is probably most important and is 0 for his desired value, but he says he checks all 3 registers for 0 before trying to further init and exits init if all 3 are 0;Those unintended reads to data could also end up after the bss section, where a memory value could be anything, and will most likely be different at every power on. That would be fun.

author: HwAoRrDk
author_rank: Super Contributor
timestamp: January 06, 2019, 06:15:04 am
content: Thanks to cv007 for the great example showing how the different ways variables are defined relate to how they end up in different sections of the binary. Made me understand a bit more about what the .data, .bss and .text sections are about.After sitting down and poring over the disassembly of mycan_init()function, I think T3sl4co1l might be on to something: that the compiler might have optimised away the accesses tocan_bus_speed_bt_conf.Here is the section of code of interest:Code:2ee:80 91 64 01ldsr24, 0x0164; 0x800164 <__data_start+0x64>2f2:88 23andr24, r242f4:a1 f0breq.+40; 0x31e <can_init+0x6a>2f6:81 30cpir24, 0x01; 12f8:49 f0breq.+18; 0x30c <can_init+0x58>2fa:82 30cpir24, 0x02; 22fc:51 f0breq.+20; 0x312 <can_init+0x5e>2fe:83 30cpir24, 0x03; 3300:59 f0breq.+22; 0x318 <can_init+0x64>302:84 30cpir24, 0x04; 4304:21 f5brne.+72; 0x34e <can_init+0x9a>It starts off with what I believe is loading the value ofconfig.can.bus_speedinto r24. Then there is a logical-AND-ing of r24 with itself, which I'm not sure of the reason for. After that, it is doing a sequence of copying a constant value to r24 and then branching if r24 is equal to something.The constant values immediately jump out at me as being the exact implicit values thecan_bus_speed_enum_tmembers will have! So, what I think has happened here is that the for-loop has been unrolled, and the compiler has 'hard-coded' the values ofcan_bus_speed_bt_conf[​i].bus_speed!The AND instruction still puzzles me though. Although, because it's also followed by a branch instruction, I guess it's somehow the test for the first enum member (which has a value of zero)?Also puzzling is the locations the branch instructions jump to. Becauseconfig.can.bus_speedis set to 500kbps, i.e. a value of 4, I would assume that last branch to 0x34e would be the path chosen. However, that goes straight to a piece of code that appears to be theCANBT1 == 0, etc. test (some loading of those registers and comparing/skipping). That doesn't make any sense, as it would be omitting the loading and assignment of the registers altogether! Maybe that branch test is never true? Kind of makes sense, as if that path is not taken, it drops straight into some code that again loads a constant 16-bit value of 4 into two adjacent registers (r24 & r25) and then jumps into the code that features the three LPM instructions which sets the CANBTn registers.Here's the whole section of assembler for reference, in case anyone has a different interpretation:Code:2ee:80 91 64 01ldsr24, 0x0164; 0x800164 <__data_start+0x64>2f2:88 23andr24, r242f4:a1 f0breq.+40; 0x31e <can_init+0x6a>2f6:81 30cpir24, 0x01; 12f8:49 f0breq.+18; 0x30c <can_init+0x58>2fa:82 30cpir24, 0x02; 22fc:51 f0breq.+20; 0x312 <can_init+0x5e>2fe:83 30cpir24, 0x03; 3300:59 f0breq.+22; 0x318 <can_init+0x64>302:84 30cpir24, 0x04; 4304:21 f5brne.+72; 0x34e <can_init+0x9a>306:84 e0ldir24, 0x04; 4308:90 e0ldir25, 0x00; 030a:0b c0rjmp.+22; 0x322 <can_init+0x6e>30c:81 e0ldir24, 0x01; 130e:90 e0ldir25, 0x00; 0310:08 c0rjmp.+16; 0x322 <can_init+0x6e>312:82 e0ldir24, 0x02; 2314:90 e0ldir25, 0x00; 0316:05 c0rjmp.+10; 0x322 <can_init+0x6e>318:83 e0ldir24, 0x03; 331a:90 e0ldir25, 0x00; 031c:02 c0rjmp.+4; 0x322 <can_init+0x6e>31e:80 e0ldir24, 0x00; 0320:90 e0ldir25, 0x00; 0322:88 0faddr24, r24324:99 1fadcr25, r25326:88 0faddr24, r24328:99 1fadcr25, r2532a:fc 01movwr30, r2432c:e3 56subir30, 0x63; 9932e:ff 4fsbcir31, 0xFF; 255330:e4 91lpmr30, Z332:e0 93 e2 00sts0x00E2, r30; 0x8000e2 <__TEXT_REGION_LENGTH__+0x7e00e2>336:fc 01movwr30, r24338:e2 56subir30, 0x62; 9833a:ff 4fsbcir31, 0xFF; 25533c:e4 91lpmr30, Z33e:e0 93 e3 00sts0x00E3, r30; 0x8000e3 <__TEXT_REGION_LENGTH__+0x7e00e3>342:fc 01movwr30, r24344:e1 56subir30, 0x61; 97346:ff 4fsbcir31, 0xFF; 255348:84 91lpmr24, Z34a:80 93 e4 00sts0x00E4, r24; 0x8000e4 <__TEXT_REGION_LENGTH__+0x7e00e4>34e:80 91 e2 00ldsr24, 0x00E2; 0x8000e2 <__TEXT_REGION_LENGTH__+0x7e00e2>352:81 11cpser24, r1354:0b c0rjmp.+22; 0x36c <can_init+0xb8>356:80 91 e3 00ldsr24, 0x00E3; 0x8000e3 <__TEXT_REGION_LENGTH__+0x7e00e3>35a:81 11cpser24, r135c:09 c0rjmp.+18; 0x370 <can_init+0xbc>35e:80 91 e4 00ldsr24, 0x00E4; 0x8000e4 <__TEXT_REGION_LENGTH__+0x7e00e4>362:88 23andr24, r24364:09 f4brne.+2; 0x368 <can_init+0xb4>366:a5 c0rjmp.+330; 0x4b2 <__LOCK_REGION_LENGTH__+0xb2>368:80 e0ldir24, 0x00; 036a:03 c0rjmp.+6; 0x372 <can_init+0xbe>36c:80 e0ldir24, 0x00; 036e:01 c0rjmp.+2; 0x372 <can_init+0xbe>370:80 e0ldir24, 0x00; 0

author: T3sl4co1l
author_rank: Super Contributor
timestamp: January 06, 2019, 08:46:08 am
content: Ah, nice.AND regN, regN is shorthand for putting the register into flags: in particular the zero flag is desired here. On x86, TEST reg, reg is more common (same thing, but doesn't write result back to dest -- not that the result has changed in this case!). CMP reg,0 probably isn't any slower or longer, the compiler just chose to emit AND instead. OR regN, regN would do just as well, of course (on any instruction set I'm aware of, pretty much all logical operations take the same time and space).So, it's just an if(0) else if (1) else if (2) else... statement.This is also a common structure for switch() statements: below a certain length, it's just built as an if-elseif chain. Above a certain length, it's bounds-checked then indexed to a jump table. The jump table can be quite costly in space (i.e., as many words of PROGMEM as elements between bounds), but relatively fast (in particular, O(1) rather than O(N)).Tim

author: cv007
author_rank: Frequent Contributor
timestamp: January 06, 2019, 02:44:37 pm
content: I could be wrong, but I think what is happening is this line is simply eliminated-Quoteas the enums match up to the array indexes AND the values are restricted to just those enumsso it simply becomes a matter of getting which index is in config.can.bus_speed, then compute the offsets into can_bus_speed_bt_conf and load the values from flash and store them in the canbt registers.which also means you cannot fail to get a value so the test for all canbt==0 is not needed (unless you have a stored set with all 0's I guess)(that is one reason enums are useful- you restricted the values without having to check them- the compiler does it)change those enum values to something else and you will probably not recognize the resulting assembly from the previous(or change them to #defines and see what happens)