Wanted: Technical advice/tips for a PIC18F Bootloader - Page 1
Forum section: electronics / general

author: igendel
author_rank: Frequent Contributor
timestamp: November 09, 2016, 07:40:24 am
content: Hi all,I'm about to start my first project with a bootloader of my own in it. The MCU will be a PIC18F45K22 (or a very close one), and I'll be using MPLAB X / XC8.The thing I'm not sure about it how to build the code so that the bootloader and the primary code will go each to its proper place. That is, technically, how do I tell the compiler that program X is a bootloader and should be written to the FLASH at position , and that the HEX file for program Y should have FLASH addresses elsewhere... is there a compiler flag? Should I edit some map file (and how do I do it)?I searched for PIC bootloaders and app notes, but so far could only find information that's irrelevant to my specific question.Any help, suggestions and gotcha warnings will be appreciated

author: hans
author_rank: Super Contributor
timestamp: November 09, 2016, 07:54:43 am
content: Linker file

author: igendel
author_rank: Frequent Contributor
timestamp: November 09, 2016, 08:12:46 am
content: Quote from: hans on November 09, 2016, 07:54:43 amWell, that's definitely terseI'll explore Google, but meanwhile, if someone can help with some specific details or things to watch out for...Thanks[Edit: I found thishttp://www.microchip.com/webinars.microchip.com/WebinarDetails.aspx?dDocName=en558478, seems to be very useful but it's 4 years old - hopefully the compiler and IDE didn't change too much since]

author: JPortici
author_rank: Super Contributor
timestamp: November 09, 2016, 11:59:15 am
content: I'm in the process of writing one myself (i also received a notification from the forum. was it you?)provided you are using XC8, in the project settings you select XC8-Linker. Select the tab "Additional Options" and adjust offset.the Offset parameter shift the start of the code at that particular address, leaving everything below empty.It is different than adding a fixed number to each address of the hex file or something like that, as that would break every access at program memory like reading a constant.So, you write your code and set Offset to be 800 or whatever (hex) address is the start of the second memory block. I'm writing one for the 26k22 which should have the same boot/page size but please refer to the datasheet of you MCU.

author: JPortici
author_rank: Super Contributor
timestamp: November 09, 2016, 01:40:18 pm
content: surely you want to avoid compiler generated crap in the interrupt handling. Maybe this is not necessary (I have to investigate) but here it is:I don't need interrupts in the bootloader as the communication will need handshacking and the PC side will wait for a redy signal from the bootloaderSo my interrupt handler just need to be a goto to the application interrupt handler.. but if i write thisCode:void __interrupt() intHroutine() {/* High priority interrupt routine */asm("GOTO 0x802");}the compiler will waste a lot of time saving and restsoring all the context prior actually executing the interrupt routine.. and your application will be doing that AGAIN. Being unable to disable context saving in free mode i solved the issue in this wayCode:void __interrupt() __section("intHsect") intHroutine() {/* The use of section moves all the compiler crap away. interrupt vector is* actually a goto to the application vector.* add to linker command this line: "-L-pintHsect=800h" */NOP();}(cleaner ways to do so are appreciated)

author: hans
author_rank: Super Contributor
timestamp: November 09, 2016, 02:05:53 pm
content: Quote from: igendel on November 09, 2016, 08:12:46 amSorry, didn't have that much this morning, but wanted to give at least 1 hint to what you should look out for.In the linker file you can move which flash regions the program should be compiled to. You can limit the region for the bootloader and the application.How you define those regions is up to you, and sometimes defined by the constraints of the chip. E.g. if the chip has bootloader FLASH, it can make sense to put the bootloader in that space.It's ofcourse important to link and write the program to the right space in FLASH because all calls are done by absolute addresses.Other 2 challenges are:1) Jumping to program (usually you can figure out the main() function location from the vector table)2) Remapping interrupts to those of the application. Interrupt handlers are referred by the interrupt vector table. For example on most ARM processors for example you can put this table into RAM and set a register in the CPU to use the RAM vector table. I'm not sure how this works on PIC18; possibly there is also a register that remaps this address correctly.

author: igendel
author_rank: Frequent Contributor
timestamp: November 09, 2016, 05:44:30 pm
content: It actually looks easier in that Microchip video I found (see above)I think I know enough now to start experimenting, but there's one question that occurred to me while reading your answers: what's all this about redirecting/sharing interrupts? From the video, it seems each code (bootloader/primary) has its own interrupt vector "table" (it's PIC18F, so it's just two entries). Under what circumstances would I want to enable a primary-code interrupt while the bootloader is still running?Thanks again

author: JPortici
author_rank: Super Contributor
timestamp: November 09, 2016, 06:09:20 pm
content: in the pic18 there are only two interrupt vectors (high and low priority.. but i think the priority system is bugged and i never have seen it in use anyway) that are not relocatable. and of course only one reset vector.this means that either the bootloader reside in the lower part of memory or it resides in other parts (usually the highest memory page).analyzing the two possibilities:BOOTLOADER IN THE BOOT SECTOR- MCU start and reset in bootloader.- Interrupt vector is in the bootloader- Ideally you don't change and even write protect the bootloader (to be sure it doesn't write itself due to a bug) so in this case the whole boot block gets locked. this means that you can't change the interrupt vector and whenever the application needs to service an interrupt it will jump back in the bootloader area. even if you don't use interrupts in the bootloader you will have to get outside and back in the application in some wayBOOTLOADER IN ANOTHER SECTOR- The application start first- No need to change the interrupt vector/table/location- But entering the interrupt depends on the application. if that part is omitted or gets corrupted you're screwed. you have to re-flash everything

author: igendel
author_rank: Frequent Contributor
timestamp: November 09, 2016, 07:59:40 pm
content: Quote from: JPortici on November 09, 2016, 06:09:20 pmOK, that seems correct... in that video they indeed define a JMP from the normal vector (at the low address) to the application's vector, using some compiler magic words.Which brings me to a different question - what if I want both the bootloader AND the application to use interrupts? Do I have to include someifin the ISR to see who is calling it, and act accordingly?

author: JPortici
author_rank: Super Contributor
timestamp: November 09, 2016, 11:25:13 pm
content: exactly. for example declare a variable, common between the two, at a fixed address. if it's clear go to the application ISR otherwise skip.. or whatever.but why would you need interrupts? you can't implement some sort of flow control?

author: igendel
author_rank: Frequent Contributor
timestamp: November 10, 2016, 08:10:44 am
content: Quote from: JPortici on November 09, 2016, 11:25:13 pmI'm not sure yet, but my Bootloader may use UART as well as a primitive UI (blinking LED / Switch). It doesn't feel right to do it all with just pollingA fixed-address RAM variable as a flag seems a bit of a hack - but it makes perfect sense. I'll try all these techniques and update this thread when I have results.Thanks again everyone!

author: JPortici
author_rank: Super Contributor
timestamp: November 10, 2016, 08:28:37 am
content: but are you running at 4 MHz? in that case it would be understandableor if you are using higher / more complex protocols like spi/i2c. or if you don't have flow control, handshacking, .. , sure.but ideally your bootloader should be idling and checking for new data most of the time. the only time intensive task is the actual programming but that can't be made faster. it's physics! and you don't always do that, you have to erase/program 64 bytes at a time.

author: igendel
author_rank: Frequent Contributor
timestamp: November 10, 2016, 09:25:19 am
content: Quote from: JPortici on November 10, 2016, 08:28:37 amHmmm... I see now a strong argument against shared interrupts - they will create a very unhealthy dependency between the bootloader code and the application code (after all, a truly shared ISR can only be written in one of them).However, suppose I write ISRs in both codes, but instead of just redirecting the interrupt vector I do this:Code:// in Bootloader code#define APP_INT (0x300+0x8)char isBootloader @ 0x05 = 1;// ...void interrupt blISR() {if (isBootloader) {// Do bootloader stuff} else asm("goto " ___mkstr(APP_INT));}Code:// Application codechar isBootloader @ 0x05 = 0;// ...void interrupt appISR() {// Do app stuff}

author: voltsandjolts
author_rank: Unknown
timestamp: November 10, 2016, 09:42:29 am
content: Maybe not applicable to your scenario but Mike discusses some interesting bootloader ideas here:(Broken link? You Tube keywords: mikeselectricstuff bootloader)

author: igendel
author_rank: Frequent Contributor
timestamp: November 10, 2016, 11:10:58 am
content: Quote from: voltsandjolts on November 10, 2016, 09:42:29 amGreat video, although as you said, it's not related specifically to PIC18F bootloader technique.

author: JPortici
author_rank: Super Contributor
timestamp: November 10, 2016, 11:34:06 am
content: Quote from: igendel on November 10, 2016, 09:25:19 amwhatever you end up doing, check the actual program memory at the interrupt vector and see if it does what you want. with the code i posted before the interrupt vector became GOTO 808h which was exactly what i wanted (you guessed it, the application interrupt vector is at address 808h)

author: Ian.M
author_rank: Super Contributor
timestamp: November 10, 2016, 11:39:13 am
content: The problem with interrupts and PIC bootloaders is preserving the context while still being able to jump to the application interrupt vector cleanly without excessive lag. If you write the interrupt dispatcher in C, there is a lot of overhead. It is possible to write it in assembler, and if you are careful to only use instructions that do not affect STATUS, WREG or BSR and put your flag in the Access bank, you can jump to the corrisponding vector in the main program if not in bootloader mode with minimal extra latency (only a few Tcy).However its far simpler and lower latency to simply place a GOTO at the interrupt vector and use a polling bootloader rather than an interrupt driven one.Seehttp://www.microchip.com/forums/m899466.aspxfor details of how to place assembler code of your choice at the interrupt vectors cleanly.

author: igendel
author_rank: Frequent Contributor
timestamp: November 10, 2016, 01:41:25 pm
content: That's not a bootloader, that's one deep rabbit hole...

author: JPortici
author_rank: Super Contributor
timestamp: November 10, 2016, 01:44:34 pm
content: QuoteAs usual, thank you Ian for pointing that out. I was doing something let's say unorthodox? that is indeed a much cleaner way to do the same thing, which magically freed up 300 program words that were being allocated and counted but not compiled (because i relocated the interrupt function outside the program memory.nowmybootloader is getting to a respectable size. not that i care too much, the unused boot space is wasted anyway

author: kony
author_rank: Regular Contributor
timestamp: November 10, 2016, 04:05:12 pm
content: Funny thing about (some of*) the 18F series is that fusebits are actually stored at the end of device FLASH instead of its own dedicated nonvolatile memory.This leads to dilema how to use flash write protection mechanism to protect bootloader - if memory protection is enabled starting from the start (0x0000) of the program flash, last page with CONFIGx bits is left vunerable to erase/overwrite wich might result in bricked device (and write protection configuration itself is stored there as well).If protection from the end of memory space is used, initial reset and IRQ vectors handlers are left vulnerable to overwrites (even worse so).Best solution so far seems to allocate bootloader on the top of the memory space and reserve bottommost page for single time programmable constants at the time of manufacturing and do a soft check for writepage adress inside the bootloader to protect data stored there against unallowed manipulation.*Currently working with 18F47J53 and 18F97J94's in some project.

author: JPortici
author_rank: Super Contributor
timestamp: November 10, 2016, 05:54:09 pm
content: care to elaborate?i should check a couple of datasheets but i'm pretty sure that read/write protection fuses can be reset only with the ICSP "erase all" command.regarding other things... for example setting the oscillator to an external source but no oscillator is actually attached, during startup the oscillator fail bit should be set en automatic switchover to internal oscillator should happen.

author: kony
author_rank: Regular Contributor
timestamp: November 10, 2016, 07:54:28 pm
content: Except when this automatic switchover is actually not broken (always check your errata).I'd be glad to get prooven otherwise, but so far it does not seem to be the case.Cited from PIC18F47J53 datasheet (DS39964B):QuoteAnd yes, this includes program memory readout protection bits.During flash write, write lock sequence mechanism is used (0x55; 0xAA write to EECON2) - which is the first safeguard against unintenional changes to the (program) memory contents, but inherently has to be disabled due to bootloader trying to achieve exactly this.IOLOCK and one-way lock for some config bits are valid only for RAM loaded values to serve as a safeguard agaisnt unintentional changes during runtime on the RAM registers stored values.EDIT:Thanks for stubble RTFM suggestion.After re-checking the datasheets of both devices troughly, I managed to find the missing bit (pun intended) in the CONFIGx registers.It is CONFIG4L -> WPCFG , wichmustbe cleared along with CONFIG4H -> WPDIS, otherwise all written above stands true.One should be cautious asthis bit invokes protection over whole last page of flash memory, not just the CONFIGx preset values. (This might make for some nasty bug in future, better to prohibit code placement in the last page in linker settings to be sure). So in the end all said previously - except for the soft-checks inside booltoader (when that particular bit is cleared) - holds true.

author: JPortici
author_rank: Super Contributor
timestamp: November 10, 2016, 08:08:55 pm
content: jesuswhy?whyyyyy?I only use K series today, never ever used the J series so i didn't know about that. Again, i may be forgetting about something but i will check tomorrow. it's night time"workaround" for the oscillator: in my programs the actual oscillator configuration is set between the first 10 instruction. in the bootloader i'm writing for example the very first instruction after the reset vector disable the interrupts. the second one select the FRC as the clock source.