Firmware for PIC16F877A woes and worries.. - Page 1
Forum section: electronics / general

author: R-M-I
author_rank: Contributor
timestamp: January 11, 2013, 04:16:25 pm
content: Hi there..I've been tinkering with a project for a while now..I have some circuit-boards for some hobby intems here that I'm trying to figure out a way to firmware-Flash.I have a PicKit3 and have made a connector that goes from the ICSP to the boards update-connector.I also have the BIN files from the manufacturer, aswell as a custom firmwareflasher-application from the same manufacturer.. The problem is that the application will not detect the PicKit as a programmer..Neither can I load the BIN-files directly into MpLab as it will need HEX-files to work with the board..A direct bin2hex conversion won't work I assume.Is there a way I can:1. Somehow use my PicKit with this program? (The original programmer is a USB-to-serial programmer(FTDI D2xx).2. Extract/convert the BIN-files into hex-files?3. Burn BIN-files directly to the PIC using my current programmer?I'm sure I've left things out here, if I'm unclear on something please let me know and I'll try my best to clarify..

author: R-M-I
author_rank: Contributor
timestamp: January 11, 2013, 09:09:52 pm
content: Is there a way to run the pickit3 with a virtual serial port/com port or something?

author: ptricks
author_rank: Frequent Contributor
timestamp: January 11, 2013, 10:17:40 pm
content: copy the .bin file and save it as .hex then try loading it with the pickit. If it doesn't work the part can still be flashed again when you get the right file, PIC firmware flashing is a hardware feature when using the pickit so it is next to impossible to make a chip unusable .

author: AlfBaz
author_rank: Super Contributor
timestamp: January 11, 2013, 10:32:43 pm
content: If the bin file is the actual binary code, then you should be able to import it into mplab and view it as asm code. You can then export it in the appropriate hex file format or simply write it to the pic chip directly. Unfortunately I don't have MPLAB installed right now so I cant give you details

author: R-M-I
author_rank: Contributor
timestamp: January 11, 2013, 10:33:59 pm
content: That won't work at all..Neither will converting the BIN to hex via. BIN2HEX.I'm just really wondering what the firmwareflasher-app does.. All you do there is open the program, load the BIN with the firmware you want to Flash, hold down the om/off button on the circuit-board and press the program-key..That conversion is what I don't understand.. Why can that program load a BIN file onto the PIC just like that and not MPLab for instance?I would hate it if I needed to buy a FTDI USB/UART /serial-thing to make that work.. I mean, it should be quite possible to program it via. MPLab and the pickit..Am I Missing something here?EDIT:To AlfBaz, thanks.. I'll need to check again I guess..Simply clicking import and choosing the BIN will not work.. So I assume I need to open it in debuging mode or something? I'm a bit lost here..

author: AlfBaz
author_rank: Super Contributor
timestamp: January 11, 2013, 10:46:05 pm
content: Yes the intel hex file format is an ascii representation of the binary code. It includes on each line the address where the procedding code goes, the type of data that is about to follow and a check sum at the end of the line. Where as a bin file is pretty much a straight binary dump of the flash contents of the chipI'm downloading the latest version of mplab now (wow they really have wound back development of MPLAB, last version I D/L's about 2yrs ago was 8.63, new vers is 8.89! looks like mplabX is where the development is). I'll have a quick look and let you know

author: R-M-I
author_rank: Contributor
timestamp: January 11, 2013, 10:56:57 pm
content: Thank you

author: AlfBaz
author_rank: Super Contributor
timestamp: January 11, 2013, 11:23:41 pm
content: Can you upload one of the bin files? I don't have any on hand.I'm fairly sure it can be done, its just been a long time

author: R-M-I
author_rank: Contributor
timestamp: January 12, 2013, 07:02:03 am
content: I'll uploaded it as soon as I get home. Should be shortly.

author: R-M-I
author_rank: Contributor
timestamp: January 12, 2013, 08:27:22 am
content: Here's one of the bin-files.. Needed to rename the extension from BIN to HEX to be able to attach it in here for some reason.

author: R-M-I
author_rank: Contributor
timestamp: January 12, 2013, 09:53:29 pm
content: Anybody? :/Could it be that i need to be using a usb->UART cable to get the binary onto the PIC?

author: Unknown
author_rank: Unknown
timestamp: January 12, 2013, 10:40:28 pm
content: http://tinypixels.co.uk/projects/misc/110_real.hexDoes that hex file work now?

author: R-M-I
author_rank: Contributor
timestamp: January 12, 2013, 10:42:30 pm
content: I'll try it in about 40 minutes When I get back to my modest workshopThanks either way for trying

author: R-M-I
author_rank: Contributor
timestamp: January 12, 2013, 11:38:07 pm
content: Didn't work I'm affraid...Board does not light up as it should :/

author: AlfBaz
author_rank: Super Contributor
timestamp: January 13, 2013, 12:40:48 am
content: Trying this now, I don't remember this being that difficult, so maybe I rememebred the wrong thing...I'll look a little further for you.Just to be clear, your end goal is to program that bin file into another chip, and you are sure that the bin file was meant for a 16F877A?

author: R-M-I
author_rank: Contributor
timestamp: January 13, 2013, 07:32:03 am
content: Yes.. I'm 100% sure it's for the PIC16F877A.

author: amyk
author_rank: Super Contributor
timestamp: January 13, 2013, 11:02:30 am
content: It's either compressed or encrypted. The entropy is too high for a raw memory dump.As the chip supports self-programming and you mention a process that looks very different from regular ICSP, that may be how it is done. You'll need to do a little more investigation, starting with capturing the communication between the PC and the board when you flash the firmware. It might be bootloaded like an Arduino.

author: Skimask
author_rank: Super Contributor
timestamp: January 13, 2013, 11:26:44 am
content: IF in fact that file was for the 16F877A, it has no embedded config bits, no viable reset vector, no viable interrupt vector, and the logical flow of the file, when loaded into MPLAB and you look at the dis-assembly makes no sense at all.It would REALLY help a LOT if we knew what the device was that this 'file' was being loaded into.

author: R-M-I
author_rank: Contributor
timestamp: January 13, 2013, 11:43:31 am
content: Hmm..Again the file IS for a PIC16F877A...I know this because I've flashed several of these boards when I owned the original firmware-flasher... (I don't anymore)..I think the original programmer used was a FTDI 232\245BM cable\module.. USB in one end.. a small box with the FTDI chips in the middle, with a RJ11-port on the other end.I think it used the D2XX drivers from FTDI aswell.IF I get a device that will work via. USB\serial like this using a virtual com-port, I MIGHT be able to use the original firmware update utility (which I still have)..I'll admit that I'm no expert on this topic, but I am still VERY interested in learning about microcontrollers, how things interact and work..As I've always beenCould anyone possibly direct me towards an item on eBay, that would help me be able to interface with the board in such a manner as described above? I love experimenting with this stuffAlso:I REALLY APPRECIATE THE FEW PEOPLE THAT ACTUALLY BOTHERS TO TAKE THE TIME TO TRY TO HELP OUTI just had to type that out in capital letters there.

author: notsob
author_rank: Frequent Contributor
timestamp: January 13, 2013, 11:58:14 am
content: Sparkfun have some FTDI productsthis may be interestinghttps://www.sparkfun.com/products/9717?

author: Skimask
author_rank: Super Contributor
timestamp: January 13, 2013, 12:10:14 pm
content: Quote from: R-M-I on January 13, 2013, 11:43:31 amIf that's the case, then I would venture to guess that what you are trying to do is take that firmware and using the PICKIT3, reprogram the ENTIRE memory region of the 16F877A, whereas the original setup likely used a bootloader in the '877 in the machine, which then loaded this file (and possibly did some decryption or descrambling of the data) into a different region of memory while leaving the original bootloader area untouched.And if you tried to reprogram the '877A with the PICKIT3 and the config bits were originally set with the code protect bits enabled, the only way to write to the '877A is to erase the entire thing first, which would wipe out the bootloader, thereby rendering it useless to you in that configuration. Therefore, you likely have to re-write the entire PIC itself to regain it's former use.If you have the PICKIT3 (or PICKIT2), AND can get to another working device, AND are lucky enough that the code protect bits haven't been set, you could probably use the PICKIT3 to read out the entire PIC memory, including the config bits, ID locations, etc.Even if the code protect bits are set, the '877A chips are old enough that there are a few hacks out there for getting the data out of it. I personally haven't done any of them. Some of them look a bit hokey, some of them look genuinely doable. Up to you if you want to try to go that route.Also, how many pins are on that RJ-11 connector? 4 or 6? ...clarification needed... How many pins are actually USED on that RJ-11 connector?

author: R-M-I
author_rank: Contributor
timestamp: January 13, 2013, 12:17:32 pm
content: There's 5 pins on update-port on the board..so I figure that's what goes out of the RJ11 connector

author: Skimask
author_rank: Super Contributor
timestamp: January 13, 2013, 12:43:56 pm
content: Could be USB with an extra ground, could be an actual PICKIT2 port. Tough to tell now. I was kinda hoping you would've said 4 pins 'cause that would've meant that the interface was almost surely not a PICKIT type interface.I'd say probe around the board. Figure out which pins on the RJ-11 go to which pins on the PIC. That'll tell a lot about what's happening.

author: R-M-I
author_rank: Contributor
timestamp: January 13, 2013, 12:51:52 pm
content: I know exactly which pin in the update-port on the board goes to what pin on the PIC.I'll put up a the info later today.

author: Skimask
author_rank: Super Contributor
timestamp: January 13, 2013, 01:03:40 pm
content: On the PIC itself, if those RJ-11 pins go to pins 1, 39, 40, and the Vdd/Vss pins (middle of the PIC, 10/11 and 20/21 I think), then most likely the RJ-11 jack gets connected to a programmer.The programmer interface MUST use at least pin #1 (MCLR), #39 (PGC), and $40 (PGD), if it's going to be programmed by a PIC programmer. The power pins could go to other things, filters, buffers, regulators, something, but they don't NEED to go directly to the PIC, 'cause it might have it's own power supply while it's being programmed.Any other pins (such as the PIC's onboard serial port on pin #18 and #19 I think) and it's going to be a custom serial interface of some sort.

author: R-M-I
author_rank: Contributor
timestamp: January 13, 2013, 01:25:35 pm
content: If I didn't mess up when checking continuity from the legs on the PIC and the updateport-pins..this is the pins that are in use by the programmer:PIN 7 - VDDPIN 13 - NCPIN 16 - PGCPIN 18 - MCLR\VPPPIN 32 - OSC1/CLKIBut...from my previous documentation I think this looks more right (that these are the pins in use).vss groundmclr Vpppgcpgdvdd targetEDIT: I'll check again when the kids go to sleep tonight..I have access to more identical devices aswell as some similar running different chips.The code-protection-bits ARE set on the device I'm trying to get working now. If it wasn't I guess it'd be a extremely easy task to get usable hex just by reading it out in MPLab and exporting it to file. Then program it to another device. Unfortunately this is not an option for me.I am lucky enough to have:- The original firmware or bootloader-files or whatever the hell it is that I have (the BIN-files with the various firmwares).- The original firmware update utility for the boards I want to update.I also know the type of programmer that was originally used, but I don't have it myself.I'll include info on that aswell later today..And again, thank you so much for trying everyone..

author: Skimask
author_rank: Super Contributor
timestamp: January 13, 2013, 05:38:29 pm
content: Quote from: R-M-I on January 13, 2013, 01:25:35 pmThat would likely mean that an actual programmer is in use vs a bootloader.QuoteIf those cases hold true, then you shouldn't have any problem...unless...the programmer being used had a specific modification, along with the target board having a specific modification, which would disallow any standard programmers to work.K, bootloader vs firmware vs whatever-the-hell...a quicky...If you've got a full copy of the firmware, you've got the whole thing.It's kinda like having a "dumbass" in the next room, and you have to give him a task to do. Problem is, he's a "complete dumbass", so you gotta tell him step-by-step-by-freeking-step every little bitty thing to do.If you've got a firmware that requires a bootloader, you've only got a part of the whole thing.Using the same analogy, it's kinda like having a smart kid from next door. He knows how to do stuff, just needs to know the title of what needs to be done.Similarly (and this is one scenario of many), if you've got a bootloader and firmware (the program that normally runs) already programmed into the PIC, upon power-up, this bootloader will wait X number of seconds for a specific sequence to come across some set of pins...whether it's something down the serial port, or a certain pin combination, or whatever, it'll wait for it. If it sees this sequence, it'll jump to a different mode which will overwrite the main program with whatever program comes down the lines. When it's finished writing that code, it starts it up.If you've got firmware without a bootloader, it starts up and runs. That's it. It doesn't necessarily wait for anything to come along and change it...It just runs.If you've got a PIC that HAD a bootloader and firmware, but you overwrote the bootloader and/or firmware with something else, the "programmer" may or may not know about this (sounds like it doesn't) and just go duhhhhhhhhhhhhhh.......and it's game over.A bootloader is just a small subset of the complete firmware that makes other stuff happen independently of the main program.You said you were using a PICKIT3?Some people swear by it, some swear at it. I used to swear at it loudly. Since the newer firmware for the PICKIT3 has come out, I still swear at it, but not as much.If you can, I'd try to get your hands on a PICKIT2. It was around "back in the day" when the 16F877A was more popular and might be a bit more adept at programming it. Sounds almost stupid I know, but that's the way it is.It would still help if we knew WHAT this magical piece of hardware you are dealing with is...for that matter, anything else.This cloak and dagger stuff ain't the way this game is played. It ain't my job to figure out what you're trying to do. It's your job to feed us any information you can about what you're trying to do so we can best help you.

author: R-M-I
author_rank: Contributor
timestamp: January 13, 2013, 06:28:00 pm
content: Here's some info on the original programmer:Just to give you a picture of how it looks, here are twoI cracked the box open to check what was inside (ofcourse):the chips in there were these:------------FTDIFT232BM0233------------ATMELATMEGA816AI 0444I--------------93LC46BI/SNAs you can see from the picture, there's one side with a RJ11 connector on it.. going from that to the plug that fits into the circuitboard I'm working with getting up and running againthe other side is obviously the USB-cable that goes into the computer.In addition to this, I needed to install the FTDI D2XX drivers to get it working.I also seem to remember there were some virtual COM-ports in the mix here, but guessing they come with the drivers?

author: Skimask
author_rank: Super Contributor
timestamp: January 14, 2013, 12:23:34 am
content: www.ftdichip.comfor the FTDI drivers.The FT232BM is an older chip, but it's still used.You've got a USB-serial converter, an ATMega MCU, and an eeprom in that programmer.No telling what it actually does to the data going into and out of it, at least not without doing a bunch of probing, datalogging, and so on....

author: ptricks
author_rank: Frequent Contributor
timestamp: January 14, 2013, 03:24:47 am
content: What is the target chip , the pic, used for ? Is it just a demo board , what task is the pic supposed to be performing ?The last time I saw a programmer like that was one of the ones that used a micro in the programmer to read the bin file and also in that file were the parameters of the target device so it would know how to write the actual data into the chip. Usually if you look at the .bin file in a hex editor you can find the part where the configuration data for the programmer starts and stops and where the programming data begins, lot of 00 00 00 stuff or other repeating characters.

author: amyk
author_rank: Super Contributor
timestamp: January 14, 2013, 06:41:10 am
content: It looks like a somewhat intelligent programmer, the FT232/Atmega8 combination is used in AVR programmers too. If you can get a logic analyser on the pins and log at least the start of the programming process you can figure out whether or not it's the standard PIC programming protocol. Otherwise the hope is that the software running on the PC is what does the decryption/decompression of the .bin file and you can try disassembling that and analysing it (rather steep learning curve if you're not familiar with RE already...)QuoteThere is a bit of a header in the file posted above, but the bulk of it doesn't have the characteristic texture of machine instructions.

author: R-M-I
author_rank: Contributor
timestamp: January 15, 2013, 11:18:01 am
content: I've ran the bin-file in with magic\hexwalk..And it identifies it as "G3 data, byte-padded"I'm also trying to look through the FUU(Firmware Upload Utility) in IDA Pro. I need to be pulling in some people more knowledgeable about reverse engineering\ASM and what to look for than myself on this one though.. Hopefully I'll get it working in the end

author: JTR
author_rank: Regular Contributor
timestamp: January 18, 2013, 06:15:56 am
content: The Bin file is encrypted. You aint going to have any luck with any asm experts until you figure out the rather simple encryption method used. Really, just look at it. It is begging to be broken...

author: AlfBaz
author_rank: Super Contributor
timestamp: January 18, 2013, 10:41:15 am
content: I swear I had nothing to do with this

author: R-M-I
author_rank: Contributor
timestamp: January 19, 2013, 08:06:29 pm
content: THE PLOT THICKENS!! *insert dramatic chipmunk here*

author: JoannaK
author_rank: Frequent Contributor
timestamp: January 19, 2013, 11:06:58 pm
content: For a old hack, that binary file looks real odd. Obviously this is not any 14-bit wide Pic code, and most of it seems way too repeating to be any code at all.. And for even to be graphical/audio data.. Not likely, cause this is just a small pic.Code:000015B0 410C 4FCA 3D68 0BE6 B944 4782 B5A0 039E A.OÊ=h.æ¹DG‚µ .ž000015C0 317C 3F3A 2DD8 FB56 A9B4 37F2 A510 F30E 1|?:-ØûV©´7ò¥.ó.000015D0 21EC 2FAA 1D48 EBC6 9924 2762 9580 E37E !ì/ª.HëÆ™$'b•€ã~000015E0 115C 1F1A 0DB8 DB36 8994 17D2 85F0 D3EE .\...¸Û6‰”.Ò…ðÓî000015F0 01CC 0F8A FD28 CBA6 7904 0742 7560 C35E .Ì.Šý(Ë¦y..Bu`Ã^00001600 F13C FFFA ED98 BB16 6974 F7B2 65D0 B3CE ñ<ÿúí˜».it÷²eÐ³Î00001610 E1AC EF6A DD08 AB86 59E4 E722 5540 A33E á¬ïjÝ.«†Yäç"U@£>00001620 D11C DFDA CD78 9BF6 4954 D792 45B0 93AE Ñ.ßÚÍx›öIT×’E°“®00001630 C18C CF4A BDE8 8B66 39C4 C702 3520 831E ÁŒÏJ½è‹f9ÄÇ.5 ƒ.00001640 B1FC BFBA AD58 7BD6 2934 B772 2590 738E ±ü¿º.X{Ö)4·r%.sŽ00001650 A16C AF2A 9DC8 6B46 19A4 A7E2 1500 63FE ¡l¯*.ÈkF.¤§â..cþ00001660 91DC 9F9A 8D38 5BB6 0914 9752 0570 536E ‘ÜŸš.8[¶..—R.pSn00001670 814C 8F0A 7DA8 4B26 F984 87C2 F5E0 43DE .L..}¨K&ù„‡ÂõàCÞ00001680 2D43 31B5 16E7 E761 9519 8BDA 97AF 43B0 -C1µ.çça•.‹Ú—¯C°00001690 1DC5 9302 2E64 D8ED A589 9B4A 2637 5C41 .Å“..dØí¥‰›J&7\A000016A0 2D71 A3B6 110D 6083 E51B ACEF 39D2 EFFA -q£¶..`ƒå.¬ï9Òïú000016B0 34F2 B835 3D68 0BE6 B944 4782 B5A0 039E 4ò¸5=h.æ¹DG‚µ .ž000016C0 317C 3F3A 2DD8 FB56 A9B4 37F2 A510 F30E 1|?:-ØûV©´7ò¥.ó.000016D0 21EC 2FAA 1D48 EBC6 9924 2762 9580 E37E !ì/ª.HëÆ™$'b•€ã~000016E0 115C 1F1A 0DB8 DB36 8994 17D2 85F0 D3EE .\...¸Û6‰”.Ò…ðÓî000016F0 01CC 0F8A FD28 CBA6 7904 0742 7560 C35E .Ì.Šý(Ë¦y..Bu`Ã^00001700 F13C FFFA ED98 BB16 6974 F7B2 65D0 B3CE ñ<ÿúí˜».it÷²eÐ³ÎIn the middle of this hex list there are couple lines that obviously look different. Most likely those are real code/data and the rest is just Blank rom with XOR:ed with some pseudorandom pattern.

author: R-M-I
author_rank: Contributor
timestamp: January 19, 2013, 11:22:02 pm
content: I know for a fact that the file works perfectly when loaded to the PIC vis the usb->serial device through the firmware uploaded utility that went with it..The file has be obfuscuated somehow though.. Compression? Encryption?I again thank you all for your input.. Know that I am thankful for it.

author: AlfBaz
author_rank: Super Contributor
timestamp: January 20, 2013, 12:23:37 am
content: Do you have any company/manufacturers names for the original programer and software or for who built the boards?Even if they don't have any technical details on their programming methods, the marketing spiel may give up some clues as to what format the bin file is

author: R-M-I
author_rank: Contributor
timestamp: January 20, 2013, 03:33:31 pm
content: I believe the programmer was made by NEO Electronics LTD in the Uk.That's what came up in my device-manager when I checked it on my computer atleast..I'm building a new programmer now.. USB/Serial using FTDI's RS232TL chip.Hoping I'll be able to communicate with the original firmware-utility with this oneupdating when/if there are news..If someone has any more pointers I'm gladly accepting them..