Serial Message Decoding - Page 1
Forum section: electronics / projects

author: Jon Wilder
author_rank: Newbie
timestamp: October 25, 2017, 09:13:57 pm
content: Hi all. I'm using my Rigol DS1054Z scope (with all features hack enabled) to decode a serial output string from a bootloader host application. The purpose of the project is to update a product for a manufacturer, to include an updated PICmicro processor. Unfortunately they do not have the target bootloader source code nor do they have the source code for the host application, as the product is a few years out of date and the original developers are no longer with them.Upon analysis of the host application serial output, it is a very crudely written host application that simply reads out each line of the hex file to the serial port in ASCII. Each line of the file is terminated with a carriage return, and the target device sends back an "XOFF" message (0x13) to stop transmission. The target device must then convert the ASCII into the actual hex code it represents (like converting say ASCII "3A" (0x33 0x41) into a single byte value of hex 0x3A), then write the new hex values to a holding register. Once 64 bytes have been received and processed, it is all then written to flash ROM.Once the target device has completed processing the line of code (converting from ASCII to hex and writing to the flash memory holding registers), it then replies with ACK (0x06) followed by XON (0x11). The host application then sends the next line of the file. This sequence repeats until the target device detects the End Of File record.However...prior to the first line of the file being written, it sends some sort of ASCII string. It first sends "%c" (0x25 0x63). Then after a 768uS pause, it writes out "A19B5!C67#" (0x41 0x31 0x39 0x42 0x35 0x21 0x43 0x36 0x37 0x23). Following that, it then writes out the first line of the file and begins the sequence described above.If you try to send the file without the target device connected, the host application will make 5 attempts to send the "%cA19B5!C67#" sequence followed by the first line of the file, then wait for the XON, ACK, XOFF flow control/handshake messages. If no response from target, the host application sets a timeout error.This was decoded not only with the DS1054Z, but also with the Arduino IDE serial monitor using an Arduino UNO. Target device and host application communicate at 115.2Kbps over a USB-RS232 cable.In the attached photo, RxD is the host application sending to the target device, while TxD is the target device sending back to the host application.Has anyone ever seen this character sequence before? If so, can you decode what it means?

author: NivagSwerdna
author_rank: Super Contributor
timestamp: October 25, 2017, 09:20:21 pm
content: What is the product? Give us a clue.QuoteDoesn't sound very credible... or at best incompetent.

author: Leo Bodnar
author_rank: Frequent Contributor
timestamp: October 25, 2017, 09:33:35 pm
content: Why do you have red question marks in the trace?Leo

author: bsudbrink
author_rank: Frequent Contributor
timestamp: October 25, 2017, 09:56:29 pm
content: Total wild guess...%c <A bug in the code that was "adapted to" (note that %c is theC printf format string for transmit one character). It was originallyintended to send some other character, but "oh well".>A19B5 <four digit hex target address>! <separator character>C67 <count, count of blocks? count of lines? hex or decimal? (I haveseen really bad programmers mix hex and decimal in one message)># <ready to transmit>

author: Jon Wilder
author_rank: Newbie
timestamp: October 25, 2017, 09:58:33 pm
content: Quote from: NivagSwerdna on October 25, 2017, 09:20:21 pmI'm sure there's more of a backstory than that. I think the product was developed by a former company that this company acquired ownership of. Either way stuff got lost at some point along the way. I'm contracted to the company. Not part of them so their personal inside stuff is irrelevant.As to the red question marks...sometimes they appear even if it's correct. As I stated in my post, I also confirmed it with the Arduino UNO serial monitor and it yielded the same output.

author: Siwastaja
author_rank: Super Contributor
timestamp: October 26, 2017, 05:54:17 am
content: Rigol randomly adds those red question marks in decoded serial data, so it's meaningless.Agree that the hex blurb at the start could be offset (A19B5) + size (C67); I'd do that. Check it out.

author: NivagSwerdna
author_rank: Super Contributor
timestamp: October 29, 2017, 09:39:56 pm
content: If you can give some more details... e.g. PIC version it might help guesstimate the vintage and hence the possible boot loaders.

author: Jon Wilder
author_rank: Newbie
timestamp: October 30, 2017, 12:38:01 am
content: That is what I meant by the host application being a "crude" design. It does not care what the target device is. All it does is send the ASCII characters contained in a hex file from a serial port (or USB-serial cable), one line at a time, followed by a carriage return character. Upon receipt of the CR character, the target device sends an XOFF message (0x13, or ASCII Device Control 3) to the host application, then proceeds to process the packet.It converts the packet from ASCII to the hex code it represents (like converting an ASCII A (0x3A) to the value of 0x0A, verifies the checksum, then writes it to the TABLAT holding registers via tblwt* and tblrd*+ instructions. Once the line is stored and ready to write...or once 4 lines have been received and it has written them to flash, it then sends back ACK (ASCII 0x06), immediately followed by XON (ASCII 0x11, or Device Control 1). Upon receipt of XON, the host sends the next line of the file and the cycle repeats until the target has received the end of file record. ACK followed by XON is sent one last time upon receipt of the EOF record, at which time the target jumps out of the bootloader and runs the newly loaded main application.Prior to sending the first line of the selected file, it first sends %c, waits 768uS, then A19B5!C67#, followed by the first line of the file.If the target device is not connected and the host application has not received a handshake message from a target, it will make 5 attempts to send that character sequence followed by the first line of the file. Upon 5 unsuccessful attempts, it then times out and sets a communication error.This particular product's original target device is a PIC18F2525, It does this regardless of the product' target device as this utility is used with all of their devices.

author: abraxa
author_rank: Frequent Contributor
timestamp: October 30, 2017, 09:02:08 am
content: QuoteWhich parts are you "updating"? Does that equal rewriting? Do both ends of the communication need to remain compatible?QuoteYou could make a backup of the hex file and see if the header changes if you chop off some lines from the hex file.I'm also with everyone else here that the %c is likely to be a bug that the author didn't know about or didn't bother to fix. Looking at the string itself and knowing how egocentric some people can be (MZ, anyone?), I wouldn't be surprised if the header is simply A, B, C with the birth date of the developer: 19/05/1967. Of course I may be mistaken but it's something worth considering if you can't get the numbers to change somehow (or you can look at the executable of the host application and can see that the string is fixed).

author: NivagSwerdna
author_rank: Super Contributor
timestamp: October 30, 2017, 10:23:26 am
content: Quote from: Jon Wilder on October 30, 2017, 12:38:01 amThe reason I asked is that there are clearly two ends to this interaction... the sending device and the bootloader in the target. The target bootloader has two obvious implementations... plaintext and encrypted. In the plaintext form you would expect to see simply addresses and bytes... in the encrypted form you would potentially see a key exchange.Even in the plaintext exchange you may have a handshake that indicates say... version of sender software, version of firmware being sent, target supported etc.Do you have different firmware versions? What difference do you get when you load them? Which bits change?

author: RES
author_rank: Regular Contributor
timestamp: October 30, 2017, 12:52:08 pm
content: Could it be a UTF code?https://en.wikipedia.org/wiki/UTF-1https://en.wikipedia.org/wiki/UTF-8

author: Leo Bodnar
author_rank: Frequent Contributor
timestamp: October 30, 2017, 09:46:13 pm
content: Can you post the contents of the whole stream, end-to-end?Leo

author: Jon Wilder
author_rank: Newbie
timestamp: November 14, 2017, 03:41:23 am
content: So after much testing with their dashboard software, I was able to put two and two together. It's their own message protocol. When the host application sends to the target, it uses % as a start delimiter and # as an end delimiter. The character which follows the start delimiter differentiates the type of messages it is.%c is a connection request from the flash utility specifically.%< denotes serial open/close messages.%Z denotes connection acknowledged from the host.Clicking on the 'save' button in the dashboard starts with %A, followed by AxBx!Cxx#, with x being a value. A is the parameter number while B is the value to write to that parameter by the target. I'm guessing C is a checksum value, but I haven't been able to figure out exactly how the checksum is calculated yet.