121GW bluetooth protocol - Page 1
Forum section: electronics / testgear

author: nuclearcat
author_rank: Unknown
timestamp: January 12, 2018, 09:28:34 am
content: I prefer to create separate thread, as (hopefully) here we can discuss bluetooth protocol of 121GW.It looks like quite trivial to integrate to various software.As far as i see specification is not published, and honestly github C# sources are not clear(no comments whatsoever, but i am usual OSS/Linux fanboy,and C# is something i just dont like)So till now what i reached, i am subscribing over BLE, to indication, over handle 0x9This is what i get:n@n:~$ gatttool -b 88:6B:0F:41:0A:8F --char-write-req -a 0x0009 -n 0300 --listenCharacteristic value was written successfullyIndication handle = 0x0008 value: f2 31 37 30 38 30 30 30 30 30 30 32 30 30 30 32 45 34 30 36Indication handle = 0x0008 value: 31 32 30 30 30 30 30 30 30 30 31 36 34 30 30 30 30 36 31 32Indication handle = 0x0008 value: 30 30 30 30 30 30 30 30 30 30 30 30 42 32 0d 0a f2 31 37 30Indication handle = 0x0008 value: 38 30 30 30 0a f2 31 37 30 38 30 30 30 30 30 30 32 30 30 30Indication handle = 0x0008 value: 32 31 43 30 0a f2 31 37 30 38 30 30 30 30 30 30 32 30 30 30Indication handle = 0x0008 value: 32 31 43 30 0a f2 31 37 30 38 30 30 30 30 30 30 32 30 30 30Indication handle = 0x0008 value: 32 31 43 30 36 31 32 30 30 30 30 30 30 30 30 31 36 34 30 30As i suspect, BLE indication is limited to 20 bytes, and probably real data needs more, so it come over several BLE indication packets.Or as ASCII something like that (over simple python script):A notification was received: ?1708000000000000064A notification was received: 0100B30B000240000000A notification was received: 0000000000009F?170A notification was received: 8000?17080000000000A notification was received: 0006?17080000000000A notification was received: 000640100B30B0002400A notification was received: 0000?17080000000000A notification was received: 000640100B30B0002400Probably 0xF2 signaling start of data? (very likely according C# code)So string to decode is "17080000000000000640100B30B0002400000000000000000009F"I will try first:17,08,00000,0,00,Then according to source, first 2 bytes should be year:public int Year => DecimalNibbleToValue(0, 2) + 2000;I have 2018 set correctly, but it seems showing 2017?Next is month 2 - 08? august?Serial 00000 - well, no idea why zeroes, maybe not implemented yet.00 Voltage Low Z - which is correctI switched to voltage:17,08,00000,02,0001B8061200000000164000061And 02 is correct, "Voltage AC (V)", thats what i see on screen.So what you think should be done next? Maybe, i can do some easy to hack python script based on bluepy library.This will open bunch of opportunities to hook multimeter to raspberry, use it for triggering some specific events (like voltage threshold reached, and something should be done)Suggestions?

author: nuclearcat
author_rank: Unknown
timestamp: January 12, 2018, 11:07:52 am
content: With simple python script to accumulate data from notifications from 0xf2 to 0xf2, with fairly trivial script, here is what i got:It looks like crc errors?I dont exclude it is some python issues (usually i use C/c++/Go, but programming BLE with them is pain and not very cross-platform)Accumulated data(55): array('c', '17080000001000001640100BD00000E4000000000000000000096\r\n')Accumulated data(43): array('c', '17080000001000001640100BD00000E40000000000\n')Accumulated data(55): array('c', '17080000001000001640100BD00000E4000000000000000000096\r\n')Accumulated data(43): array('c', '17080000001000001640100BD00000E40000000000\n')Accumulated data(55): array('c', '1708000\x81001000001640100BD00000E4000000000000000000096\r\n') <-- this one for example 0x81 appeared in serial numberAccumulated data(43): array('c', '17080000001000001640100B"00000E40000000000\n')Accumulated data(55): array('c', '1708000000100000064&100BD00000E4000000000000000000097\r\n')Accumulated data(43): array('c', '17080000001000000640100BD00000E40000000000\n')Accumulated data(55): array('c', '17080000001000000640100BD00000E4000000000000000000097\r\n')Accumulated data(43): array('c', '17080000001000000640100BD00000E40000000000\n')Accumulated data(55): array('c', '17080000001000001640100BE00000E4000000000000000000095\r\n')Accumulated data(43): array('c', '17080000001000001640100BE00000E40000000000\n')

author: Iagash
author_rank: Regular Contributor
timestamp: January 12, 2018, 11:35:11 am
content: Hi,Quote from: nuclearcat on January 12, 2018, 11:07:52 amwow this is cool. I tried your gattool command and can also receive data from my 121gw.I just tried with my laptop but will look through my hardware box if I can find a BLE capable usb bluetooth receiver for my desktop machine.Could you add your python script to your post?

author: nuclearcat
author_rank: Unknown
timestamp: January 12, 2018, 11:38:36 am
content: It is still VEEERY early debug, i am just searching for a way to retrieve data reliably and then will start parsing itCode:#!/usr/bin/pythonfrom bluepy import btlefrom bluepy.btle import UUID, Peripheral,DefaultDelegateimport binasciiimport arrayfrom struct import *def parse_data(x):xlen = len(x)for offset in range(0,xlen):if (ord(x[offset:offset+1]) == 0xF2):# Process accumulated dataprint ("Accumulated data(%d): %s" % (len(parse_data.accumulated_data), str(parse_data.accumulated_data)))parse_data.accumulated_data = array.array('c')else:parse_data.accumulated_data.append(x[offset:offset+1])class MyDelegate(btle.DefaultDelegate):def __init__(self):btle.DefaultDelegate.__init__(self)def handleNotification(self, cHandle, data):parse_data(data)parse_data.accumulated_data = array.array('c')# Put your multimeter mac herep = btle.Peripheral("88:6B:XX:XX:XX:XX", btle.ADDR_TYPE_PUBLIC)p.setDelegate( MyDelegate() )print('Ready!')p.writeCharacteristic(0x9, "\x03\x00")while True:if p.waitForNotifications(1.0):continueprint("Waiting...")

author: Iagash
author_rank: Regular Contributor
timestamp: January 12, 2018, 02:50:35 pm
content: Thanks a lot, I found an usb-adapter that seems to support BLE and I got your script running after compiling and installing bluepy.I also get these strange escapes:Accumulated data(39): array('c', '17080000004\x0300000061200000100164000061\n')

author: nuclearcat
author_rank: Unknown
timestamp: January 12, 2018, 03:55:02 pm
content: Probably we need to wait official comments and documentation for protocol(ot at least ideas, why data is sometimes corrupted), then i can continue development.

author: hwti
author_rank: Contributor
timestamp: January 12, 2018, 04:22:46 pm
content: Using text doesn't seem very efficient !But even on BLE, 5 measurements (ie 100 bytes ?) by second should be doable, but it depends on the connection interval.

author: nuclearcat
author_rank: Unknown
timestamp: January 12, 2018, 04:29:08 pm
content: On my experience with BLE (but with NRF51), it can do much more.But in that case i developed a bit custom protocol, while in 121GW is bluegiga module, probably used in "serial cable extender" mode, i'm not very familiar with this module, and not even sure it is possible to upload custom firmware there.

author: Iagash
author_rank: Regular Contributor
timestamp: January 12, 2018, 05:17:12 pm
content: I played around with the gatttool a bit using:Code:gatttool -b 88:6B:0F:XX:XX:XX --char-write-req -a 0x0009 -n 0300 --listen | ./121gw.plThe perl script triggers on the 0xf2 like Dave2's app does. However the app calls the PacketProcessor with a length of 52.Code:private PacketProcessor MyProcessor = new PacketProcessor(0xF2, 52);I don't understand where the 52 is from. From gatttool I get most of the time 78 hex chars or 39 bytes until the next 0xf2 follows..The perl script is that one (sorry, I only know perl, yes I'm that old):Code:#!/usr/bin/perl$start = 0;while(<>) {if ( /^Indication handle = 0x0008 value: (.*)/ ) {@in = split(' ',$1);foreach $b (@in) {if ( $b eq "f2" ) {print "\n";$start = 1;} else {if ( $start == 1 ) {print "$b";}}}}}print "\n";The output looks like this:QuoteSo there seem to be incomplete records. You can also find flipped bits in the "complete" records.It looks like the BLE transmission is either inherently unreliable (I don't know anything about BLE) or the meter is sending garbage and incomplete records.In Posthttps://www.eevblog.com/forum/testgear/eevblog-121gw-multimeter-issues/msg1397729/#msg1397729I also quoted bluetooth logged VA values, which where way of.Did you find anything in the app that looks like a CRC or something else that makes sure the data is correct? I didn't find anything.If there is no data integrity check and the BLE protocol doesn't make sure the data arrives correct, then this bluetooth feature is completely useless.

author: hwti
author_rank: Contributor
timestamp: January 12, 2018, 05:57:13 pm
content: There should be a CRC on the lower layers.BLE indications are also acknowledged, so nothing should be lost.If the meter is more than 1.5m away from my laptop, the gattool command sometimes fails, and even when it works the connection drops after a few seconds.Perhaps the transmission power is very low (but it could be my laptop's fault too).

author: Iagash
author_rank: Regular Contributor
timestamp: January 12, 2018, 06:14:23 pm
content: Yes, I did some reading on BLE and the GATT profile. The lower layers should take care about transmission errors.This makes the variable length and "unexpected" bytes in the "records" even more suspicious. Unfortunately the 121gw is my only BLE device, so I can't test if I get correct data from other devices.I'd also be interested in the reasoning of sending the data basically ASCII coded when transmission speed is already low.

author: riyadh144
author_rank: Unknown
timestamp: January 12, 2018, 06:29:43 pm
content: That is pretty great work, I will start testing and playing with it once I get my meter.

author: beanflying
author_rank: Super Contributor
timestamp: January 14, 2018, 05:58:48 am
content: Using one of these ANT+ dongles I use on my indoor Trainer (yes I am a Lycra wearer too)https://www.ebay.com.au/itm/USB-ANT-Stick-for-Garmin-Forerunner-310XT-405-405CX-410-610-910-/232297420959?hash=item361601589fAdded it on the shack PC and it hooked up to the 121GW without a problem.One of the first things to do if possible now is to get access to the internal SD card so log files can be read, deleted etc!Log from the install"Driver Management concluded the process to install driver bthleenum.inf_x86_3bbedb3abba072b6\bthleenum.inf for Device Instance ID BTHLE\DEV_886B0F731C99\6&3338274&0&886B0F731C99 with the following status: 0x0."

author: EEVblog
author_rank: Unknown
timestamp: January 14, 2018, 08:26:10 am
content: Quote from: nuclearcat on January 12, 2018, 03:55:02 pmWe saw data corruption too, David dealt with it in software somehow, he may be able to clarify.

author: EEVblog
author_rank: Unknown
timestamp: January 14, 2018, 08:26:55 am
content: Quote from: beanflying on January 14, 2018, 05:58:48 amIt's not. The firmware has no support to do this that I know of.

author: EEVblog
author_rank: Unknown
timestamp: January 14, 2018, 08:28:54 am
content: Quote from: nuclearcat on January 12, 2018, 04:29:08 pmThe Bluegiga 112 module uses the standard Cable Replacement mode. There is an internal header to reprogram this is you really want to.

author: nuclearcat
author_rank: Unknown
timestamp: January 14, 2018, 08:50:44 am
content: Quote from: EEVblog on January 14, 2018, 08:28:54 amProbably if i will do anything, only if it is useful for rest of community. For myself - existing functionality is more than enough, just data corruption a bit irritating.Yes, it is solvable (by checking length of data, printability of characters in the middle of data chunk, etc), but such trivial issue should be fixed in firmware, if possible.I'm realistic about this subject as well, sometimes it is not possible, i have experience working with proprietary OEM blobs where vendor just don't care to fix such issues, and only way left - ugly workarounds.And about retrieving data from sdcard, it is for sure on stm32 and i believe it is definitely possible (unless bluegiga hooked on TX only), but again - this depends if someone might get luck with reverse engineering it(no idea how welcome it is), or get into some NDA and make alternative firmware.

author: EEVblog
author_rank: Unknown
timestamp: January 14, 2018, 08:56:01 am
content: Quote from: nuclearcat on January 14, 2018, 08:50:44 amIt was discussed early on and UEi said it was not easy to add, so we decided to leave it out, I would be stunned if it's been included.

author: nuclearcat
author_rank: Unknown
timestamp: January 14, 2018, 09:02:24 am
content: Quote from: EEVblog on January 14, 2018, 08:56:01 amThe only respectful reason, i believe, if they are out of resources on stm32Any chance data corruption will be solved in upcoming firmware releases?If not, i finish python examples with workarounds for this corruption.

author: EEVblog
author_rank: Unknown
timestamp: January 14, 2018, 09:22:26 am
content: Quote from: nuclearcat on January 14, 2018, 09:02:24 amReport it on the Issues thread and it'll go on the list.

author: Kean
author_rank: Unknown
timestamp: January 14, 2018, 06:02:03 pm
content: I wonder if the data corruption mentioned above is a slight mismatch in baud rate between STM32 and Bluegiga module. I'm guessing they're both using crystal oscillators.Alternatively, is there any circuitry between the STM32 serial transmit and the Bluegiga receive - e.g. level converter?

author: savageautomate
author_rank: Unknown
timestamp: January 15, 2018, 07:43:38 pm
content: I have written a sample NodeJS (Javascript) program that can parse and decode the 121GW data protocol if anyone is interested.I gathered most of the info needed from David's C# sources.I also have a rough prototype working on an ESP32 board (http://amzn.to/2mydmoq) that can capture the BLE packets and forward them to the USB connected serial port. The BLE implementation is still pretty wonky and the platform is a bit unstable. I'm really looking to implement this logic on an Adafruit nRF52 board (http://amzn.to/2FECcvP) as its much more stable but I'm waiting for a bug to be addressed (https://github.com/adafruit/Adafruit_nRF52_Arduino/issues/113) before I can move forward with it.Thanks, Robert

author: Iagash
author_rank: Regular Contributor
timestamp: January 15, 2018, 08:45:15 pm
content: Here is the further developed version of my simple perl-script. It now parses the output of gatttool and displays it in readable form.You should rename the file from121gw.txtto121gw.pl.Then you can run it via:gatttool -b 88:6B:0F:73:XX:XX --char-write-req -a 0x0009 -n 0300 --listen | ./121gw.plIt reads the output of gatttool and parses the packet the multimeter sends.It detects a lot of error but not all of them, so it's something to play with but until the bluetooth transmission is fixed in the firmware I would not rely on the data.If the meter is in VA mode most of the data is wrong. This can't be fixed in the script and is also the case in the Android app.The output looks like this:QuoteAt the beginning is a unix timestamp when the record was received.The year and month is always the same since the meter doesn't seem to send the current one.The third column is the "serial number" of the meter. You can set this value on the meter by cycling through the setup until you reach the display with00000. If you long pressSETUPyou can change the digits to some other number. This number is send by the meter in every bluetooth frame.

author: amspire
author_rank: Super Contributor
timestamp: January 15, 2018, 10:23:00 pm
content: Quote from: Iagash on January 15, 2018, 08:45:15 pmThank you. I have been wondering what that number is for.

author: Iagash
author_rank: Regular Contributor
timestamp: January 16, 2018, 08:43:09 am
content: Hi,Quote from: savageautomate on January 15, 2018, 07:43:38 pmthat's were I also gathered my information for the script.But did you find a way to get correct readings in VA mode?In VA mode I get every now an then a correct reading but most of the readings are way off, but not invalid in the packet data..

author: savageautomate
author_rank: Unknown
timestamp: January 18, 2018, 10:47:24 pm
content: Quote from: Iagash on January 16, 2018, 08:43:09 amNo .. I'm seeing the same thing you are .. wildly erratic values for VA readings.I'll have to look again, but I think also some incorrect values for secondary measurements too while on a VA mode.

author: nuclearcat
author_rank: Unknown
timestamp: January 28, 2018, 10:07:30 pm
content: Quote from: Iagash on January 15, 2018, 08:45:15 pmHere is some PoC i created (unfortunately no sources yet, as its too ugly and mostly just PoC, it is linux specific, C++), idea that might be handy for EE:Measuring resistance in auto range mode, it is a bit slow to get to correct range.https://www.youtube.com/watch?v=7mnrvE271fA&feature=youtu.beI used cheapo chinese microscope ($70) and, sure, 121GWNot sure if worth to continue this app.

author: benst
author_rank: Regular Contributor
timestamp: January 29, 2018, 11:32:01 pm
content: Quote from: nuclearcat on January 28, 2018, 10:07:30 pmThat's a great idea!Ben

author: prof
author_rank: Contributor
timestamp: February 01, 2018, 11:41:48 pm
content: I tried analysing the protocol with the "Bluetooth Explorer.app" on the Mac a while back but didn't have a lot of success because the format is rather weird (as you've figured out) and the values are extremely erratic. Glad you figure out at least that much!

author: nuclearcat
author_rank: Unknown
timestamp: February 01, 2018, 11:45:13 pm
content: I'm waiting firmware with a bit more reliable data(or statement that it is going to stay like this and best way to validate data), or if there is some successful efforts to make custom stm32 firmware.Unfortunately i dont have much time to spend on reverse engineering or making workarounds that wont be needed in future.Then i might continue on video app with OSD.

author: kgmuzungu
author_rank: Newbie
timestamp: December 08, 2019, 06:53:45 pm
content: Hi,did someone get it running? I mean to directly read from the bluetooth (COM) port and log to a file? If you have some code, also if buggy, can you post or link it?thanks