yashsingh1/xdaforums huggingface dataset:
[General Accessories] wachino:
Im not getting that stranges values, but I have wrong timedates, I will try to study it.
[General Accessories] kueters:
Hi.

The result of the raw data extraction is very strange. Can someone check my data. I attached as example data from yesterday.

Attachement data_as_txt.rar contains the following files:

DATE_DATA - data.txt (data from table DATE_DATA field data)

DATE_DATA - summary.txt (data from table DATE_DATA field summary -> looks good)

extract_raw_hr.txt (result of raw data extraction)

extract_raw_summary_hr2.txt (summary of the extracted raw data)

Attachement test_cmd_files.rar contains my testfiles for local processing

miband2.sql

miband2_raw.sql

run_lokal2.bat

THX for your investigation and please post some example data from your databases.
[General Accessories] Sauve:
No CSV file generated

Hi!

I was trying to use your tool to extract data from my Mi Band 2 from a non-rooted phone to Windows 7. However I keep getting the following error in the log file:

Extract Started

di 20-09-2016

Renaming

ADB SU copy to sdcard

ADB pull

Cannot find database files. Non-rooted phone? Attemting backup approach

unpacking backup file

miniz.c version: 9.1.15

Mode: d, Level: 9

Input File: "mi.zlb"

Output File: "mi.tar"

Input file size: 17

Total input bytes: 17

Total output bytes: 1024

Success.

bin\tar: Record size = 2 blocks

bin\tar: apps/com.xiaomi.hm.health/db/origin_db*: Not found in archive

bin\tar: Error exit delayed from previous errors

bin\tar: Record size = 2 blocks

bin\tar: apps/com.xiaomi.hm.health/db/mihealth*: Not found in archive

bin\tar: Error exit delayed from previous errors

deleting temp files

The system cannot find the path specified.

The system cannot find the path specified.

The system cannot find the path specified.

Still cannot find files. Restoring original files

The connection with the phone seems all right as I receive a backup request, though after confirming this I don't get a CSV file on my laptop.

Could anyone help me with this problem?

Thanks!
[General Accessories] rubenix:
hello there, I'm trying to get this to work with no success.

Rooted Redmi 3 Pro Mifit 2.2.1 & Miband 2 firm ver 1.0.0.44

Connection succesful to PC but I think database names changed on recent releases.

here my output:

* daemon started successfully *

cp: /data/data/com.xiaomi.hm.health/databases/mihealth*: No such file or directory

2533 KB/s (77824 bytes in 0.030s)

1135 KB/s (8720 bytes in 0.007s)

remote object '/sdcard/mihealth.db' does not exist

remote object '/sdcard/mihealth.db-journal' does not exist

In fact in recent mifit releases db in directory are mistat.db & sports.db --> I think code needs some modification to work

Can someone point me to the right direction? Thanks so much in advance!
[General Accessories] Dave_Scream:
Your tables works for me! Thanks.

Im use Mi Fit 2.2.1 + Mi-Band 2 - File names are changed.

my directory listing:

There is no "mihealth_db" - what data was in that file in prev. versions?

And what data is in mistat.db now?

Here is my stats using kueters miband2.sql and miband2_raw.sql yadi.sk/d/RMHZA1O_vvUMV

----

Does anyone have problem with steps counter. My Miband 2 shows bigger amount than Mi Fit day stats. Day stats often skip periods, so 15-20% are loose. Can it be because of bad connection? Anyone have problem with it?
[General Accessories] Dave_Scream:
there is my working version, using new miband2.sql from previous posts. Checked on Mi Fit 2.2.1 + MiBand-2

drive.google.com/open?id=0B2Qb94YlMoWSY1dFb3YyQkNVeEU

In run_NEW.bat you need change following strings:

set SDPath=/storage/sdcard0
set origin_db=origin_db_1610064220

SDPath is path for temporary folder on your phone.

origin_db_1610064220 - here you need specify number of my-fit profile. You can look it in Mi Fit -> Profile -> My Personal Info -> ID: 1610064220

Your phone must have access with adb.

run bin\adb.exe devices

here you phone must persist and "authorized". status If unauthorized then you need to go developer setting in phone turn on usb debugging and "revoke previous accepts". It it not help, run to your windows profile, and try delete (rename) ".android" folder. Then your phone must start asking you to give permission to access with adb. If not, try this

Every another try kill server first because need restart: adb.exe kill-server.

Raw data exports looks bad.
[General Accessories] vobrok:
Hi.

On linux, I was able to export the data by combining the scripts from guys above by these commands (quick and dirty):

adb shell # enter adb shell
su root # get root
cp -r /data/data/com.xiaomi.hm.health/databases /sdcard/Download/databases
# exit the adb mobile shell
adb pull /sdcard/Download/databases db/ # i'm in the root of the MiBand2 archive from the previous page
cd db
cp databases/* .
sqlite3 origin_db_1588231393 < miband2.sql # the number will correspond to your UID
echo "INSERT INTO _PersonParams (Height,Weight) VALUES (170, 50);" > health.sql # replace 170, 50 by your height, weight
# now edit line number 3 in miband2_raw.sql from ".read db/health.sql" to ".read health.sql"
sqlite3 ./origin_db_1588231393 < miband2_raw.sql # again your UID, not this one

and the result is this:

https://www.dropbox.com/s/0zfovq49q6t1zxj/exported_files.zip?dl=0
[General Accessories] hydeah:
When I asked someone if I can use a desktop application to extract data from Mi Band, they pointed me to this thread, but I don't see any windows application listed here. Is it possible to extract data from the band?
[General Accessories] Dave_Scream:
Try my version for Windows.

As mentioned before, to use it you need first edit "extract_data.bat" file with notepad:

set origin_db=origin_db_1610064220 - set your ID here. Go MiFit -> Profile -> My Personal Info -> ID: 1610064220
set SDPathTemp=/storage/sdcard0 - set your path to SDCard

Also I add some csv imports to Access Database and then to Excel for analysis. Try use it

1. extract data from your miband, so _extract folder have .csv files

2. run with Microsoft Access _miband_analytics_v560\excel\Database.accdb

3. In Access go "External Data" and Linked Tables Daemon

4. Check "Mi-Band2 extract" table and click OK then specify your path to _extract\extract2.csv

5. Open in Excel _miband_analytics_v560\excel\Excel analysis.xlsx

6. Data -> Refresh - it will ask you to update path to _miband_analytics_v560\excel\Database.accdb

Analyze your data with excel and share useful excel analysis methods.

Raw extraction give bad results. Who can repair it?
[General Accessories] u363840:
Hi,

for non-rooted users (you may have reasons for this), here's a modified version of MiFit app v2.1.9 (current latest play store version) :

AndroidManifest.xml modified to allow backup (thanks Pridexs for the information)

PackageName changed to com.xiaomi.hm.health.backup so you can install it along with the original version and so you don't have to uninstall original version (if you're afraid to loose data for example).

Link : https://mega.nz/#!188iHT6D!vx5NuoqOllABmyX0tUvMF-cl1f3YQcjcCLQmmYIUv5Y

Summary of the steps to create APK (adapted from https://www.youtube.com/watch?v=-0j6DLi3IPA) :

retreive apk from apkmirror

unpack using apktool

modify AndroidManifest, setting allowbackup to true and removing the 2 permissions with android:protectionLevel="signature"

repack, then unzip the modified version

use AndroidManifestBinaryXml_ChangePackageName to rename the package name

java -jar rename\setAxmlPkgName.jar com.xiaomi.hm.health_2.1.9-backup_2\AndroidManifest.xml com.xiaomi.hm.health.backup!

rezip the renamed version (yeah, not really clean, should be zip-aligned or something, but I tried to minimize the amount of tool to use .. and it's still correct / working)

sign it (with something like http://xdaforums.com/showthread.php?t=822388 if you don't want to struggle with jdk)

nb : you can't reinstall the modified version over the original one (INSTALL_PARSE_FAILED_INCONSISTENT_CERTIFICATES), you can't uninstall original version keeping it's data (pm uninstall -k packagename) to reinstall a modified package, as the signature is different you would get INSTALL_FAILED_UPDATE_INCOMPATIBLE, and removing the protectionlevel permissions is required to be able to install alongside original version (would fail with INSTALL_FAILED_DUPLICATE_PERMISSION otherwise).

Usage :

install apk

log-in

wait for the data to be retreived, check in the history

try to export data, using adb, Helium, https://github.com/hanxue/mifit-data-export, .. but don't forget that the package name you want to backup is "com.xiaomi.hm.health.backup"

use the data

Enjoy
[General Accessories] vobrok:
Has been anybody successful getting out raw data about steps on Xiaomi Mi Band 2? I can get the summary and even raw files, but it doesn't adds up to something meaningful (like 140k steps per day and so on...).
[General Accessories] Dave_Scream:
I cant understand how raw data extraction works. If I could understand algorithm, maybe I could make some software to decode it.

I open origin_db in SQLiteStudio. It have table named "DATE_DATA" It have SUMMARY field - wich used for days summary extract2.csv (works now), looks like

Also there is two other fields DATA and DATA_HR - I think they are RAW data.

DATA field have something like BASE64 (but not BASE64) data like:

DATA_HR have binary data.. How to export it? Need to understand how algorithm work and then fix it or write new.
[General Accessories] vobrok:
Yeah, I understand and have the same problem... No idea how they do it. I had a crazy idea, which is definitely suboptimal, but at least something and it actually may work (at least we would be able to estimate the error). We could use the raw data (where it is not binary) and then try to predict those summarised ones by some machine learning model, e.g. linear regression or whatever. I have to little data points to do that currently (have the band less than 14 days). If someone could provide me at least hundreds of data - raw files with summary or better - sum of summary per day and then "true" summary, I could create the model and try that.
[General Accessories] mibanddd:
Hey fellows!

I succesfully decoded the new date_data format. It's encoded in base64.

Follow these steps:

1) use SQLite Browser to export content in data field of a specific day in table date_data of e.g. origin_db_123456789

2) copy content of "value" (e.g. ATwfED4AAU0aAU0nAVo.... as pasted by Dave_Scream)

3) go to

conv.darkbyte.ru

and paste it into [ Base64 ], then click on decode

4) now in [Dec] you have the data in the following format in groups of three for each minute:

5) use Notepad++ to replace using regex:

(\d+) (\d+) (\d+)

to:

\1\t\2\t\3\n

6) here you have the data:

1	60	31
16	62	0
1	77	26
1	77	39
1	90	30
1	75	23
0	74	12
0	23	0
0	1	0
0	1	0
0	23	0
.... and so on

1440 lines, each line for each minute of the day. You can copy - paste it to an excel file (e.g. one where you have imported the created .csv) to the according day.

I noticed that lines starting with 126 (and then 0 0) seem to be invalid. E.g. the last day (current day) apparently is filled up with such "126 0 0" lines to have the same length in the data (1440 lines), so for example if you extract data at 17:52 o'clock, after that time the lines will be filled up with "126 0 0" lines.

==>

I guess we only have to include a base64decode function in the extract raw script.
[General Accessories] mibanddd:
Now raw data extraction is fixed for Mi Band 2!

Here the new version for windows:

As mentioned before, to use it you need to edit "extract_data.bat" file with notepad:

set origin_db=origin_db_1610064220 - set your ID here. Go MiFit -> Profile -> My Personal Info -> ID: 1610064220
set SDPathTemp=/storage/sdcard0 - set your path to SDCard
set UserHeight=180 - set your height here
set UserWeight=65 - set your weight here
set ExtractRaw=Y - make sure to set Y for yes, if you want to get the raw output (extract_raw_hr2.csv). Since it can take a long time, you can switch it to N in case you don't need it.

To use the Access Database and the Excel file in the folder _miband_analytics_v560\excel\

go to Dave_Screams post for an explanation.

Download the new miband2_extract.rar here:

http://www.file-upload.net/download-12053318/miband2_extract.rar.html

If you ony want to download the changed miband2_raw.sql and place it in your /db/ folder, you can download it here:

http://www.file-upload.net/download-12053252/miband2_raw.sql.html

Here is a pastebin of the new miband2_raw.sql :

http://pastebin.com/kDaBV3ru

---------- Post added at 10:26 PM ---------- Previous post was at 10:01 PM ----------

Note that in the raw data (extract_raw_hr2.csv), the sleep is not recognized anymore. For Mi Band 1, sleep was coded with "RawActivity" 5 and light sleep with 4. Using Mi Band 2 and the new app, there only appears a 0 during sleep, the same as being idle when awake.

This does not affect the normal, generalized sleep analysis. Only in the raw data this distinction is now missing.

However if you have activated the "heart rate sleep assistant" in the Mi Fit app (profile - mi band 2 - heart rate assistant), then during sleep the heart rate appears in the column HeartRate.

Also, sleep phase information still seems to be included in the "mili_log.txt" in /data/data/com.xiaomi.hm.health/files
[General Accessories] emarcin:
Perfectly
[General Accessories] ayttk:
Perfectly
[General Accessories] dong2014:
Hi have a MiBand 1 (FW v1.0.12.5), Mi Fit (v2.2.4), Samsung S3 running Android 4.3 and am having problems extracting my data.

I have teh the extract 5.92v.rar package and I found that “adb -d backup -f mi.ab -noapk -noshared com.xiaomi.hm.health” on windows 8.1 is returning a 0Kb file for “mi.ab” Any ideas why?
[General Accessories] u363840:
If you're rooted you shouldn't use this method. There's another method that put the relevant files on the sd card / a storage on which files can be retreived.

If you don't want to root, you have to use a modified version of the app package, one that allow android backup, either by replacing the official package (but you have to uninstall first, your data should be redownloaded), or using a repacked version under another packagename like http://xdaforums.com/showpost.php?p=69133139&postcount=266 (can be installed aside the official, but only one of the will get the update from the band (I'd advise to disable bluetooth during the operation) so you should uninstall after you retreived the data (the official app will say you're connected elsewhere and will ask you to reconnect)), or downgrading to a version that still allow android backup (but may have side effects ?).
[General Accessories] malush:
someone know how to convert number of "day of week"(0,1,2...) into sun,sat,fri... ?
[General Accessories] marekyggdrasil:
resulting mi.ab is empty

Anyone had similar issue?

After executing manually

adb backup -f mi.ab -noapk -noshared com.xiaomi.hm.health

The backup procedure is recognised ( I get the query for authorising it on the phone screen).

My resulting mi.ab is

$ cat mi.ab
ANDROID BACKUP
3
1
none
x?c`?`?T

Which further results with empty archive. I tried to debug script myself, I made sure package name is correct

$ adb shell 'pm list packages -f' | grep health
package:/data/app/com.xiaomi.hm.health-1/base.apk=com.xiaomi.hm.health

My MiFit app is

2.2.10

my phone is

MI4LTE

Android

6.0.1 MMB29M

and I am using MIUI

MIUI 8.1 | Stable 8.1.3.0(MXDCNDI)
[General Accessories] rubenix:
Hello there, I've been using this awesome script to extract all data from mifit databases and exported to html files for laters review...

Yesterday got mifit updated to v 2.2.11 and script no longer works. Has anyone facing the same??

I think they (xiaomi) changed something on their db...

edit: forgot to paste errors

* daemon started successfully *

pull: building file list...

pull: /storage/sdcard0/mistats/databases/origin_db_ad7a58db267f0968f30dbc68c6d93d1f-journal -> .\temp/origin_db_ad7a58db267f

pull: /storage/sdcard0/mistats/databases/origin_db_ad7a58db267f0968f30dbc68c6d93d1f -> .\temp/origin_db_ad7a58db267f0968f30d

pull: /storage/sdcard0/mistats/databases/sports.db-journal -> .\temp/sports.db-journal

pull: /storage/sdcard0/mistats/databases/sports.db -> .\temp/sports.db

pull: /storage/sdcard0/mistats/databases/origin_db-journal -> .\temp/origin_db-journal

pull: /storage/sdcard0/mistats/databases/origin_db -> .\temp/origin_db

pull: /storage/sdcard0/mistats/databases/origin_db_43960891-journal -> .\temp/origin_db_43960891-journal

pull: /storage/sdcard0/mistats/databases/mistat.db-journal -> .\temp/mistat.db-journal

pull: /storage/sdcard0/mistats/databases/mistat.db -> .\temp/mistat.db

9 files pulled. 0 files skipped.

1818 KB/s (891576 bytes in 0.478s)

Error: near line 7: no such table: date_data

Error: near line 49: no such table: date_data

Error: near line 88: no such table: date_data

Error: near line 114: no such table: WEIGHT_INFOS

Error: near line 121: no such table: HEART_RATE

Error: near line 128: no such table: android_metadata

*** Extract RAW data ***

Error: near line 36: no such table: date_data

Error: near line 74: no such table: date_data

Error: near line 101: no such table: date_data

Error: near line 131: no such table: heart_rate

*** End ***

Pulling data from phone still works but the way to extract data doesn't... Hope this post still alive & can fix this in a meanwhile.

edit 2: got it fixed!!! it just changed the origin_db so as per Dave_Scream post

just do the following to find out your origin_db as mine changed to something completely new...

Quote:

adb shell

su

/data/data/com.xiaomi.hm.health/databases/

ls

Just replace the name in the extract_data.bat to get it sorted
[General Accessories] YoungSeok:
I'm curious about raw data.

Hello.

I downloaded miband_extract_v592.zip.

To extract sensor data, I set the ExtractRaw parameter to Y. It is extracted normally.

But I did not understand exactly what this was. It's RawActivity and RawSensorData.

1. Can you explain exactly what this data is (Raw Activity, Raw SensorData)?

2. What is the sampling rate?

3. In fact, I need information about the x-axis, y-axis, and z-axis associated with the accelerometer. Can I get this?

I hope you have a good day.

Thanks.
[General Accessories] kaefert:
The sampling rate of the data that is stored is 1/60Hz, which you can easily see from getting one line per minute..

Sadly there is no access to "really raw" sensor data separately for each of the 3 spatial axis of the accelerometer. I believe that the column "RawSensorData" is some kind of aggregation for all 3 axis, so higher values means more (changes of) movement without any filter for specific movement patterns (like steps of human walk)
[General Accessories] Baldorus:
Can anyone help me? I have successfully copied mi band 2 files from the database on my phone to the desktop. What should I do next to extract the raw data?
[General Accessories] crc3212:
Can anyone help me? I have followed all the instructions but when I run the bat file it gaves me an empty CSV file.

I have a Mi Band 1S firmware version: 4.16.4.22, Mi fit app 2.4.1, a note 3 neo and a laptop with Win 10

Thx to anyone will help me
[General Accessories] Gollob:
Can anyone help me please, I cannot seem to get the extraction going.

I have a Redmi Note 4 (Android 6.0 MRA58K, MIUI Global 8.1 | Stable) and I use a Windows 7 x64 computer.

Ok, I made some progress since my last post... I think the reason I could not communicate with my Redmi Note 4 might have been an outdated version of adb.exe. I updated the exe:

old> adb.exe 815,104 09/11/2012 20:54

new> adb.exe 944,128 09/10/2015 23:58

Now I get a little bit further in this challenge. It seems that because my phone is not rooted the backup approach is tried by the script (after setting ForceBackupMode=Y). Sadly, I never get this dialog on my mobile to allow backups, all I get is this in the cmd window:

c:\Z\miband>run

The system cannot find the file specified.

The system cannot find the file specified.

The system cannot find the file specified.

The system cannot find the file specified.

Press "Backup My Data" button on device...

* daemon not running. starting it now on port 5037 *

* daemon started successfully *

Now unlock your device and confirm the backup operation.
[General Accessories] pocketalpha:
Wow! So cool. I am waiting for the Xiaomi Mi Band 2. Can I use this for the Xiaomi Mi Band 2?
[General Accessories] Juhele:
Hi,

hope this will not be considered as off-topic post as I am talking about other app. I am new owner of Mi Band Pulse 1S and I also wanted to export all the data for my own analyses etc. Found article about Mi Band Master app which looked really promising so I installed it. It supports Mi Band 1/ 1A/ 1S/ 2 according to Google Play information.

When browsing the app menus and settings I found the "Export data" option (sorry I cannot force other language on my phone):

After some attempts using trial and error I selected Total Commander in the dialog after choosing the Export data option and got about 130 KB db.sqlite file. After transfer to my PC and opening in DB Browser for SQLite I was able to see all the data structure:

and the data:

At least from my initial look on the db it seems to be really good. There are all the bracelet data although:

- the time values are obviously written as Unix timestamps so you have to use LO Calc/ MS Excel formula =A2/86400+25569 so if you have for example in A2 value "1494234255" then it turns in "08.05.2017 09:04:15". (still it is in UTC so you need to add or subtract some hours depending on your time zone)

- did only found number of steps but not distance - this is probably calculated by the app but it should be easy to find out the "conversion factor"

- the sleep data is written like "10L16D39L62D17L39D45L18D5L14D8L" so you would have to separate it in particular "light" and "deep" sleep data somehow

But at least we have the data in machine readable format.

This is all from the free version of the app. I am going to buy the PRO in a short time so we will see....
[General Accessories] xalien:
Mi Band data extraction for Mi Fit 2.4.x

hi all, any chance to have extraction tool working for Mi Fit 2.4.x?

thanks
[General Accessories] mibanddd:
Try to follow the procedures described in posts #271, #275 and #279 on page 28

https://xdaforums.com/general/accessories/xiaomi-mi-band-data-extraction-t3019156/page28

I don't know of any change with version 2.4.x yet. Just that since around somewhen in 2016 you need root access or a specially changed apk file that supports backup, as mentioned in post #275.

However if you are able to download your individual origin db file and see the data in an SQLite Editor, but the data extraction script is not working correctly, then please tell us details about the problem.
[General Accessories] EspElement:
This no longer works with the latest version of Mi Fit 3.x.x. I just went back to older version, but not sure what is causing the break.