Accessing and Cleaning Data from Garmin Wearables for Analysis | by Adam Brownell | Towards Data Science

When I started working on a project to analyze my Garmin Fenix data, it wasn’t clear how to access this amazing dataset, and required some technical know-how.
Garmin’s documentation was also hard to follow and outdated, resulting in me having to dig through code repos and old internet forums...
Hopefully this article can transform what took me a non-trivial amount of time into something simple and easy to understand.
All images in the article unless otherwise noted are by the author
While there a bunch of factors on what data is available, ranging from what Garmin model you are using to if you are consistently wearing the chest heart rate monitor, here are the features I was able to extract from my Garmin Fenix without a chest strap:
General Fitness Data:
Wake Time
Steps
Cycles
Ascent/Descent
Heart Rate
Activity Type and Minutes (Moderate and Vigorous)
Total Moving Time
Walking Step Length, Average Speed
Calories burned, Total fat calories burned
Training Stress Score
Max & Average Temperature
Stress Level
Speed & Intensity
Resting Metabolic Rate, Resting Heart Rate
Swimming: Pool length, stroke length and distance, swimming cadence, first lap index
Running: Max running cadence, max general cadence, total strides
Exercise-Specific Event Data:
Time
Distance travelled
Location (Longitude/Latitude)
Altitude & Temperature
Heart Rate
Cadence (and Fractional Cadence)
Speed & Power
Left-Right Balance
While you can also access this data via Garmin’s SDK called “Garmin Connect IQ”, I will go over the manual way to access this data. It is actually a lot easier, especially for personal use, but Garmin has outdated documentation so it seems complicated...
First, head to your Garmin Account Data Management (https://www.garmin.com/en-US/account/datamanagement/) and log in:
Then head to “Export your Data” and clickRequest Data Export.
You will then receive a message stating:
Your request has successfully submitted.You can request a copy of all personal data, including your profile, order history, information from Garmin Connect and other applications, subscriptions, registered devices and more.We will send a link to download your export file to [yourAccountEmail@email.com]Files typically take about 48 hours to prepare but, depending on the number of requests being processed and the amount of data associated with your profile, could take up to 30 days.
Congrats! You will get your data soon. While the message says up to 30 days, I got mine within 30 minutes. (Also why would it take them 30 days to process a data request? I requested it from inside my account, so there aren’t any permission/security issues... do they have someone manually sending me my data...?)
Eventually you’ll receive a .zip file. Open this to reveal the folder structure:
For me, most of these folders were empty because I was not using them. And of the ones that weren’t empty, nearly all of them contained a single JSON file of customer data or aggregated exercise data.
The only one we care about is DI_CONNECT > DI-Connect-Fitness-Uploaded-Files:
Click on the .zip file in this nested folders, and you will create a folder filled with .fit filetypes
This is the trickiest part, as we need to convert these from .fit to .CSV using the Command Line. .fit is a specialty filetype Garmin uses that none of your data analysis tools (Excel, Atom, Jupyter) can use.
We will be relying on Garmin’s FIT CSV Tool (https://developer.garmin.com/fit/fitcsvtool/) from the FIT SDK. It’s free but requires setup.
First, open Command Line Terminal. If you’re unfamiliar with CLI, it should look like this:
Next we need to check if we have JAVA installed. An easy way to check is to type java -version into our terminal. If we get a number returned, it’s installed. If we get an error message, we’ll need to install it
Installing Java: Depending if you’re running MacOS/Windows/Linux, you will need to download different files. Like the terminal message says, head to https://www.java.com/en/download/ and download the java package. This will prompt an installer window which you can follow along here. After download success, close + reopen your terminal and verify java was installed via the same java -version test.
Once we know Java has been downloaded, go ahead and Download the FIT SDK as well (https://developer.garmin.com/fit/download/) and decompress the .zip
This will give you a folder with the following contents:
Now, we can convert our .fit files into .csv 🚀 The jar file that will accomplish this is in a java folder called FitCSVTool.jar
Type the following into the command line, replacing <file> with the file path of a .fit file:
java -jar FitCSVTool.jar <file>
If you are getting errors such as Unable to access jarfile FitCSVTool or (No such file or directory) , you’ll need to figure out the right file paths. On a Mac, it’s as easy as dragging/dropping the file directly into terminal. In windows you’ll need to copy/paste file paths from file properties.
If you are successful, a .csv file will be created in the same directory as the original .fit file.
This could be the end of conversion process, but typing out the above command for every single file is a pain. I had over 8,000 .fit files, and you may have more. Luckily we can automate this.
The Windows and Mac guides will now diverge for this step. As mentioned in the documentation:
The FIT SDK includes multiple Windows batch files that provide a drag-n-drop interface for FitCSVTool. These batch files are not compatible with OSX; however, the same functionality can be accomplished using an Automator application.
For Macs, we can create an Automator as outlined in the above docs; Open up Automator and...
Select “Application” from the window and click Choose
Give it a Name
Find “Filter Finder Items” in the Library folder and drag it into the right-side screen. Type in “.fit” to the contains field
Find “Run Shell Script” in the Library folder and drag into the right-side screen. Fill out with the following code snippet, replacing path/to/fit/sdk/java/FitCSVTool.jar with your path to the tool used above (remember you can drag and drop!):
FitCSVToolJar=path/to/fit/sdk/java/FitCSVTool.jarwhile read inputfiledooutputfile=${inputfile/.fit/}java -jar $FitCSVToolJar -b "$inputfile" "$outputfile"_rawjava -jar $FitCSVToolJar -b "$inputfile" "$outputfile"_definitions --data nonejava -jar $FitCSVToolJar -b "$inputfile" "$outputfile"_records --defn none --data recorddone
After all this your Automator Application should look like this:
Save this to desktop
Highlight and drag all of the .fit files on to the Application Desktop icon, seen here
This should start to automatically convert all of the thousands of documents you have. For me, this made my MacBook fan very loud and took about a half hour.
While I don’t have a Windows machine to test this on, I do believe that Windows do a similar drag/drop system by dragging all of the .fit files onto one of the .bat files in the FitSDK folder:
As mentioned in the Windows docs:
An easier way to use FitCSVTool is to use one of the five batch files included in the SDK. These files are located in the same folder as FitCSVTool.jar. Drag-n-drop a FIT file onto any of the batch files and the output .csv files will be created in the same folder as the source FIT file.
As a consequence of the previous step of converting from .fit to .csv, we now have several .csv files per .fit file:
A _definitions files outlining which fields are present in the data
A _records_data file with a summarized data of distances, speed of an exercise routine
A _records file containing location, speed, and distance of an exercise routine moment by moment
A _raw file containing everything, including stress level, heart rate, and other metrics. (These are extremely messy as seen in the next section)
🎉 Congrats, you have your data in CSV format! 🎉
Everything in this step and beyond is data cleaning that I recommend, but optional. You may also have different data than me, so these cleaning processes may not work for you.
With that said, I am pretty confident the following steps will help anyone looking to analyze their Garmin data.
Looking at the definitions files, they don’t seem to contain anything useful. From spot checks, all the Value columns seem to be 1. I’m assuming this is important for .fit files, but not for our purposes.
To remove definitions files, navigate to the location of your csv files in terminal by running:
cd path/to/csv/files
and then running:
rm *definitions*
This could also be easily be done by removing the java -jar $FitCSVToolJar -b "$inputfile" "$outputfile".definitions --data none from our automator originally, but wanted to ensure anyone who wanted the file could use it
We still have an issue of having thousands of CSVs to parse through to look at our data. You can combine all CSVs with:
cat *.csv >garmin_combined.csv
or if you are planning to generate the same data schema as I have below:
cat *_records.csv >combined_records_full.csvcat *_raw.csv >combined_raw_full.csv
which creates 2 different combined CSVs for us to use.
😃Good News: The _records files generated from our .fit conversion look good.
🙁Bad News: The _raw data files are SO MESSY AND UGLY. Empty fields, special characters, column values meaning different things... oh my.
I highly recommend pivoting by timestamp so that the columns are meaningful. This way you can quickly and easily check any metric of interest by the time of recording, which is assumably the most common use of this data.
By aggregating by timestamp, we also greatly reduce the file size without losing any data. A win-win!
I have written a cleaner function in Python to create these columns. I’ve uploaded it to a Github Repo, file clean_garmin.py, or copy paste this following into you own clean_garmin.py file:
Installing Python: You’ll need Python on your computer to run the above function. Go to https://www.python.org/downloads/ and install Python. Then go to terminal to make sure download successful with python --version.
Once you save the above code block, you can run in terminal:
python clean_garmin.py -i combined_records_full.csv -o combined_records_clean.csvpython clean_garmin.py -i combined_raw_full.csv -o combined_raw_clean.csv
Note: This will take a long time, especially the raw files. It took my computer 10min for records, and about 2.5hr for raw. The reason for this is that sometimes the timestamp on a data entry is local rather than global, meaning it references the most recent global timestamp. So every time we do this conversion the code needs to find the most recent global timestamp and convert the local timestamp into global... But this runs in O(n2) time 💀
...And our data looks a lot better 🙌
If my clean function fails on your data, or if you have a better implementation of the data clean, please leave a comment so we can collaborate!
We still have thousands of files in our folder, and it stresses me out. Go ahead and remove all of the non-aggregated CSVs with:
rm *_raw.csv*rm *_records.csv*rm *_records_data.csv*
Now that we have a clean, usable CSV of our Garmin data, we can start analyzing and gaining insights from this beautiful dataset.
This is the fun part of data science/analytics, and hopefully made all the hoops we jumped through worth it!
Since this article is already quite long, I will write a follow-up with a few methods to analyze Garmin data to draw interesting insights on our exercise and fitness habits.
If any of the above steps did not work for you, PLEASE leave a comment so I can try to help debug and add the solution to the article. My hope is that no one has to go digging through the forums for answers again, getting all the answers here.
Hope you enjoy, and good luck in your analytics journey!