Validity assessment of biometrics for the first-generation Apple Watch and Microsoft Band during steady-state exercise

VALIDITY ASSESSMENT OF BIOMETRICS FOR THE FIRST-GENERATION APPLE WATCH AND MICROSOFT BAND DURING STEADY-STATE EXERCISE William C. Kalamaras, M.S. Ed. Department of Kinesiology and Physical Education Northern Illinois University, 2016 Dr. Steve Howell, Co-Director Dr. Amanda Salacinski, Co-Director In recent years, wearable technology equipped with advanced biometric sensors has grown in both market share and consumer popularity. With these devices depending heavily upon the use of algorithms to estimate physiological parameters, the validation of biometric readings from these devices is paramount in establishing the credibility necessary to substitute for conventionally used equipment. PURPOSE: To assess the biometric validity and reliability of heart rate (HR) and energy expenditure (EE) recorded by the first-generation Apple Watch (42mm) and Microsoft Band during submaximal aerobic exercise on a treadmill at steady state. METHODS: Twenty-three participants (16 males, 7 females; mean age: 22.7 + 3.8 years old) participated in three, 20-minute testing trials performing submaximal aerobic exercise: two trials assessed biometric validity for HR and EE in each wearable smartwatch (at 50% and 75% VO2 max). A third testing trial (repeat of 75% VO2 max; alternate wrist placement) assessed device reliability agreement across left and right wrists. Criterion values were obtained for HR (bpm) and EE (kCals) using a Polar H7 heart rate monitor (HRM) (Polar Electro Oy, Kemple, FIN) and ParvoMedics TrueOne 2400 Metabolic System (ParvoMedics, Salt Lake City, UT), respectively. Mean differences were assessed through a repeated-measures analysis of variance (ANOVA) (=0.05) for HR and EE independently. Follow-up paired t tests were run after significant main effects occurred. Pearson correlation coefficients (r) and standard error of estimates (SEE) were used to establish validity of each biometric, while intraclass correlation coefficients (ICC) based on a two-way random effects ANOVA model and an absolute agreement definition [ICC(2,1)] were used to assess device reliability agreement across left and right wrists. RESULTS: Mean HR (bpm) values for testing trials one (50% VO2 max) and two (75% VO2 max) were recorded for their respective intensities at 145.2 ± 8.4 and 172.5 ± 9.3 bpm for the Apple Watch; 141.1 ± 8.0 and 165.1 ± 8.6 bpm for the Microsoft Band; and 145.1 ± 8.4 and 172.3 ± 9.5 for the Polar H7 HRM. Criterion comparisons revealed no significant difference between devices for Apple Watch HR (p = .164), and a 5.6 bpm difference (lower) for the Microsoft Band (p < .0005). Mean EE (kCals) values for testing trials one (50% VO2 max) and two (75% VO2 max) were measured for their respective intensities at 160.1 ± 51.1 and 212.2 ± 55.9 kCals for the Apple Watch; 182.7 ± 55.9 and 264.4 ± 55.0 kCals for the Microsoft Band; and 164.2 ± 51.2 and 221.1 ± 62.0 for the ParvoMedics 2400 TrueOne Metabolic System. Criterion comparisons revealed a 6.5 kCal difference (lower) for the Apple Watch (p = .002) and a 22.0 kCal difference (higher) for Microsoft Band (p =.03). Pearson correlation coefficients (r) along with 95% confidence intervals (CI) and standard error of estimates (SEE, bpm) for HR at the respective intensities (50%, 75% VO2 max) were computed at r=1.00 (1.00 – 1.00), (SEE= 0.54) and r=0.99 (0.99 – 1.00), (SEE=1.04) for the Apple Watch and r=0.91 (0.80 – 0.96), (SEE=3.57) and r=0.90 (0.71 – 0.94), (SEE=4.83) for the Microsoft Band. Correlation coefficients and SEE (kCals) for EE were computed at r=0.98 (0.95 – 0.99), (SEE=10.07) and r=0.98 (0.95 – 0.99), (SEE=12.34) for the Apple Watch and r=0.49 (0.10 – 0.75), (SEE=45.59) and r=0.74 (0.49 – 0.88), (SEE=37.60) for the Microsoft Band. Mean HR across wrists did not differ in the reliability agreement trials (repeated 75% VO2 max trial) for either the Apple Watch (p = .18) or Microsoft Band (p = .75). Mean EE between wrists also did not differ for either the Apple Watch (p = .88) or Microsoft Band (p = .80). ICCs for HR reliability agreement along with 95% CIs were 0.61 (0.28 – 0.81) for the Apple Watch and 0.51 (0.13 – 0.76) for the Microsoft Band; ICCs for EE reliability agreement (within a 95% CI) were 0.98 (0.96 – 0.99) for the Apple Watch and 0.73 (0.46 – 0.88) for the Microsoft Band. CONCLUSION: The Apple Watch provides valid measurements of both HR and EE biometrics during both submaximal aerobic exercise intensities at steady state (50% and 75% VO2 max). While evidence for the Apple Watch does not establish reliability agreement across left and right wrists for HR, it does establish excellent reliability agreement for EE. The Microsoft Band provides less valid measures of HR, which are more valid at a lower exercise intensity (50% VO2 max). The Microsoft Band does not provide valid measures of EE, nor does it provide reliable measures across left and right wrists for either HR or EE.