A new medical study from Stanford University focusing on consumer fitness tracker reliability, published Wednesday in the Journal of Personalized Medicine, has crowned Apple Watch the king of heart rate monitoring while pointing out shortcomings in its calorie counting feature.
“People are basing life decisions on the data provided by these devices,” Euan Ashley, DPhil, FRCP, professor of cardiovascular medicine, of genetics and of biomedical data science at Stanford said in a statement.
The study included 29 male and 31 female volunteers who wore several fitness trackers like Basis Peak, Fitbit Surge, Microsoft Band, MIO Alpha 2, PulseOn, Samsung Gear S2 and Apple Watch. The study pitted the wearable gadgets against FDA-approved equipment.
The participants were asked to complete a total of 80 physical tests, including such activities as cycling, running and walking. They compared data against an FDA-approved 12-lead electrocardiograph for measuring heart rate and clinical-grade indirect calorimetry, which determines calories burned by measuring oxygen and carbon dioxide expelled when breathing.
Heart-rate monitoring via Apple Watch achieved the highest accuracy across measured modes of activity with an error rate of two percent, followed by Basis Peak and Fitbit Surge.
Samsung's Gear S2 had the highest heart rate error rate at 6.8 percent.
Researchers set an acceptable error rate at five percent, meaning Samsung's device fell just outside the study's acceptable buffer.
All fitness devices they tested fell short in calorie counting.
In terms of determining the amount of calories burned, Fitbit's Surge was the most accurate device with an error rate of 27.4 percent. PulseOn was the least accurate tracker in terms of calorie count with an astounding error rate of 92.6 percent. Apple Watch had an error rate near 40 percent while Microsoft Band came in at around 33 percent.
Low-impact activities like sitting caused the most inaccuracies with an average error rate of 52.4 percent compared against high-impact activities, such as walking and running.
This is due to the differences in how people exercise. “People are so variable,” Ashely said. “Some people walk smoothly and others waddle along, and that has an impact."
“The heart rate measurements performed far better than we expected, but the energy expenditure measures were way off the mark,” she added.
“The magnitude of just how bad they were surprised me.”
Each of the tested devices uses its own proprietary algorithm for calculating calorie burn, which could explain the wildly differing readings in terms of energy expenditure rates.
