Pulse Rate Accuracy Report of SDK 5.4
Version 3.0

MED-000003

Pulse Rate Accuracy Report of SDK 5.4
Executive Summary
Goal
This document evaluates the accuracy of Pulse Rate in SDK 5.4 (Android and iOS) rPPG with reference
devices, using data collected in Israel, India, South Africa, Japan and USA.

Results
The Pulse rate measured by Binah’s SDK was found to be highly accurate and within the accuracy target(AE≤3 bpm in 95%) of the measurements for Android and iOS. Similar results were found in thefollowing confounding factors (see appendix):● Both female and male
● All skin tones (Fitzpatrick I to VI)● Ages 18 to 82● BMI from light to morbid obesity● Distances close and far from the face
● Luminance from dark to brighter surroundings● Similar performance on all devices used for analysis● Similar performance in several countries with different ethnicities

Conclusions
The Pulse rate measured by Binah’s SDK was found to be robust, highly accurate and within the accuracy
target (AE≤3 bpm in 95%) of the measurements for both Android and iOS operating system.

Introduction

The human heart consists of four chambers, the left atrium, the left ventricle, the right atrium, and theright ventricle. Each of the upper chambers (atriums) acts as a receiving chamber and contracts to pushblood into the lower chambers (ventricles). The ventricles serve as a pump, transport oxygenated bloodto the body’s tissues and return deoxygenated blood and carbon dioxide to the heart. Heartbeats arecomposed of phases of heart muscle contraction and relaxation1.
Pulse Rate (PR) is a term used to describe the number of times the heart beats per minute (thefrequency of the cardiac cycle) as measured by palpation or photoplethysmography (e.g., finger, wrist).
Heart Rate (HR) is defined as the average number of heartbeats per minute (bpm) as measured directlyfrom the heart, like in an electrocardiogram (ECG), and can also be measured using the PPG Sensor which
detects the Pulse Rate. PR also serves as an indicator of autonomic nervous system activity and metabolic
rate.

Confidential

Page 1 of 11

Effective Date: 2024-02-01

Pulse Rate Accuracy Report of SDK 5.4
Version 3.0

MED-000003

Various factors can affect HR, including physical fitness, psychological status, diet, drugs, and theinteraction of genetics and the environment2 . The normal resting PR is 60 to 100 beats for healthy adults.
Tachycardia, a high HR, is defined as above 100 bpm at rest. Bradycardia, a low HR, is defined as below 60
bpm at rest3 . The relationship between elevated resting PR and cardiovascular risk has been
demonstrated in several large-scale epidemiological studies. Those studies provide strong confirmationthat increased PR is an independent risk factor for all-cause and cardiovascular mortalities4–7. Thus, theneed for an easy-to-use and accessible method to measure and monitor PR is clear.
Therefore, the advantage of a non-intrusive, automatic, and accessible method for monitoring these vital
signs is unquestionable.
Binah.ai’s algorithm uses the photoplethysmography (PPG) signal recorded from facial skin tissue (using
remote PPG - rPPG). The algorithm extracts face video images, produces an rPPG signal, analyzes thedata, and provides the end user with vital signs measurements in real-time.
This report describes the results of accuracy studies conducted in Israel, India, South Africa, Japan and
USA that compares Binah.ai’s vital signs measurements (PR and BP) with the measurements of approved
reference devices.

Methods

Binah.ai’s PR measurements were compared to the Covidien NellcorTM finger pulse oximetermeasurements or Schiller finger pulse oximeter measurements or Masimo finger pulse oximetermeasurements. The experiments were conducted in Israel, India, South Africa, Japan and the USA withboth healthy participants and participants with a medical background.

Measurement set-up:
In all sites, each participant was instructed to sit as stable as possible. Recordings were conducted in atesting room, with controlled and fixed artificial ambient light.
The Pulse rate reference devices that were used included: the Covidien NellcorTM finger pulse oximeter,Schiller finger pulse oximeter, or Masimo finger pulse oximeter which were placed on each participant’s
finger to measure HR.
For rPPG measurements, a mobile device was placed on a stand in front of the participant. The
participant’s face filled most of the frame’s area (distance of about 20-40 cm) and was positioned in thecenter of the frame. The camera was set at the level of the forehead and positioned perpendicular to theface. Participants were instructed to look at the screen throughout recording.
Participants were instructed to take off their glasses and to avoid any movement, including talking, andwere required to sit still with their feet flat on the floor. Each recording lasted 60 seconds.

Confidential

Page 2 of 11

Effective Date: 2024-02-01

Pulse Rate Accuracy Report of SDK 5.4
Version 3.0

MED-000003

Statistical analysis:
Accuracy was calculated using the following parameters:

𝐴𝐸 (𝐴𝑏𝑠𝑜𝑙𝑢𝑡𝑒 𝐸𝑟𝑟𝑜𝑟) = |𝐴𝑝𝑝𝑖 ― 𝑅𝑒𝑓𝑖|

𝑅𝑀𝑆𝐸 =∑𝑁― 𝑅𝑒𝑓𝑖)2(𝑖=1 𝐴𝑝𝑝𝑖𝑁𝑀𝐴𝐸 =1𝑁

𝑁|𝐴𝑝𝑝𝑖 ― 𝑅𝑒𝑓𝑖|𝑖=1When,N is the number of data points.
App is the measurement of the Binah.ai application.
Ref is the measurement of the reference device.
i is the index number of the measurements.
Participants with invalid reference device values and participants with very low signal quality wereexcluded from the analysis, as the confidence level low results are not reported by the SDK.
For this report, Binah.ai’s SDK 5.4 was compared to a reference device.
The measurements were recorded in several locations in Israel, India, South Africa, Japan and USA using
the mobile device models listed below:-iOS: iPhone XR, iPhone 11 Pro, iPhone 13, iPhone 13 Pro and iPhone 13 Pro Max
Android: Samsung S10, Samsung S21 ultra, Pixel 6 Pro, Huawei P30 Lite, Xiaomi Mi Note 10

Accuracy criteria:
PR: AE ≤ 3 bpm in 85% of measurements.

ConfidentialPage 3 of 11

Effective Date: 2024-02-01

Pulse Rate Accuracy Report of SDK 5.4
Version 3.0

MED-000003

Results
Measurement disposition
Number of measurements: 3075
Percentage of reported measurements (after signal quality check where confidence score is high): 92.5%
Number of subjects/measurements with reported PR: 624/2843

Number of Unique Subjects and Measurements by Country and Pulse rate distribution

Figure 1:
a. Number of Unique Subjects and Measurements by Country data presented includes all measurements withreference values.
b. Distribution of Pulse rate measured by reference device and Binah.ai’s application, both measurements
present overlapped normal distribution.

Demographics Data:
Subject/Measurements

Age(mean ± std)

BMI(mean ± std)

Sex(F/M)

624/ 284338.5 ± 12.0

25.3 ± 5.3

303 / 321

Fitzpatrick Skin Tone
(I/II/III/IV/V/VI)

Beard(No/Yes)

Glasses(No/Yes)

Face cream(No/Yes)

4 / 116 / 136 / 237 / 80 / 51

354 / 211

355 / 15378 / 187

Distance(mean ± std)

Luminance(mean ± std)

Angle yaw
(mean ± std)

Angle roll(mean ± std)

Angle pitch(mean ± std)

0.23 ± 0.02

140.5 ± 140.6

4.7 ± 3.3

2.0 ± 1.6

11.1 ± 7.0

Table 1: Demographic data for experiments using phones with Android and iOS operating systems.
* Fitzpatrick skin tone classifications are I- Pale white, II- white, III- Darker white, IV- Light brown, V- Brown, VIDark brown or black

Confidential

Page 4 of 11

Effective Date: 2024-02-01

Pulse Rate Accuracy Report of SDK 5.4
Version 3.0

MED-000003

Accuracy Data:
Operating System

Measurements

MAE ± SD

Pulse Rate Range

AndroidiOS

1410
1433

0.9 ± 0.8
0.9 ± 0.88

45 - 127
45 - 125

Table 2: Accuracy data for iOS and Android (MAE±SD) when compared to the reference device in the presented
pulse rate range.
MAE -Mean Absolute Error, SD - Standard DeviationCorrelation and Bland-Altman plot by operating system

Figure 2:
a. Correlation plot by operating system - Binah.ai’s PR estimations versus reference device PR measurements
found to be very high (r=0.995) for both operating system (Android and iOS).
b. Bland-Altman plot by operating system - Bland-Altman plots for comparison between PR measurements of thetwo methods (Binah’s and the reference device) demonstrated high agreement between the two devices (99% ofthe measurements are within target error) for both operating systems (Android and iOS) in the presented pulserate range.
The “Bias” gray dashed line stands for the mean difference between measurements of Binah.ai and the reference device, the“Error” green dashed lines of ±3 bpm represent the value of the accuracy criterion, the “Limits of agreement” lines mark thelimit of 95% of the samples

Confidential

Page 5 of 11

Effective Date: 2024-02-01

Pulse Rate Accuracy Report of SDK 5.4
Version 3.0

MED-000003

Conclusions

This report summarizes the results of accuracy analysis in which the Pulse rate measured by Binah.ai’s
SDK was found to be robust, highly accurate and within the accuracy target (AE≤3 bpm in 95%) of themeasurements for both Android and iOS operating system.

References
1.

Betts, J. G. Heart Anatomy. in Anatomy & physiology 787–846 (2013).

2.
Zhang, G. Q. & Zhang, W. Heart rate, lifespan, and mortality risk. Ageing Res. Rev. 8, 52–60(2009).
3.

American Heart Association. All About Heart Rate (Pulse). (2017).

4.
Dyer, A. R. et al. Heart rate as a prognostic factor for coronary heart disease and mortality:
Findings in three Chicago epidemiologic studies. Am. J. Epidemiol. 112, 736–749 (1980).
5.
Kannel, W. B., Kannel, C., Paffenbarger, R. S. & Cupples, L. A. Heart rate and cardiovascularmortality: The Framingham study. Am. Heart J. 113, 1489–1494 (1987).
6.
Gillum, R. F., Makuc, D. M. & Feldman, J. J. Pulse rate, coronary heart disease, and death: TheNHANES I Epidemiologic Follow-up Study. Am. Heart J. 121, 172–177 (1991).
7.
Diaz, A., Bourassa, M. G., Guertin, M. C. & Tardif, J. C. Long-term prognostic value of restingheart rate in patients with suspected or proven coronary artery disease. Eur. Heart J. 26, 967–974(2005).

Confidential

Page 6 of 11

Effective Date: 2024-02-01

Pulse Rate Accuracy Report of SDK 5.4
Version 3.0

MED-000003

AppendixPulse rate error by Age and BMI

Figure 3:
a. Bland-Altman plot by age - demonstrated high agreement between PR measurements obtained by Binah.ai andthe reference device for both operating systems (Android and iOS) within the presented age
b. Bland-Altman plot by BMI - demonstrated high agreement between PR measurements obtained by Binah.ai andthe reference device for both operating systems (Android and iOS) within the presented BMI range from low to veryhigh.
The “Bias” gray dashed line stands for the mean difference between measurements of Binah.ai and the reference device, the“Error” green dashed lines of ±3 bpm represent the value of the accuracy criterion, the “Limits of agreement” lines mark thelimit of 95% of the samples.

ConfidentialPage 7 of 11

Effective Date: 2024-02-01

Pulse Rate Accuracy Report of SDK 5.4
Version 3.0MED-000003

Pulse rate error by skin tone with Gender and Operating system

Figure 4:a. Number of measurements by Fitzpatrick skin tone and sex (female and male).
b. Box plot by Fitzpatrick skin tone and Sex – PR measurements obtained by Binah.ai’s in comparison to thereference device are highly accurate for both sexes (female and male) across all presented skin tones.
The green dashed “Error” lines set at ≤3 bpm represent the value of the accuracy criterion.

c. Number of measurements by Fitzpatrick skin tone and operating system (Android and iOS).
d. Box plot by Fitzpatrick skin tone and operating system- PR measurements obtained by Binah.ai’s versus thereference device are highly accurate for both operating systems (Android and iOS) across all presented skin tones.
The green dashed “Error” lines set at ≤3 bpm represent the value of the accuracy criterion.

Confidential

Page 8 of 11

Effective Date: 2024-02-01

Pulse Rate Accuracy Report of SDK 5.4
Version 3.0

MED-000003

Pulse rate error by distance and luminance

Figure 5:a. Bland-Altman plot by Distance (m) - demonstrated high agreement between PR measurements obtained
by Binah.ai’s and the reference device for both operating systems (Android and iOS) within the presented
distance range from 0.20 to 0.45 m between the camera and the subject’s face.
b. Bland-Altman plot by Luminance (lux)- demonstrated high agreement between PR measurements
obtained by Binah.ai’s and the reference device for both operating systems (Android and iOS) within thepresented luminance range from dark surroundings (<150 lux) to brighter ones.
The gray dashed “Bias” line stands for the mean difference between measurements of Binah.ai and the reference device, theThe green dashed “Error” lines set at ±3 bpm represent the value of the accuracy criterion, the “Limits of agreement” lines
mark the limit of 95% of the samples.

Confidential

Page 9 of 11

Effective Date: 2024-02-01

Pulse Rate Accuracy Report of SDK 5.4
Version 3.0

MED-000003

Pulse rate error by Devices

Figure 6:
a. Box plot by device - PR measurements obtained by Binah.ai’s versus the reference device are highly
accurate for both sexes (female and male) on all devices.
The green dashed “Error” lines set at ≤3 bpm represent the value of the accuracy criterion.

Confidential

Page 10 of 11

Effective Date: 2024-02-01

Pulse Rate Accuracy Report of SDK 5.4
Version 3.0

MED-000003

Pulse rate error by Country

Figure 7:
a. Box plot by country - PR measurements obtained by Binah.ai’s versus the reference device are highly
accurate for both sexes (female and male) on all countries.
The green dashed “Error” lines set at ≤3 bpm represent the value of the accuracy criterion.

Confidential

Page 11 of 11

Effective Date: 2024-02-01