Respiratory Rate Accuracy Report of SDK 5.4
Version 2.0

MED-000021

Respiratory Rate Accuracy Report of SDK 5.4
Executive Summary
Goal
This document evaluates the accuracy of Respiratory Rate in SDK 5.4 (Android and iOS) rPPG withreference devices, using data collected in Israel, India, South Africa, and Japan.

Results
The Respiratory rate measured by Binah’s SDK was found to be highly accurate and within the accuracy
target (AE≤3 rpm in >93%) of the measurements for Android and iOS. Similar results were found in thefollowing confounding factors (see appendix):● Both female and male
● All skin tones (Fitzpatrick I to VI)● Ages 19 to 77● BMI from light to morbid obesity● Distances close and far from the face
● Luminance from dark to brighter surroundings● Similar performance on all devices used for analysis.
● Similar performance in several countries with different ethnicities

Conclusions
The Respiratory rate measured by Binah’s SDK was found to be robust, highly accurate and within theaccuracy target (AE≤3 rpm in >93%) of the measurements for both Android and iOS operating system.

Introduction

The main function of the respiratory system is gas exchange. Oxygen is transferred from the environment
into the bloodstream, while carbon dioxide is expelled. When inhaling, the air passes to the lungs. Gasexchange occurs when oxygen diffuses into the lung capillaries in exchange with carbon dioxide.
Exhalation starts after the gas exchange, and the air containing carbon dioxide returns to the externalambient through the nose or mouth. In addition, the respiratory system has other secondary functions
including filtering, warming, and humidifying the inhaled air.1,2 There is a close relationship betweenrespiration and heart activity. Heart rate is regulated by respiration, increases during inhalation, anddecreases during exhalation3.
Respiration Rate (RR), defined as the number of respirations per minute (rpm), is a clinical parameter that
represents ventilation, i.e., the movement of air in and out of the lungs.1 The normal RR varies fromperson to person, but it generally lies between 12-20 respirations per minute at rest.4 RR is a valuable
diagnostic and prognostic marker of health used in a range of clinical settings to identify abnormalities.5Confidential

Page 1 of 11

Effective Date: 2024-02-01

Respiratory Rate Accuracy Report of SDK 5.4
Version 2.0

MED-000021

In hospital healthcare, it is a highly sensitive marker of acute deterioration.6 For instance, elevated RR is apredictor of cardiac arrest7 and in-hospital mortality8, and can indicate respiratory dysfunction9.
RR is usually still measured by manually counting chest wall movements (outside of intensive care). This
process is time consuming, inaccurate10,11, and poorly executed12,13. Therefore, there is a great need for anon-intrusive, automatic method of measuring RR.
Therefore, the advantage of a non-intrusive, automatic, and accessible method for monitoring these vital
signs is unquestionable.
Binah.ai’s algorithm uses the photoplethysmography (PPG) signal recorded from facial skin tissue (remote
PPG - rPPG). The algorithm extracts face video images, produces an rPPG signal, analyzes the data, andprovides the end user with vital signs measurements in real-time.
This report describes the results of accuracy studies conducted in Israel, India, South Africa, and Japanthat compares Binah.ai’s vital signs measurements with the measurements of approved reference
devices.

Methods

Binah.ai’s RR - measurements were compared to the Vernier Go Direct® Respiration Belt and/or finger
pulse oximeter measurements Masimo finger pulse oximeter measurements. The experiments wereconducted in Israel, India, South Africa, and Japan with both healthy participants and participants with a
medical background.

Measurement set-up:
In all sites, each participant was instructed to sit as stable as possible. Recordings were conducted in a
testing room, with controlled and fixed artificial ambient light.
The Respiration rate reference devices that were used included: the Vernier Go Direct® Respiration Belt
and/or Masimo finger pulse oximeter which were placed on each participant’s finger to measure RR.
For rPPG measurements, a mobile device was placed on a stand or held by the subject’s hand, in front of
the participant. The participant’s face filled most of the frame’s area (distance of about 20-40 cm) andwas positioned in the center of the frame. The camera was set at the level of the forehead and positioned
perpendicular to the face. Participants were instructed to look at the screen throughout recording.
Participants were instructed to take off their glasses and to avoid any movement, including talking, andwere required to sit still with their feet flat on the floor. Each recording lasted 60 seconds.

Confidential

Page 2 of 11

Effective Date: 2024-02-01

Respiratory Rate Accuracy Report of SDK 5.4
Version 2.0

MED-000021

Statistical analysis:
Accuracy was calculated using the following parameters:

𝐴𝐸 (𝐴𝑏𝑠𝑜𝑙𝑢𝑡𝑒 𝐸𝑟𝑟𝑜𝑟) = |𝐴𝑝𝑝𝑖 ― 𝑅𝑒𝑓𝑖|𝑅𝑀𝑆𝐸 =∑𝑁― 𝑅𝑒𝑓𝑖)2(𝑖=1 𝐴𝑝𝑝𝑖𝑁𝑀𝐴𝐸 =1
𝑁

𝑁|𝐴𝑝𝑝𝑖 ― 𝑅𝑒𝑓𝑖|
𝑖=1

When,
N is the number of data points.
App is the measurement of the Binah.ai application.
Ref is the measurement of the reference device.
i is the index number of the measurements.
Participants with invalid reference device values and participants with very low signal quality wereexcluded from the analysis, as the confidence level low results are not reported by the SDK.
For this report, Binah.ai’s SDK 5.4 was compared to a reference device.
The measurements were recorded in several locations in Israel, India, South Africa, and Japan using themobile device models listed below:-iOS: iPhone XR, iPhone 11 Pro, iPhone 13, iPhone 13 Pro and iPhone 13 Pro Max
Android: Samsung S10, Samsung S21 Ultra, Pixel 6 Pro, Huawei P30 Lite, Xiaomi Mi Note 10

Accuracy criteria:
RR: AE ≤ 3 rpm in 85% of measurements.

Confidential

Page 3 of 11

Effective Date: 2024-02-01

Respiratory Rate Accuracy Report of SDK 5.4
Version 2.0

MED-000021

Results
Measurement disposition
Number of measurements: 1515
Percentage of reported measurements (after signal quality check where confidence score is high): 90.6%
Number of subjects/measurements with reported HRV: 536/1372

Number of Unique Subjects and Measurements by Country and Respiration Rate distribution

Figure 1:
a. Number of Unique Subjects and Measurements by Country data presented includes all measurements withreference values.
b. Distribution of Respiration rate measured by reference device and Binah.ai’s application, both measurements
present overlapped normal distribution.

Demographics Data:
Subject/Measurements

Age(mean ± std)

BMI
(mean ± std)

Sex(F/M)

536 / 1372

37.5 ± 11.4

25.0 ± 5.2

272 / 264

Fitzpatrick Skin Tone(I/II/III/IV/V/VI)

Beard(No/Yes)

Glasses(No/Yes)

Face cream(No/Yes)

3 / 96 / 114 / 212 / 72 / 39329 / 182

329 / 12338 / 173

Distance(mean ± std)

Luminance(mean ± std)

Angle yaw
(mean ± std)

Angle roll(mean ± std)

Angle pitch(mean ± std)

0.23 ± 0.02

136.7 ± 134.4

5.0 ± 3.4

1.9 ± 1.5

10.9 ± 7.1

Table 1: Demographic data for experiments using phones with Android and iOS operating systems.
* Fitzpatrick skin tone classifications are I- Pale white, II- white, III- Darker white, IV- Light brown, V- Brown, VIDark brown or black.

Confidential

Page 4 of 11

Effective Date: 2024-02-01

Respiratory Rate Accuracy Report of SDK 5.4
Version 2.0

MED-000021

Accuracy Data:
Operation System

Measurements

MAE ± SD

Respiration Rate Range

AndroidiOS

657
715

1.1 ± 1.0
1.0 ± 1.0

8 - 298 - 26

Table 2: Accuracy data for Android and iOS (MAE±SD) when compared to the reference device in the presented
respiration rate range.
MAE -Mean Absolute Error, SD - Standard Deviation

Correlation and Bland-Altman plot by Operating System

Figure 2:a. Correlation plot by operating system - Binah.ai’s RR estimations versus reference device RR measurements
found to be with correlation of r= 0.909 for both operating system (Android and iOS).
b. Bland-Altman plot by operating system - Bland-Altman plots for comparison between RR measurements of thetwo methods (Binah’s and the reference device) demonstrated agreement between the two devices (93% of themeasurements are within target error) for both operating systems (Android and iOS) in the presented respiration
rate range.
The “Bias” gray dashed line stands for the mean difference between measurements of Binah.ai and the reference device, the“Error” green dashed lines of ±3 rpm represent the value of the accuracy criterion, the “Limits of agreement” lines mark thelimit of 95% of the samples.

Confidential

Page 5 of 11

Effective Date: 2024-02-01

Respiratory Rate Accuracy Report of SDK 5.4
Version 2.0

MED-000021

Conclusions
This report summarizes the results of accuracy analysis in which the Respiratory rate measured byBinah.ai’s SDK was found to be robust, highly accurate and within the accuracy target (AE≤3 rpm in 93%)of the measurements for both Android and iOS operating system.

References
1.
2.
3.
4.
5.
6.
7.
8.

9.
10.
11.

12.
13.

Confidential

Ali, M., Elsayed, A., Mendez, A., Savaria, Y. & Sawan, M. Contact and remote breathing ratemonitoring techniques: A review. IEEE Sens. J. (2021).
Tu, J., Inthavong, K. & Ahmadi, G. Computational fluid and particle dynamics in the humanrespiratory system. (Springer Science & Business Media, 2012).
Pallas-Areny, R., Colominas-Balague, J. & Rosell, F. J. The effect of respiration-induced heart
movements on the ECG. IEEE Trans. Biomed. Eng. 36, 585–590 (1989).
Barrett, K. E., Barman, S. M. & Boitano, S. Ganong’s review of medical physiology McGraw-Hill
Education. (2019).
Charlton, P. H. et al. Breathing rate estimation from the electrocardiogram andphotoplethysmogram: A review. IEEE Rev. Biomed. Eng. 11, 2–20 (2017).
Cretikos, M. et al. The objective medical emergency team activation criteria: a case–control study.
Resuscitation 73, 62–72 (2007).
Schein, R. M. H., Hazday, N., Pena, M., Ruben, B. H. & Sprung, C. L. Clinical antecedents to inhospital cardiopulmonary arrest. Chest 98, 1388–1392 (1990).
Duckitt, R. W. et al. Worthing physiological scoring system: derivation and validation of aphysiological early-warning system for medical admissions. An observational, population-based
single-centre study. Br. J. Anaesth. 98, 769–774 (2007).
Gravelyn, T. R. & Weg, J. G. Respiratory rate as an indicator of acute respiratory dysfunction.
Jama 244, 1123–1125 (1980).
Philip, K. E. J. et al. The accuracy of respiratory rate assessment by doctors in a London teaching
hospital: a cross-sectional study. J. Clin. Monit. Comput. 29, 455–460 (2015).
Lovett, P. B., Buchwald, J. M., Stürmann, K. & Bijur, P. The vexatious vital: neither clinicalmeasurements by nurses nor an electronic monitor provides accurate measurements of respiratory
rate in triage. Ann. Emerg. Med. 45, 68–76 (2005).
Pimentel, M. A. F., Charlton, P. H. & Clifton, D. A. Probabilistic estimation of respiratory ratefrom wearable sensors. in Wearable electronics sensors 241–262 (Springer, 2015).
Philip, K., Richardson, R. & Cohen, M. Staff perceptions of respiratory rate measurement in a
general hospital. Br. J. Nurs. 22, 570–574 (2013).

Page 6 of 11

Effective Date: 2024-02-01

Respiratory Rate Accuracy Report of SDK 5.4
Version 2.0

MED-000021

Appendix
Respiration Rate error by Age and BMI

Figure 3:
a. Bland-Altman plot by age - demonstrated RR measurements obtained by Binah.ai and the reference device forboth operating systems (Android and iOS) within the presented AGE range.
b. Bland-Altman plot by BMI - demonstrated RR measurements obtained by Binah.ai and the reference device forboth operating systems (Android and iOS) within the presented BMI range from low to very high.
The “Bias” gray dashed line stands for the mean difference between measurements of Binah.ai and the reference device, the“Error” green dashed lines of ±3 rpm represent the value of the accuracy criterion, the “Limits of agreement” lines mark thelimit of 95% of the samples.

Confidential

Page 7 of 11

Effective Date: 2024-02-01

Respiratory Rate Accuracy Report of SDK 5.4
Version 2.0MED-000021

Respiration Rate error by skin tone with Gender and Operating system

Figure 4:a. Number of measurements by Fitzpatrick skin tone and sex (female and male).
b. Box plot by Fitzpatrick Skin tone and Sex – RR measurements obtained by Binah.ai’s in comparison to thereference device are highly accurate for both sexes (female and male) across most of presented skin tones.
The green dashed “Error” lines set at ≤3 rpm represents the value of the accuracy criterionc. Number of measurements by Fitzpatrick skin tone and operating system (Android and iOS).
d. Box plot by Fitzpatrick skin tone and operating system- RR measurements obtained by Binah.ai’s versus thereference device are highly accurate for both operating systems (Android and iOS) across all presented skin tones.
The green dashed “Error” lines set at ≤3 rpm represents the value of the accuracy criterion.

ConfidentialPage 8 of 11

Effective Date: 2024-02-01

Respiratory Rate Accuracy Report of SDK 5.4
Version 2.0

MED-000021

Respiration Rate error by Distance and Luminance

Figure 5:a. Bland-Altman plot by Distance (m) - demonstrated agreement between RR measurements obtained by Binah.ai
and the reference device for both operating systems (Android and iOS) within the presented distance range
between the camera and the subject’s face.
b. Bland-Altman plot by Luminance (lux) - demonstrated agreement between RR measurements obtained byBinah.ai’s and the reference device for both operating systems (Android and iOS) within the presented luminancerange from dark surroundings (<150 lux) to brighter ones.
The “Bias” gray dashed line stands for the mean difference between measurements of Binah.ai and the reference device, the“Error” green dashed lines of ±3 rpm represent the value of the accuracy criterion, the “Limits of agreement” lines mark thelimit of 95% of the samples.

ConfidentialPage 9 of 11

Effective Date: 2024-02-01

Respiratory Rate Accuracy Report of SDK 5.4
Version 2.0

MED-000021

Respiration Rate Error by Devices

Figure 6:
a. Box plot by device - RR measurements obtained by Binah.ai versus the reference device for both sexes (femaleand male) on all devices.
The green dashed “Error” lines set at ≤3 rpm represents the value of the accuracy criterion

Confidential

Page 10 of 11

Effective Date: 2024-02-01

Respiratory Rate Accuracy Report of SDK 5.4
Version 2.0

MED-000021

Respiration Rate Error by Country

Figure 7:
a. Box plot by country - RR measurements obtained by Binah.ai versus the reference device for both sexes (femaleand male) in all countries.
The green dashed “Error” lines set at ≤3 rpm represents the value of the accuracy criterion.

Confidential

Page 11 of 11

Effective Date: 2024-02-01