Source: https://patents.justia.com/patent/20190223737
Timestamp: 2019-09-21 22:25:08
Document Index: 660780214

Matched Legal Cases: ['Application No. 62', 'Application No. 62', 'Application No. 62', 'Application No. 62', 'Application No. 62', 'Application No. 62']

US Patent Application for Blood pressure from inward-facing head-mounted cameras Patent Application (Application #20190223737 issued July 25, 2019) - Justia Patents Search
Justia Patents US Patent Application for Blood pressure from inward-facing head-mounted cameras Patent Application (Application #20190223737)
Apr 4, 2019 - Facense Ltd.
Latest Facense Ltd. Patents:
U.S. Ser. No. 16/156,493, filed Oct. 10, 2018 is also a Continuation-In-Part of U.S. application Ser. No. 15/832,855, filed Dec. 6, 2017, now U.S. Pat. No. 10,130,308, which claims priority to U.S. Provisional Patent Application No. 62/456,105, filed Feb. 7, 2017, and U.S. Provisional Patent Application No. 62/480,496, filed Apr. 2, 2017, and U.S. Provisional Patent Application No. 62/566,572, filed Oct. 2, 2017.U.S. Ser. No. 15/832,855 is a Continuation-In-Part of U.S. application Ser. No. 15/182,592, filed Jun. 14, 2016, now U.S. Pat. No. 10,165,949, a Continuation-In-Part of U.S. application Ser. No. 15/231,276, filed Aug. 8, 2016,a Continuation-In-Part of U.S. application Ser. No. 15/284,528, filed Oct. 3, 2016, now U.S. Pat. No. 10,113,913, a Continuation-In-Part of U.S. application Ser. No. 15/635,178, filed Jun. 27, 2017, now U.S. Pat. No. 10,136,856, and a Continuation-In-Part of U.S. application Ser. No. 15/722,434, filed Oct. 2, 2017.
U.S. Ser. No. 15/284,528, filed Oct. 3, 2016, now U.S. Pat. No. 10,113,913, claims priority to U.S. Provisional Patent Application No. 62/236,868, filed Oct. 3, 2015, and U.S. Provisional Patent Application No. 62/354,833, filed Jun. 27, 2016, and U.S. Provisional Patent Application No. 62/372,063,filed Aug. 8, 2016.
Gil Thieberger would like to thank his holy and beloved teacher, Lama Dvora-hla, for her extraordinary teachings and manifestation of wisdom, love, compassion and morality, and for her endless efforts, support, and skills in guiding him and others on their paths to freedom and ultimate happiness. Gil would also like to thank his beloved parents for raising him exactly as they did.
In one embodiment, a system configured to calculate blood pressure includes at least a first inward-facing head-mounted camera (HCAM1) configured to capture images of a first region of interest (ROI1) located on the face below the eyes and above the lips (IMROI1) of a user, and a second inward-facing head-mounted camera (HCAM2) configured to capture images of a second region of interest (ROI2) comprising a portion of a temple and/or forehead (IMROI2) of the user. Optionally, the center of ROI1 is located more than 6 cm away from the center of ROI2; whereby changes in IMROI1 that are due to a cardiac pulse wave occur at least 10 ms before, or at least 10 ms after, changes in IMROI2 occur, which are due to the same cardiac pulse wave. The system also includes a computer configured to calculate a blood pressure value for the user based on first and second imaging photoplethysmographic signals (iPPG1, iPPG2) recognizable in IMROI1 and IMROI2, respectively. Optionally, iPPG1 and iPPG2 are indicative of pulse arrival times at ROI1 and ROI2, respectively. Optionally, the computer is further configured to extract iPPG1 and iPPG2 from IMROI1 and IMROI2, respectively, and to calculate the blood pressure value based on a magnitude of a difference in the pulse arrival times at ROI1 and ROI2. Optionally, the computer is further configured to utilize calibration measurements of the user's blood pressure, taken by a different device, to calculate parameters that are utilized by the computer to calculate the blood pressure value based on iPPG1 and iPPG2; whereby the calibration measurements are indicative of the differences between the pulse arrival times at ROI1 and ROI2 that correspond to different blood pressure values of the user. Optionally, the computer is configured to generate feature values based on data comprising IMROI1 and IMROI2, and to utilize a model to calculate the blood pressure value based on the feature values, where the model was generated based on training data comprising: previously taken IMROI1 and IMROI2, and blood pressure values measured at times corresponding to the previously taken IMROI1 and IMROI2.
In some embodiments, a device, such as a camera, may be positioned such that it occludes an ROI on the user's face, while in other embodiments, the device may be positioned such that it does not occlude the ROI. Sentences in the form of “the system/camera does not occlude the ROI” indicate that the ROI can be observed by a third person located in front of the user and looking at the ROI, such as illustrated by all the ROIs in FIG. 7 and FIG. 11. Sentences in the form of “the system/camera occludes the ROI” indicate that some of the ROIs cannot be observed directly by that third person, such as ROIs 19 and 37 that are occluded by the lenses in FIG. 1a, and ROIs 97 and 102 that are occluded by cameras 91 and 96, respectively, in FIG. 9.
FIG. 21 illustrates one embodiment of a system configured to calculate blood pressure, which includes inward-facing HCAMs as well as outward-facing HCAMs. The illustrated system includes frame 600, to which several HCAMs are coupled. These include inward-facing HCAMs 602a and 602b that are configured to capture images of ROIs 603a and 603b (portions of left and right sides of the forehead, respectively). Additionally, the illustrated system includes outward-facing HCAMs 604a and 604b. In the illustration, at least one of HCAMs 604a and 604b captures images that include the user's hand (ROI 605).
As opposed the ROIs on the face, which typically do not change their positon with respect to an inward-facing HCAM, an ROI that includes a portion of the hand may change its position in IMROI (due to movements of the head and/or hand), and may not appear in certain images at all. Thus, in some embodiments, to detect what portions of IMROI include exposed skin located between the wrist and the fingertips (e.g., palm or back of hand), and/or whether an image includes portions of the user's hand, the computer may utilize various image detection algorithms known in the art. Some examples of algorithmic approaches that may be utilized are described in Kölsch et al., “Robust Hand Detection.” FGR. 2004,which describe hand detection using a variant of the recognition method of Viola and Jones. Another approach to hand detection is given by Mittal et al., “Hand detection using multiple proposals”, BMVC, 2011, which describe a two-stage method for detecting hands and their orientation in unconstrained images. Additional methods for detecting hands in images are reviewed in Erol et al. “Vision-based hand pose estimation: A review”, Computer Vision and Image Understanding 108.1-2 (2007): 52-73.
The computer may utilize various preprocessing approaches to assist in calculations and/or in extraction of an iPPG signal from IMROI. Optionally, IMROI may undergo various preprocessing steps prior to being used by the computer to detect the physiological response, and/or as part of the process of the detection of the physiological response. Some non-limiting examples of the preprocessing include: normalization of pixel intensities (e.g., to obtain a zero-mean unit variance time series signal), and conditioning a time series signal by constructing a square wave, a sine wave, or a user defined shape, such as that obtained from an ECG signal or a PPG signal as described in U.S. Pat. No. 8,617,081.Additionally or alternatively, images may undergo various preprocessing to improve the signal, such as color space transformation (e.g., transforming RGB images into a monochromatic color or images in a different color space), blind source separation using algorithms such as independent component analysis (ICA) or principal component analysis (PCA), and various filtering techniques, such as detrending, bandpass filtering, and/or continuous wavelet transform (CWT). Various preprocessing techniques known in the art that may assist in extracting an iPPG signal from IMROI are discussed in Zaunseder et al. (2018), “Cardiovascular assessment by imaging photoplethysmography-a review”, Biomedical Engineering 63(5), 617-634. An example of preprocessing that may be used in some embodiments is given in U.S. Pat. No. 9,020,185, titled “Systems and methods for non-contact heart rate sensing”, which describes how a times-series signals obtained from video of a user can be filtered and processed to separate an underlying pulsing signal by, for example, using an ICA algorithm.
In another approach, the computer may utilize machine learning methods to calculate the blood pressure from IMROIs captured by HCAMs. In some embodiments, the computer calculates feature values based on data comprising IMROIs (e.g., IMROI and IMROI2 of one of the embodiments mentioned above) and utilizes a model to calculate, based on the feature values, the blood pressure value of the user. The following are some examples of the various types of feature values that may be generated based on IMROIs by the computer.
2. The system of claim 1, wherein iPPG1 and iPPG2 are indicative of pulse arrival times at ROI1 and ROI2, respectively; and wherein the computer is further configured to extract iPPG1 and iPPG2 from IMROI1 and IMROI2, respectively, and to calculate the blood pressure value based on a magnitude of a difference in the pulse arrival times at ROI1 and ROI2.
3. The system of claim 2, wherein the computer is further configured to utilize calibration measurements of the user's blood pressure, taken by a different device, to calculate parameters that are utilized by the computer to calculate the blood pressure value based on iPPG1 and iPPG2; whereby the calibration measurements are indicative of the differences between the pulse arrival times at ROI1 and ROI2 that correspond to different blood pressure values of the user.
4. The system of claim 1, wherein the blood pressure value is indicative of one or more of the following values: the systolic blood pressure of the user, the diastolic blood pressure of the user, and the mean arterial pressure of the user.
8. The system of claim 1, wherein the center of ROI1 is located more than 6 cm away from the center of ROI2; whereby changes in IMROI1 that are due to a cardiac pulse wave occur at least 10 ms before, or at least 10 ms after, changes in IMROI2 occur, which are due to the same cardiac pulse wave.
9. The system of claim 1, wherein the computer is further configured to generate feature values based on data comprising IMROI1 and IMROI2, and to utilize a model to calculate the blood pressure value based on the feature values; wherein the model was generated based on training data comprising: previously taken IMROI1 and IMROI2, and blood pressure values measured at times corresponding to the previously taken IMROI1 and IMROI2.
10. The system of claim 9, wherein iPPG1 and iPPG2 are indicative of pulse arrival times at ROI1 and ROI2, respectively; wherein the computer is further configured to extract iPPG1 and iPPG2 from IMROI1 and IMROI2, respectively; and wherein the feature values comprise a value that is indicative of a difference between when a pulse wave manifested in IMROI1 and IMROI2.
11. The system of claim 9, wherein iPPG1 and iPPG2 are indicative of pulse arrival times at ROI1 and ROI2, respectively; wherein the computer is further configured to extract iPPG1 and iPPG2 from IMROI1 and IMROI2, respectively; and wherein the feature values comprise one or more values that are indicative of: (i) a certain blood pressure value of the user that was measured during a certain previous period, and (ii) a difference between when pulse waves of the user manifested in IMROI1 and IMROI2 taken during the certain previous period.
14. The system of claim 13, further comprising an inward-facing head-mounted thermal camera configured to take thermal measurements of at least one of the following regions on the user's face: a periorbital region, a region on the forehead, and a region on the nose;
18. The system of claim 9, wherein the computer is further configured to: receive a value indicative of the movement of the user's body, and generate at least one of the feature values based on the received value.
Applicant: Facense Ltd. (Kiryat Tivon)
Inventors: Arie Tzvieli (Berkeley, CA), Ari M Frank (Haifa), Ori Tzvieli (Berkeley, CA), Gil Thieberger (Kiryat Tivon)
Application Number: 16/375,841
International Classification: A61B 5/024 (20060101); A61B 5/00 (20060101); A61B 5/0295 (20060101); G02B 27/01 (20060101);