Patent Description:
Generally, methods of non-invasively measuring blood pressure without damaging a human body include a method to measure blood pressure by measuring a cuff-based pressure and a method to estimate blood pressure by measuring a pulse wave without the use of a cuff. A Korotkoff-sound method is one of cuff-based blood pressure measurement methods, in which a pressure in a cuff wound around an upper arm is increased and blood pressure is measured by listening to the sound generated in the blood vessel through a stethoscope while decreasing the pressure. Another cuff-based blood pressure measurement method is an oscillometric method using an automated machine, in which a cuff is wound around an upper arm, a pressure in the cuff is increased, a pressure in the cuff is continuously measured while the cuff pressure is gradually decreased, and blood pressure is measured based on a point where a change in a pressure signal is large. Cuffless blood pressure measurement methods generally include a method of measuring blood pressure by calculating a pulse transit time (PTT) and a method a pulse wave analysis (PWA) method of estimating blood pressure by analyzing a shape of a pulse wave.

<CIT> discloses a system and method for cuff-less blood pressure measurement in a mobile device. The blood pressure measurement is performed at the fingertip of a subject and reflectance-mode photoplethysmography can be used for this measurement.

<CIT> discloses an apparatus for measuring bio-information including a pulse wave sensor configured to emit light having a plurality of wavelengths onto an object, and to detect a multi-wavelength pulse wave signal from the object, and a processor configured to obtain a contact pressure signal based on the multi-wavelength pulse wave signal, the contact pressure signal indicating a pressure between the object and the pulse wave sensor, and to generate information regarding a measurement state of the object based on the contact pressure signal.

<CIT> discloses an optical measurement device providing an illumination and detection assembly configured to generate and detect light of a predetermined wavelength range in the form of a photoplethysmography (PPG) signal, as well as a pressure detection assembly configured to detect an amount of pressure applied to the measurement device by the user being measured. A feedback unit, such as a portable display device, can be coupled to the measurement device to provide the user with real-time feedback of the detected PPG signal and level of applied pressure so that the user may adjust the amount of applied pressure to change the quality of the detected PPG signal.

The publication of <NPL> discloses smartphone-based blood pressure monitoring via the oscillometric finger-pressing method.

<CIT> discloses a biometric information measurement device and method.

It is the object of the present invention to provide an improved method for estimating bio-information in a precise manner.

According to an aspect of an example embodiment, an electronic device may include a main body; a sensor part disposed on a side of the main body; and a processor configured to control a display to display a first graphical object related to a contact state of a finger based on information related to the contact state of the finger received from the sensor part before measurement of a bio-signal; and control the display to display a second graphical object related to a contact force of the finger based on information related to the contact force of the finger received from the sensor part during measurement of the bio-signal.

The sensor part may include an optical sensor comprising a light source configured to emit light to the finger in contact with a finger contact surface and a photodetector configured to detect light scattered or reflected from the finger, and a force sensor configured to measure the contact force based on the finger contacting the finger contact surface.

In response to receiving a request for estimating bio-information, the processor is further configured to control the display to display the first graphical object representing an appearance of the main body and the sensor part, and a third graphical object representing an appearance of the finger that is in normal contact with the sensor part in the first graphical object.

The processor is further configured to control the display to display the first graphical object and repeatedly display or eliminate the third graphical object at least one or more times after a predetermined period of time.

The processor is further configured to determine whether the contact state of the finger is normal based on the information related to the contact state received from the sensor part.

The processor is further configured to control the display to display a third graphical object to induce a user to exert a force with the finger toward the sensor part based on the contact state being normal.

The third graphical object represents an appearance of the finger that repeatedly moves from a predetermined position spaced apart from the sensor part of the main body to a position of the sensor part.

The third graphical object includes an arrow directed toward the sensor part.

The processor is further configured to control the display to display a fourth graphical object indicating that the contact state is normal based on the contact state being normal.

The processor is further configured to display at least one of a fifth graphical object indicating that the contact state is abnormal, a sixth graphical object visually displaying a reason for the contact state being abnormal, and text describing a reason for the contact state being abnormal based on the contact state being abnormal.

The processor is further configured to display a seventh graphical object representing a range of a reference contact force that the finger is to exert to the sensor part based on the contact state being normal.

In response to receiving the contact force from the sensor part, the processor is further configured to control the display to display an eighth graphical object representing the contact force.

The processor is further configured to control the display to display the seventh graphical object, and display a gamified screen in which the eighth graphical object moves along the seventh graphical object in response to the contact force being received in real-time from the sensor part.

The processor is further configured to control a speaker to output a warning sound or control the display to display a ninth graphical object to warn a user based on the contact force being outside of the range of the reference contact force.

The processor is further configured to extract a feature based on the bio-signal measured by the sensor part, and estimate bio-information based on at least one of the extracted feature and the contact force.

According to an aspect of an example embodiment, a method of estimating bio-information which is performed by an electronic device comprising a sensor part disposed on a side of a main body and a processor may include acquiring, by the sensor part, a contact state of a finger before measurement of a bio-signal; controlling a display, by the processor, to display a first graphical object related to the contact state of the finger; acquiring, by the sensor part, a contact force of the finger during the measurement of the bio-signal; and controlling the display, by the processor, to display a second graphical object related to the contact force of the finger.

The method may include receiving a request for estimating bio-information; and controlling the display to display the first graphical object representing an appearance of the main body including the sensor part and a third graphical object representing an appearance of the finger that is in normal contact with the sensor part in the first graphical object.

The controlling the display to display the first graphical object related to the contact state comprises determining whether the contact state of the finger is normal based on the contact state.

The controlling the display to display the first graphical object related to the contact state comprises controlling the display to display a third graphical object to induce a user to exert a force with the finger toward the sensor part based on the contact state being normal.

The controlling the display to display the first graphical object related to the contact state comprises controlling the display to display at least one of a fifth graphical object indicating that the contact state is abnormal, a sixth graphical object visually displaying a reason for the contact state being abnormal, and text describing a reason for the contact state being abnormal based on the contact state being abnormal.

The controlling the display to display the second graphical object related to the contact force comprises displaying a seventh graphical object representing a range of a reference contact force that the finger is to exert to the sensor part based on the contact state being normal.

The controlling the display to display the second graphical object related to the contact force comprises, in response to receiving the contact force from the sensor part, controlling the display to display an eighth graphical object representing the contact force.

The method may include acquiring a bio-signal by the sensor part; and estimating bio-information based on the bio-signal and the contact force.

An electronic device may include a main body; a sensor part disposed on a side of the main body; and a processor provided inside the main body, electrically connected to the sensor part, and configured to control the sensor part, and process data received from the sensor part, wherein the sensor part comprises a housing disposed to be partially externally exposed from the side of the main body, a finger contact surface formed on an exposed surface of the housing to allow a finger to contact the finger contact surface, and an optical sensor disposed inside the housing.

The finger contact surface includes a convexly curved shape along a direction parallel to a length direction of the finger that is placed on and in contact with the finger contact surface.

The finger contact surface includes a convexly curved shape along a direction perpendicular to a length direction of the finger, or includes a shape having a flat top and curved surfaces on both sides.

The finger contact surface comprises a first light transmissive region and second light transmissive region that are formed on sides of the finger contact surface, and a third light transmissive region formed between the first light transmissive region and the second light transmissive region.

The optical sensor comprises a first light source, a second light source, and a photodetector provided between the first light source and the second light source.

The housing comprises a first light path configured to direct light emitted from the first light source toward the finger through the first light transmissive region, a second light path configured to direct light emitted from the second light source toward the finger through the second light transmissive region, and a third light path configured to direct light scattered or reflected from the finger toward the photodetector through the third light transmissive region.

The housing further comprises partition walls formed between the first light path and the second light path, and between the second light path and the third light path.

The electronic device further comprises a force sensor provided in the housing, and configured to measure a contact force applied by the finger to the finger contact surface.

The processor is configured to control the sensor part to operate in a general mode or in a bio-information estimation mode; and switch a mode of the sensor part to the bio-information estimation mode based on a user manipulation of the sensor part or an input of a request for estimating bio-information through a display mounted in the main body based on the sensor part being in the general mode.

The relative size and depiction of these elements, features, and structures may be exaggerated for clarity, illustration, and convenience.

Details of example embodiments are provided in the following detailed description with reference to the accompanying drawings. The disclosure may be understood more readily by reference to the following detailed description of example embodiments and the accompanying drawings. Rather, these embodiments are provided so that the disclosure will be thorough and complete and will fully convey the concept of the invention to those skilled in the art, and the invention will only be defined by the appended claims. Like reference numerals refer to like elements throughout the specification.

It will be understood that, although the terms "first," "second," etc. may be used herein to describe various elements, these elements should not be limited by these terms. Also, the singular forms of terms are intended to include the plural forms of the terms as well, unless the context clearly indicates otherwise. In the specification, unless explicitly described to the contrary, the word "comprise," and variations such as "comprises" or "comprising," will be understood to imply the inclusion of stated elements but not the exclusion of any other elements. Terms such as "unit" and "module" denote units that process at least one function or operation, and they may be implemented by using hardware, software, or a combination of hardware and software.

<FIG> is a block diagram illustrating an electronic device according to an embodiment. <FIG> are diagrams for describing a structure of a sensor part according to an embodiment. Example embodiments of a structure of an electronic device <NUM> will be described with reference to <FIG>.

The electronic device <NUM> according to example embodiments may be a smart watch or a smart band-type wearable device. However, the electronic device <NUM> is not limited thereto, and may be a mobile device, such as a smartphone or a tablet personal computer (PC).

Referring to <FIG>, the electronic device <NUM> may include a main body MB and a strap ST.

The main body MB may include modules for performing general functions of the electronic device <NUM> and a sensor part <NUM> for estimating bio-information. A battery may be embedded in the main body MB or the strap ST to supply power to various modules. The strap ST may be connected to the main body MB. The strap ST may be flexible so as to be bent around a user's wrist. The strap ST may include a first strap and a second strap that is separated from the first strap. Respective ends of the first strap and the second strap may be connected to each end of the main body MB, and the first strap and the second strap may be fastened to each other using fastening means formed on the other sides thereof. In this case, the fastening means may be formed as Velcro fastening, pin fastening, or the like, but is not limited thereto. In addition, the strap ST may be formed as one integrated piece, such as a band, which is not separated into pieces.

A display DP may be disposed on a top surface of the main body MB to visually display various types of information. The display DP may include a touch screen panel capable of receiving a touch input of a user.

The sensor part <NUM> is disposed on the side of the main body MB, possibly in the form of a button. The sensor part <NUM> may operate in a bio-information estimation mode and in a general mode under the control of a processor. When operating in the bio-information estimation mode, the sensor part <NUM> may acquire force information applied by an object to the sensor part <NUM> when the object is in contact with the sensor part <NUM>. Also, when the object is in contact with the sensor part <NUM>, the sensor part <NUM> may acquire light information reflected or scattered from the object. When operating in the general mode, the sensor part <NUM> may perform a user interface function for controlling general functions of the electronic device <NUM>, for example, selection/execution of an application, adjustment of a graphical user interface (GUI) of the display DP, and the like.

Referring to <FIG>, the sensor part <NUM> may include a housing HS. In addition, the sensor part <NUM> may include an optical sensor <NUM> and a force sensor <NUM> which are disposed inside of the housing HS or at a lower end of the housing HS.

The housing HS may have a part in the form of a button, which is externally exposed through the side of the main body MB. For example, a supporter SP inside the main body MB may support the housing HS from at least one of the periphery and the lower end of the housing HS. In the embodiment of <FIG>, the supporter SP is illustrated as surrounding the housing HS inside the main body MB, but this is merely an example. An additional structure for preventing the housing HS from being dislodged from the main body MB may be further included in the housing HS or inside the main body MB.

The housing HS may include a finger contact surface <NUM> which contacts a finger that is placed on and in contact with the finger contact surface <NUM>. <FIG> is a diagram illustrating an embodiment showing the shape (hereinafter referred to as a "plan view") of the finger contact surface <NUM> when the sensor part <NUM> of <FIG> is viewed in a direction (Z direction) perpendicular to the finger contact surface <NUM>. In the plan view of the finger contact surface <NUM>, a first axis traversing the center of the finger contact surface <NUM> may be a long axis and a second axis traversing the center of the finger contact surface <NUM> in a different direction from that of the first axis may be a short axis. The first axis and the second axis may be perpendicular to each other. Although not illustrated, in another example, the plan view of the finger contact surface <NUM> may have first and second axes that are equal in length. In addition, in <FIG>, the plan view of the finger contact surface <NUM> is illustrated as a rounded rectangle, but the plan view of the finger contact surface <NUM> may have other shapes, such as a normal rectangle, a square, an oval, a circle, and the like.

<FIG> is an exploded perspective view of the sensor part <NUM>. <FIG> illustrates a three-dimensional shape of the finger contact surface <NUM>.

A first-axis (A-A') cross-section of the finger contact surface <NUM> may be convexly curved in an outward direction of the main body MB. For example, the first-axis cross-section (A-A' cross-section) of the finger contact surface <NUM> may have a shape in which the height of the cross-section gradually decreases as the distance to the center of the finger contact surface <NUM> increases, as shown in (<NUM>) of <FIG>. For example, the first-axis cross-section of the finger contact surface <NUM> may have the same or similar shape to a portion of a circular or elliptical shape.

In another example, the first-axis cross-section of the finger contact surface <NUM> may have a shape in which the height of a given region of the cross-section from the center of an upper portion is horizontal and the height gradually decreases thereafter as the distance to the center increases. For example, the first-axis cross-section may have a shape in which the height gradually decreases in the form of a curve, as shown in (<NUM>) of <FIG>, or in a straight line, as shown in (<NUM>), after a given point at a predetermined radius from the center of the upper portion. In this case, the finger contact surface <NUM> may be gradually lowered in a curved or straight line from the upper portion to a given point and be vertically lowered after the given point to a bottom portion, or may be continuously lowered in a curved or straight line from the upper portion to the bottom portion.

In another example, the first-axis cross-section of the finger contact surface <NUM> may be a plane. For example, as shown in (<NUM>) of <FIG>, the first-axis cross-section of the finger contact surface <NUM> is horizontal and the left and right ends each may have a right-angled shape.

The examples of the shape of the first-axis cross-section of the finger contact surface <NUM> may also be applied as examples of the shape of a second-axis cross-section (B-B' cross-section). The first-axis cross-section and the second-axis cross-section may have the same shape or different shapes. For example, the first-axis cross-section and the second-axis cross-section may both have the same shape as (<NUM>) of <FIG>, or the first-axis cross-section may have the same shape as (<NUM>) of <FIG> and the second-axis cross-section may have the same shape as (<NUM>) of <FIG>. For example, in the exemplary embodiment of <FIG>, the first axis and the second axis of the finger contact surface <NUM> may each be arranged similarly to (<NUM>) of <FIG> and the first axis is longer than the second axis.

When the finger contact surface <NUM> forms a curved surface, a deeper deformation of the finger may be made, as compared to a case of a flat surface, when the finger is pressed with the same force. Accordingly, it is possible for the user to produce the same deformation of the finger by applying less force to the finger contact surface <NUM>.

Referring back to <FIG>, the finger contact surface <NUM> may include a first light transmissive region 12a formed on one part thereof, a second light transmissive region 12b spaced apart from the first light transmissive region 12a and formed on another part thereof, and a third light transmissive region 12c formed between the first light transmissive region 12a and the second light transmissive region 12b. The remaining region of the finger contact surface <NUM> may be a non-light-transmissive region. The first light transmissive region 12a, the second light transmissive region 12b, and the third light transmissive region 12c may each include holes formed in a circular, elliptical, or polygonal shape. In addition, each hole may be closed with a cover made of a transparent material, such as glass, plastic, or the like, to pass light therethrough. In this case, an individual cover may be configured to close each hole, or one cover integrally formed may be configured to cover all three holes. The first light transmissive region 12a, the second light transmissive region 12b, and the third light transmissive region 12c may be arranged on the first axis.

The optical sensor <NUM> may be disposed inside the housing HS. However, the embodiment is not limited thereto, and the optical sensor <NUM> may be disposed at a lower end outside the housing HS.

The optical sensor <NUM> may include light sources 21a and 21b that irradiate a finger when the finger is placed on and in contact with the finger contact surface <NUM>, and a photodetector <NUM> that detects light scattered or reflected by the tissue on the surface or inside of the finger that is irradiated by the light sources 21a and 21b.

The light sources 21a and 21b may include a first light source 21a and a second light source 21b disposed on both sides of a substrate of the optical sensor <NUM> as illustrated. However, the number of light sources is not limited. In this case, the light sources 21a and 21b may include at least one of a light-emitting diode (LED), a laser diode, and a phosphor, but are not limited thereto.

The first light source 21a and the second light source 21b may be configured to emit light of different wavelengths from each other. For example, both of the first light source 21a and the second light source 21b may emit light of infrared (IR) wavelength band or green wavelength band. Alternatively, one of the first light source 21a and the second light source 21b may emit light of infrared wavelength and the other may emit light of green wavelength. In addition, each of the light sources 21a and 21b may include a plurality of LEDs and the plurality of LEDs may all be configured to emit light of the same wavelength or some of the plurality of LEDs may be configured to emit light of different wavelengths. For example, the light source 21a may include an IR LED which emits light of an infrared wavelength and a green LED which emits light of a green wavelength, and the light source 21b may also include an IR LED and a green LED.

The photodetector <NUM> may be interposed between the first light source 21a and the second light source 21b on the substrate of the optical sensor <NUM>. The photodetector <NUM> may be a complementary metal-oxide semiconductor (CMOS) image sensor, but is not limited thereto such that the photodetector <NUM> may include a photodiode, a phototransistor (PTr), a charge-coupled device (CCD) image sensor, and the like. When the light scattered or reflected by the finger is detected, the photodetector <NUM> may convert the intensity of the light into electrical digital light signal data and transmit the digital light signal data to a processor.

In addition, the force sensor <NUM> may be disposed inside of the housing HS or at the bottom outside of the housing HS. The force sensor <NUM> may be laminated on the bottom or the top of the optical sensor <NUM>. The force sensor <NUM> may measure a pressing force of a finger in contact with the finger contact surface <NUM>. For example, the force sensor <NUM> may include a strain gauge, and measure the magnitude of force at which the user presses the sensor <NUM>.

<FIG> is a perspective view of a sensor part <NUM> according to another embodiment. In <FIG>, a first-axis cross section of a finger contact surface <NUM> has a similar shape to (<NUM>) of <FIG>, and a second-axis cross section has a similar shape to (<NUM>) of <FIG>. The finger contact surface <NUM> of <FIG> includes a first light transmissive region 12a, a second light transmissive region 12b, and a third light transmissive region 12c.

<FIG> is a cross-sectional view of the sensor part <NUM> shown in <FIG>. Specifically, <FIG> is a cross-sectional view of the sensor part <NUM> of <FIG> taken along a first axis of the finger contact surface <NUM>. Referring to <FIG>, a housing HS may include a first light path 13a and a second light path 13b that guide light emitted by a first light source 21a and a second light source 21b to pass through the first light transmissive region 12a and the second light transmissive region 12b and be directed toward the finger in contact with the finger contact surface <NUM>. In addition, the housing HS may include a third light path 13c that guides light to pass through the third light transmissive region 12c and be directed toward the photodetector <NUM> when the light, which is emitted by the first light source 21a and the second light source 21b, is scattered or reflected from the surface or internal tissue of the finger in contact with the finger contact surface <NUM>.

<FIG> is a cross-sectional view of a sensor part <NUM> according to another embodiment. Referring to <FIG>, a third light path 13c may further include an optical module <NUM>, for example, a lens, to condense light scattered or reflected from a finger toward a photodetector <NUM>. In addition, a filter may be disposed on the third light path 13c to pass light of a predefined wavelength and condense the light to the photodetector <NUM>.

Also, referring to <FIG> and <FIG>, the light paths 13a, 13b, and 13c of the housing HS may be partitioned from each other by partition walls <NUM>. The partition walls <NUM> may prevent the light emitted from the light sources 21a and 21b from directly entering the photodetector <NUM>. The partition walls <NUM> may be made of a non-light-transmissive material. The partition walls <NUM> may be manufactured integrally with the housing HS.

A processor <NUM> is embedded in the main body MB of the electronic device <NUM>. The processor <NUM> is electrically connected to the sensor part <NUM>. The processor <NUM> may control the optical sensor <NUM> and the force sensor <NUM>, receive measured data from the optical sensor <NUM> and the force sensor <NUM>, and process the received data.

The processor <NUM> may control the sensor part <NUM> to operate in a bio-information measurement mode or in a general mode.

For example, the processor <NUM> may operate in the general mode, and receive a command that the user inputs by manipulating the sensor part <NUM>, and process the received command. For example, when the user manipulates the sensor part <NUM> in the general mode or requests execution of a bio-information estimation application through a display DP capable of receiving touch input, an interface related to the bio-information estimation application may be output to the display DP by executing the bio-information estimation application.

When a request for estimating bio-information is received according to manipulation of the sensor part <NUM> in the general mode or manipulation of the display DP, the processor <NUM> may switch the mode of the sensor part <NUM> to the bio-information estimation mode and control the electrically connected light sources 21a and 21b, photodetector <NUM>, and force sensor <NUM>. For example, the processor may control the intensity of light, duration of light, and on/off statuses of the light sources 21a and 21b and power supply to the force sensor <NUM>.

When the processor <NUM> receives light signal data from the photodetector <NUM> and contact force data from the force sensor <NUM> in the bio-information estimation mode, the processor <NUM> may process the light signal data and the contact force data by executing, for example, a predefined bio-information estimation algorithm. For example, the processor <NUM> may monitor an environment for measuring bio-signal by using the light signal data and/or the contact force data, thereby guiding the user to maintain a normal measurement environment, and may estimate bio-information using a measured bio-signal.

The processor <NUM> may output a data processing result using various output modules of the electronic device <NUM>, for example, the display DP, a speaker, and the like. The processor <NUM> may visually display various graphical objects that guide the environment for measuring a bio-signal on the display DP, in which case the various graphical objects may be provided as a gamified screen or in the form of various graphs, so as to intuitively arouse a user's interest.

<FIG> and <FIG> are block diagrams illustrating an electronic device according to exemplary embodiments. <FIG> are diagrams illustrating exemplary embodiments in which a graphical object related to a contact state is displayed. <FIG> are diagrams illustrating exemplary embodiments in which a graphical object related to a contact force is displayed.

Referring to <FIG>, an electronic device <NUM> includes a main body provided in various shapes, and a sensor part <NUM> and a processor, which are disposed in the main body. In this case, the main body may be in the form of a smart watch as described with reference to <FIG>, but is not limited thereto, and may be provided in the form of a mobile device, such as smartphone or a tablet PC.

The sensor part <NUM> includes an optical sensor <NUM> and a force sensor <NUM>.

The optical sensor <NUM> may include one or more light sources configured to emit light to a finger when the finger comes in contact with a finger contact surface, and a photodetector configured to detect light scattered or reflected from the surface and/or internal tissue of the finger irradiated by the light sources. The one or more light sources may emit light of different wavelengths from each other.

The force sensor <NUM> may measure a contact force applied by the finger in contact with the finger contact surface to the finger contact surface.

The processor <NUM> may include a sensor controller <NUM> and a data processor <NUM>.

The sensor controller <NUM> may control the sensor part <NUM> to operate in a general mode or in a bio-information estimation mode. For example, when the electronic device <NUM> is driven, the sensor controller <NUM> may control the sensor part <NUM> in the general mode. When the user manipulates a button of the sensor part <NUM> in the general mode or requests estimation of bio-information by performing an action, such as touch/drag of a display, the mode of the sensor part <NUM> may be switched to the bio-information estimation mode. In addition, when the user inputs a predefined gesture through a camera module mounted in the electronic device <NUM> or inputs a voice command through a microphone mounted in the electronic device <NUM>, the mode of the sensor part <NUM> may be switched to the bio-information estimation mode.

When a request for estimating bio-information is received and accordingly the mode of the sensor part <NUM> is switched to the bio-information estimation mode, the sensor controller <NUM> may, for example, drive the light sources of the optical sensor <NUM> and control the intensity of electric current, duration, or the like, of the light sources. Also, the sensor controller <NUM> may supply power to various modules including the force sensor <NUM>.

When the sensor part <NUM> measures a bio-signal, the data processor <NUM> may monitor a measurement and guide the user to normally measure the bio-signal.

For example, when the sensor controller <NUM> switches the mode of the sensor part <NUM> to the bio-information estimation mode as described above, the data processor <NUM> displays a graphical object to induce the user to bring his/her finger properly into contact with the finger contact surface. Here, the graphical object may include, but is not limited to, text, characters, icons, images, figures, and the like.

Referring to <FIG>, the data processor <NUM> displays a first graphical object <NUM> representing the appearance of the main body on the display. In addition, the data processor <NUM> also displays a second graphical object <NUM> representing the appearance of the finger on the display to induce the user to properly touch a position of the sensor part SB with a measurement site of the finger, for example, the tip of the finger. In this case, text <NUM> may be displayed to induce the contact of the finger.

For example, the data processor <NUM> may simultaneously display the first graphical object <NUM> and the second graphical object <NUM> on the display by generating one graphical image including the first graphical object <NUM> and the second graphical object <NUM>. In another example, the second graphical object <NUM> and/or the text <NUM> may repeatedly appear and disappear once or more at specific time intervals after a predetermined period of time while the first graphical object <NUM> is being displayed.

In addition, when the user touches the finger contact surface of the sensor part <NUM> with the finger according to the guidance and accordingly contact state information is received from the optical sensor <NUM>, the data processor <NUM> determines whether the contact state of the finger is normal. For example, based on the intensity of light signal received from the optical sensor <NUM>, image data, fingerprint data, and the like, the data processor <NUM> may determine the contact state, such as whether the finger is in contact, the contact position, an initial contact force of the finger received from the force sensor <NUM>, or the like.

For example, when a predefined criterion is satisfied, for example, when the force sensor <NUM> measures a contact force that is greater than or equal to a predefined threshold value, measures a contact force for a period greater than or equal to a threshold period, or measures a contact force greater than or equal to the predefined threshold value for a period greater than or equal to a threshold period, the data processor <NUM> may determine that the contact state is normal. However, the embodiment is not limited thereto.

When it is determined that the contact state is normal, the data processor <NUM> displays a graphical object on the display to induce the measurement of a bio-signal. As shown in <FIG>, the first graphical object <NUM> representing the appearance of the main body and the second graphical object <NUM> representing the appearance of the finger that is in normal contact with the main body are displayed on the display. In this case, a sensor part SB in the first graphical object <NUM> may be emphasized in a different color.

In addition, a third graphical object <NUM> is displayed to induce the user to exert a force with the finger toward the sensor part SB. In this case, the third graphical object <NUM> may include an arrow superimposed on the second graphical object <NUM> as illustrated. Moreover, in another example, the third graphical object <NUM> may be modified from the second graphical object <NUM> such that an index finger repeatedly moves from a predetermined position spaced apart from the sensor part SB to the sensor part SB.

In addition, as shown in <FIG>, the processor <NUM> may display a fourth graphical object <NUM> indicating that the contact state is normal and/or text <NUM> requesting an action of pressing with a finger.

When it is determined that the contact state is abnormal, the data processor <NUM> may display a graphical object to induce the user to bring his/her finger again into contact with the finger contact surface. For example, as shown in <FIG>, a fifth graphical object <NUM> indicating that the contact state is abnormal (a) may be displayed. In addition, for example, (b) when a thumb is not in contact normally, (c) when an initial contact force falls out of a normal threshold, and (d) when a normal measurement site of an index finger (e.g., the tip of the index finger) is not in contact, texts <NUM>, <NUM>, and <NUM> each explaining the reason for being non-normal and/or sixth graphical objects <NUM>, <NUM>, and <NUM> each visually indicating the reason for being non-normal may be displayed.

When it is determined that the contact state is normal, the data processor <NUM> guides a contact force so that the user presses the sensor part <NUM> with his/her finger at an appropriate force while a bio-signal is being measured.

For example, <FIG> illustrates a screen in a measurement initial state in which an initial contact force is adjusted with a finger, and <FIG> illustrates a screen after the initial contact force falls within a normal range.

Referring to <FIG> and <FIG>, the data processor <NUM> may display seventh graphical objects <NUM> and <NUM> representing predefined reference contact forces and eighth graphical objects <NUM>, <NUM>, and <NUM> representing an actual contact force received from the force sensor <NUM>. For example, as illustrated, the upper limit <NUM> and the lower limit <NUM> of the seventh graphical objects may include lines, continuous points, circles, ellipses, polygons, and the like. Similarly, the eighth graphical objects <NUM>, <NUM>, and <NUM> may include lines, continuous points, circles, ellipses, polygons, and the like.

Referring to <FIG>, the data processor <NUM> may display the upper limit <NUM> and the lower limit <NUM> of the seventh graphical objects on a display screen in a horizontal direction at an initial stage of measurement. In addition, when the initial contact force received from the force sensor <NUM> falls out of the upper limit and/or the lower limit of the reference contact force, for example, when the initial contact force is not measured or is less than the lower limit of the reference contact force as illustrated, the eighth graphical object <NUM> may be displayed below the lower limit object <NUM>. Also, when the actual contact force falls within the normal range of the reference contact force as the user adjusts a force of a finger pressing the sensor part <NUM>, the eighth graphical object <NUM> may be displayed at a position between the upper limit object <NUM> and the lower limit object <NUM>, corresponding to the actual contact force. In this case, the eighth graphical objects <NUM> and <NUM> may be displayed as if moving, so that the change in the actual contact force is displayed. Further, the eighth graphical object <NUM> out of the normal range and the eighth graphical object <NUM> within the normal range may be distinguished from each other by different shapes or colors, so as to be easily recognizable by the user. For example, the eighth graphical object <NUM> out of the normal range may be displayed, for example, in red, and the eighth graphical object <NUM> within the normal range may be displayed, for example, in green.

Referring to <FIG>, when the actual contact force falls within the normal range at the initial stage of measurement, the data processor <NUM> may change the shapes of the seventh graphical objects <NUM> and <NUM> horizontally arranged as shown in <FIG> to gradual upward sloping curves, such that the user gradually increases the pressing force of the finger over time. In this way, the seventh graphical objects <NUM> and <NUM> may be changed to upward sloping shapes or downward sloping shapes according to the change in the reference contact force for a period when the bio-signal is measured.

Referring to <FIG> and <FIG>, the data processor <NUM> may construct a game screen and display the game screen on the display to arouse the user's interest and to guide the user to maintain the contact force within the normal range for the period of measurement while checking the contact force easily and intuitively. For example, as illustrated, seventh graphical objects <NUM> and <NUM> may be formed as characters of "street trees" so that a space between the upper limit <NUM> and the lower limit <NUM> of the reference contact force represents a "tree-lined road. " However, the exemplary embodiment is not limited thereto, such that types of characters representing the upper limit and the lower limit and the game screen are not particularly limited. In addition, the eighth graphical object <NUM> may be a game character, such as a "car," "bird," "animal," or the like, which moves along the "tree-lined road" in response to the actual contact force received in real-time from the force sensor <NUM>. However, the type of the game character is not particularly limited.

For example, the eighth graphical object <NUM> may move upward in the game screen as the actual contact force received from the force sensor <NUM> increases, and may move downward in the game screen as the actual contact force decreases. In this case, the seventh graphical object may include a game item <NUM> to provide the user with a benefit, such as a game score, when the actual contact force received from the force sensor <NUM> is maintained normally for a predetermined period of time. In addition, when the actual contact force received from the force sensor <NUM> falls out of the reference contact force, the data processor <NUM> may display a ninth graphical object <NUM> on the display to give a visual warning as shown in <FIG>. However, the embodiment is not limited thereto, and a warning sound may be output through a voice output module, such as a speaker.

In the foregoing description, the examples in which the data processor <NUM> sequentially displays the graphical objects of <FIG>, the graphical objects of <FIG>, and the graphical objects of <FIG> (or <FIG>) on the display according to the request for estimating bio-information and the measurement environment are described. However, the present disclosure is not limited thereto, and at least some of the graphical objects of <FIG>, the graphical objects of <FIG>, the graphical objects of <FIG>, and the graphical objects of <FIG> (or <FIG>) may be omitted. The invention remains however defined by the appended claims.

For example, when a request for estimating bio-information is received, the graphical objects of <FIG> may be displayed, and when the contact state is normal, the graphical objects of <FIG> (or <FIG>) may be immediately displayed. Alternatively, when the contact state is determined without displaying the graphical objects of <FIG> and the determination result shows that the contact state is normal, the graphical objects of <FIG> may be immediately displayed and then the graphical objects of <FIG> (or <FIG>) may be displayed. Alternatively, when the graphical objects of <FIG> and <FIG> are not displayed and it is determined that the contact state is normal, the graphical objects of <FIG> (or <FIG>) may be immediately displayed. Alternatively, when the graphical objects of <FIG> are not displayed and the contact state is not normal, the graphical objects of <FIG> may be displayed, and when the contact state becomes normal, the graphical objects of <FIG> (or <FIG>) may be immediately displayed.

The data processor <NUM> may receive a bio-signal from the optical sensor <NUM> and preprocess the received bio-signal. In this case, the bio-signal may include a photoplethysmogram (PPG) signal, an impedance plethysmogram (IPG) signal, a pressure wave signal, a video plethysmogram (VPG) signal, and the like. For example, when the biosignal is received, the data processor <NUM> may remove noise by performing, for example, bandpass filtering at <NUM>-<NUM>. Alternatively, bio-signal correction may be performed through fast Fourier transform-based reconstruction of the bio-signal. However, the embodiment is not limited thereto.

In addition, the data processor <NUM> may estimate bio-information on the basis of data received from the optical sensor <NUM> and the force sensor <NUM> by performing an algorithm for estimating bio-information. In this case, the bio-information may include, but is not limited to, mean blood pressure, systolic blood pressure, diastolic blood pressure, a vascular age, arterial stiffness, an aortic artery pressure waveform, a vascular elasticity, a stress index, and a fatigue level.

For example, the data processor <NUM> may extract features including a peak amplitude value, a time of a peak amplitude point, pulse waveform components, the area of a predetermined section of a bio-signal, and the like, from the bio-signal, and estimate the bio-information by combining one or more extracted features. In this case, the bio-information may be estimated by using a predefined bio-information estimation model. The bio-information estimation model may be defined as various linear or non-linear combination functions, such as addition, subtraction, division, multiplication, logarithmic value, regression equation, and the like, with no specific limitation.

In another example, the data processor <NUM> may acquire a contact pressure on the basis of the contact force received from the force sensor and the area of the finger contact surface of the sensor part <NUM>, and estimate a blood pressure on the basis of oscillometry based on a maximum peak point of the bio-signal and the contact pressure.

<FIG> illustrates an intensity of bio-signal measured by gradually increasing force in a state where the user touches the finger contact surface with an object. <FIG> illustrates an oscillometric envelope acquired based on an intensity of bio-signal and a contact pressure. For example, the data processor <NUM> may extract a peak-to-peak point by subtracting an amplitude value in3 at a minus (-) point from an amplitude value in2 at a plus (+) point of a waveform envelope in1 at each measurement point in time and acquire the oscillometric envelope OW by plotting the amplitude of the extracted peak-to-peak point at each measurement point in time based on a contact pressure at the corresponding measurement time point.

The data processor <NUM> may acquire features for estimating blood pressure from the acquired oscillometric envelope OW. For example, the features for estimating blood pressure may include an amplitude value MA at a maximum peak point, a contact pressure value MP at the maximum peak point, contact pressure values SP and DP that are on the left and right sides and predetermined proportions (e.g., <NUM> to <NUM>) of the contact pressure value MP at the maximum peak point. When the features are acquired, the data processor <NUM> may estimate blood pressure by applying the features to a predefined blood pressure estimation model.

Referring to <FIG>, the electronic device <NUM> may include a sensor part <NUM>, a processor <NUM>, a storage <NUM>, and a communication interface <NUM>. The sensor part <NUM> may include an optical sensor <NUM> and a force sensor <NUM>, and the processor <NUM> may include a sensor controller <NUM> and a data processor <NUM>. The sensor part <NUM> and the processor <NUM> are described in detail with reference to <FIG>, and hence the following description will be made based on details that are not redundant.

The data processor <NUM> displays graphical objects related to a contact state, a contact force, and the like, through a display as described above. In addition, when a bio-information estimate value is obtained, the data processor <NUM> may visually display the obtained bio-information estimate value through the display. In this case, when a bio-information estimation result falls out of a normal range, the data processor <NUM> may visually output alarm/warning information. Alternatively, the data processor <NUM> may non-visually output warning information related to a contact state, a contact force, and a bio-information estimate value in voice or through a non-visual output means, such as a haptic device.

The storage <NUM> may store reference information for estimating bio-information and a processing result of the sensor part <NUM> and/or the processor <NUM>. In this case, the reference information may include user information, such as a user's age, gender, health condition, and the like, a normal contact state, such as a contact position of a finger or the like, a condition for driving a light source, a reference contact force, a bio-information estimation model, and the like. However, the reference information is not limited to these examples.

The storage <NUM> may include at least one type of storage medium of a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, secure digital (SD) or extreme digital (XD) memory), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, and an optical disk, but is not limited thereto.

The communication interface <NUM> may communicate with an external device under the control of the processor <NUM> and transmit and receive various types of data related to bio-information estimation. For example, the communication interface <NUM> may transmit the processing result of the processor <NUM> to the external device, so that the external device performs management of bio-information history for the user, monitoring of the user's health condition, output of bio-information history and the health condition monitoring result, and the like. In this case, the external device may include, but not limited to, a smartphone, a tablet PC, a desktop computer, a notebook computer, and devices of a medical institution including a cuff-based blood pressure measurement device.

In another example, the communication interface <NUM> may receive a bio-information estimation model required for estimating bio-information, characteristic information of a user, and the like from the external device. The received information may be stored in the storage <NUM>.

The communication interface <NUM> may communicate with the external device by using Bluetooth communication, Bluetooth low energy (BLE) communication, near field communication (NFC), wireless local access network (WLAN) communication, ZigBee communication, infrared data association (IrDA) communication, wireless fidelity (Wi-Fi) Direct (WFD) communication, ultra-wideband (UWB) communication, Ant+ communication, Wi-Fi communication, radio frequency identification (RFID) communication, <NUM> communication, <NUM> communication, and/or <NUM> communication. However, these are merely examples, and the embodiment is not limited thereto.

<FIG> is a flowchart illustrating a method of estimating bio-information according to an exemplary embodiment.

The method of <FIG> may be one embodiment of a bio-information estimating method performed by the electronic device <NUM>/<NUM>/<NUM> according to the above-described embodiments. Hereinafter, the method will be described in brief to avoid redundancy.

First, when a finger is in contact with the sensor part, the electronic device acquires a contact state of the finger through the sensor part (operation <NUM>). Upon receiving a request for estimating bio-information, the electronic device displays a graphical object on a display to guide the finger to be in contact normally. The graphical object includes a graphical object representing the appearance of the main body and a graphical object representing the appearance of the finger to guide the finger to be in normal contact with the sensor part.

Then, based on information on a contact state of the finger acquired in operation <NUM>, it is determined whether or not the contact state is normal (operation <NUM>). For example, whether the contact state is normal or abnormal may be determined by analyzing whether an accurate measurement site of the finger is in contact with the sensor part, whether an initial contact force is within a preset threshold range, whether a contact force is measured for a threshold period of time, or the like.

Then, when it is determined that the contact state is abnormal (operation <NUM> - NO), a graphical object indicating that the contact state is abnormal may be displayed in order to guide the contact state to the normal state (operation <NUM>).

When it is determined that the contact state is normal (operation <NUM> - YES), a graphical object is displayed to induce the user to apply a pressure to the sensor part with a finger for measuring a bio-signal (operation <NUM>). For example, a graphical object representing a finger may repeatedly blink at a position of the sensor part or an arrow may be displayed, in order to emphasize the contact of the finger with the sensor part for the sake of easy recognition by the user.

Then, when a contact force of the finger is acquired (operation <NUM>), at the same time, a graphical object related to the contact force is displayed to induce the user to press the sensor part with a predefined normal force (operation <NUM>). A graphical object related to a reference contact force at which the finger must press the sensor part and a graphical object representing an actual contact force measured through the force sensor are displayed. In this case, the graphical objects may be presented as a gamified screen to induce the user to maintain the contact force while arousing the user's interest.

In addition, a bio-signal may be obtained through the sensor part while the user changes the contact force with the finger according to the guidance for the contact force (operation <NUM>).

Then, bio-information is estimated based on the bio-signal and the contact force (operation <NUM>). For example, as described above, a feature of a maximum peak point may be extracted from the bio-signal, and blood pressure may be estimated using the extracted feature and a bio-information estimation model, or a contact pressure may be obtained based on the contact force and blood pressure may be estimated based on oscillometry.

Then, a bio-information estimation result may be output (operation <NUM>). The bio-information estimation result may be visually output to the display, or non-visually output by using a speaker, a haptic module, or the like.

The example embodiments can be implemented as computer-readable code stored in a non-transitory computer-readable medium. Code and code segments constituting the computer program can be inferred by a computer programmer skilled in the art. The non-transitory computer-readable medium includes all types of record media in which computer-readable data are stored.

Examples of the non-transitory computer-readable medium include a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage. Further, the record medium may be implemented in the form of a carrier wave such as Internet transmission. In addition, the non-transitory computer-readable medium may be distributed to computer systems over a network, in which computer readable codes may be stored and executed in a distributed manner.

A number of examples have been described above. Nevertheless, it will be understood that various modifications may be made. For example, suitable results may be achieved if the described techniques are performed in a different order and/or if components in a described system, architecture, device, or circuit are combined in a different manner and/or replaced or supplemented by other components.

Claim 1:
A method of estimating bio-information which is performed by an electronic device (<NUM>; <NUM>) comprising a sensor part (<NUM>) disposed on a side of a main body (MB) with a display (DP), and a processor (<NUM>), the method comprising:
receiving a request for estimating bio-information;
controlling the display to display a first graphical object (<NUM>) representing an appearance of the main body including a sensor part (SB) and a second graphical object (<NUM>) representing an appearance of a finger to guide the finger to be in normal contact with the sensor part (SB);
acquiring (<NUM>), by the sensor part (<NUM>), a contact state of a finger before measurement of a bio-signal, wherein the sensor part comprises an optical sensor (<NUM>) and a force sensor (<NUM>) configured to measure the contact force based on the finger contacting the finger contact surface;
determining (<NUM>) whether the contact state of the finger is normal based on at least one of an analysis whether an initial contact force is within a preset threshold range, and an analysis whether a contact force is measured for a threshold period of time;
in response to the contact state being normal, controlling a display, by the processor (<NUM>), to display the first graphical object (<NUM>), the second graphical object (<NUM>), and a third graphical object (<NUM>) to induce the user to apply a pressure to the sensor part (SB) with the finger for measuring the bio-signal;
acquiring (<NUM>), by the sensor part (<NUM>), a contact force of the finger during the measurement of the bio-signal;
controlling (<NUM>) the display, by the processor (<NUM>), to display a seventh graphical object (<NUM>, <NUM>) related to a reference contact force at which the finger presses the sensor part (SB) and an eighth graphical object (<NUM>, <NUM>) representing an actual contact force measured through the force sensor to induce the user to press the sensor part (SB) with a predefined normal force;
acquiring (<NUM>) the bio-signal by the sensor part (SB); and
estimating bio-information (<NUM>) based on the bio-signal and the contact force.