Patent ID: 12235342

DETAILED DESCRIPTION

Principles of the present disclosure will be described below with reference to several example embodiments illustrated in the accompanying drawings. Although the drawings show example embodiments of the present disclosure, it should be understood that these embodiments are merely described to enable those skilled in the art to better understand and further implement the present disclosure, and not to limit the scope of the present disclosure in any way.

The term “include” and variants thereof used in this text indicate open-ended inclusion, that is, “including but not limited to.” Unless specifically stated, the term “or” means “and/or.” The term “based on” means “based at least in part on.” The terms “an example embodiment” and “an embodiment” indicate “at least one example embodiment.” The term “another embodiment” indicates “at least one additional embodiment.” The terms “first,” “second,” and the like may refer to different or identical objects. Other explicit and implicit definitions may also be included below.

As mentioned above, a radar technology can not only detect a direction and position of an object within a detection range, but also carry out biological recognition by means of detecting biometric features of biological objects. For example, a millimeter wave (mmWave) radar will emit electromagnetic waves, and any object in its path will reflect a signal back. By capturing and processing a reflected signal, a radar system can determine a distance, speed, and angle of the object. Since it can provide a millimeter-level accuracy in object distance detection, the mmWave radar becomes an ideal sensing technology for human biological signals. The mmWave radar can be used for monitoring vital signs, such as a breathing rate (BR) and a heart rate (HR). Because of differences between human beings, biological features of human beings can be determined through the vital signs for recognition and authentication.

In general, a biological recognition function of the radar has been used for authentication for some electronic devices. However, these authentications are very simple. Therefore, it is desired to provide more security functions by using the biological recognition function of the radar.

An embodiment of the present disclosure provides a solution for monitoring authentication based on a radar to solve one or more of the above problems and other potential problems. In this solution, when it is confirmed that an authenticated user is accessing an electronic device, a radar is used for detecting other people around the authenticated user. If it is detected that other people are too close and can see a display apparatus of the electronic device, the user is deauthenticated. For example, the display apparatus returns to an authentication interface to prevent other people from seeing contents displayed on the electronic device. In this way, it is possible to determine whether other people are trustworthy through the high-precision identification capability of the radar, thus avoiding other untrustworthy people from seeing the contents displayed on the electronic device, and improving the security of the user's access.

FIG.1illustrates a schematic diagram of example environment100in which embodiments of the present disclosure may be implemented. As described inFIG.1, radar apparatus120is installed on electronic device110. User140is accessing the display apparatus of electronic device110. User140may be viewed as an example of what is more generally referred to herein as a “first object.” User140is authenticated through, for example, a biological feature detected by radar apparatus120. Radar apparatus120transmits a radar field and forms detection range130. Radar apparatus120can detect positions of all objects within detection range130, and can detect biological signals of biological objects in detection range130. For example, in the example shown inFIG.1, radar apparatus120can detect person150-1and person150-2in detection range130. A given one of the persons150may be viewed as an example of what is more generally referred to herein as a “second object.”

Electronic device110may include, for example, a laptop computer, a television, a smart phone, a tablet computer, and a desktop computer, but can also use other devices such as a home automation and control system, an entertainment system, an audio system, a home appliance, a security system, a netbook, and an electronic reader. Electronic device110may be wearable or non-wearable, but mobile or relatively immobile (for example, a desktop computer and an entertainment system), and the present disclosure is not intended to limit this.

An example structure of radar apparatus120is shown inFIG.1. Radar apparatus120may be a part of electronic device110, or radar apparatus120may be separate from electronic device110.

Radar apparatus120includes radar transmitting element121, antenna element122, signal processor123, and communication apparatus124. Generally, radar transmitting element121provides a radar field. For example, the radar field may be configured to be reflected from a fabric (such as clothing). The radar field can also be configured to penetrate through fabrics or other obstacles and be reflected from human skins or tissues. These fabrics or obstacles may include wood, glass, plastic, cotton, wool, nylon, and similar fibers, and the radar field is reflected from human tissues (such as human faces).

The provided radar field may be a wide field, a narrow field, a surface field, a volume field, a shaping field (for example, a hemisphere, a cube, a sector, a cone, or a column), a steering field, a non-steering field, a short range (near) field, a medium range field, or a long range field. Therefore, the radar field provided by radar transmitting element121may be of a small size, such as about one millimeter to 1.5 meters, or may be of a medium size, such as about one to 30 meters. It should be understood that these sizes are only illustrative, and any other suitable sizes or ranges of the radar field may be used.

Radar transmitting element121may also use a continuous wave signal or a pulsed Doppler signal, and use various frequencies, update rates, pulse widths, interpulse periods (IPPs), transmitting power, and modulation. For example, radar transmitting element121may transmit continuously modulated radiation, ultra-wideband radiation, or sub-millimeter frequency radiation. In some embodiments, radar transmitting element121may form radiation in beams. The beams can help antenna element122and signal processor123to determine which of these beams are interrupted, and thus determine specific objects or interaction positions within the radar field.

Antenna element122can receive reflections from objects in the radar field or sense objects in the radar field. In some embodiments, the reflections include reflections from human tissues (such as a person's face or body) in the radar field or from movements of a person's head, legs, arms, hands, or torso. In addition, the reflections may also include reflections from clothing or other materials worn by the persons in the radar field.

Signal processor123can process the reflections received within the radar field to provide authentication data associated with the received reflections. The authentication data is data based on radar reflection, which can be used for determining whether an object in the radar field is a person, and in some implementations, for determining whether the person is a specific person who can access electronic device110(for example, to authenticate the person as an authorized user). In some embodiments, antenna element122may receive reflections from a plurality of human tissue targets in the radar field, and signal processor123is configured to process the received interactions sufficient to distinguish one of the plurality of human tissue targets from another of the plurality of human tissue targets. These targets may include the faces, heads, torsos, hands, arms, and legs from the same person or from different persons. As such, many different persons can be distinguished from one another.

Communication apparatus124may include a bus or a remote signal transceiver. Thus, radar apparatus120can be used as a part of electronic device110or as a wireless peripheral. The solution for monitoring authentication based on a radar according to the present disclosure will be described in detail below with reference toFIG.2toFIG.6.

FIG.2illustrates a flow chart of example method200for monitoring authentication based on a radar according to an embodiment of the present disclosure. Method200may be performed by, for example, electronic device110inFIG.1.

At202, electronic device110determines that a first object authenticated by the electronic device is accessing the electronic device. After the first object, e.g., the current user140of electronic device110, has been authenticated, electronic device110is unlocked, so that the first object can access and operate electronic device110. At this time, in order to avoid operation contents of the user from being seen by other people that may be untrustworthy, electronic device110enables an authentication monitoring function to perform deauthentication when there is a risk of leakage. The authentication monitoring function can be enabled in response to the electronic device being unlocked after the user's authentication succeeds, or it can be manually enabled or disabled by the user.

At204, electronic device110determines whether a second object is detected within a detection range using a radar apparatus of electronic device110. If a second object is detected, method200proceeds to206. If no second object is detected, electronic device110will continue to detect a second object, for example, performing periodic detection.

At206, electronic device110determines, based on a detected signal, that the second object is a person. After the authentication monitoring function is enabled, the radar apparatus (for example, radar apparatus120inFIG.1) of electronic device110provides a radar field and forms a certain detection range. Radar apparatus120detects a detected signal of an object by receiving a signal reflected by the object within the detection range. The detected signal may include a biological feature signal (such as a breathing rate signal and a heart rate signal of the object), and can also include an azimuth signal and other signals reflected from various parts of the object's body. When receiving the signal, electronic device110can determine, according to the signal, whether the object is a person. For example, the reflections from the object may be compared with stored reference data, and whether the object is a person can be determined based on a comparison result. For example, the stored reference data may include a radar reflection profile of an exemplary human body, which may be used for distinguishing reflections from a person from reflections from other objects. The stored reference data may also include user specific radar reflection profiles that can be used for identifying and distinguishing different persons.

At208, electronic device110determines a distance and an angle between the second object and electronic device110based on an azimuth signal in the detected signal. When a person within the detection range is detected, a relative position and a relative angle of the second object relative to electronic device110are detected from the azimuth signal. Thus, it can be determined whether the second object can see the display apparatus of electronic device110. Relative position relationships between various objects and electronic device110in this application scenario will be described below with reference toFIG.3.

FIG.3illustrates a schematic diagram of example scenario300for monitoring authentication based on a radar according to an embodiment of the present disclosure. As shown inFIG.3, user140(also referred to herein as “a first object”) is accessing electronic device110. At this time, radar apparatus120of electronic device110detects that there are also second object320-1passing behind user140and second object320-2(second object320-2and second object320-1are collectively referred to as second object320) close to user140within its detection range130. First threshold range310corresponding to a first distance threshold is also configured within detection range130. When second object320enters first threshold range310, it is considered that second object320can see the display apparatus of electronic device110in an appropriate direction. According to an azimuth signal in a detected signal for second object320-2, it is detected that second object320-2has entered first threshold range310, and angle A of second object320-2from a normal of the display apparatus of electronic device110is determined. According to a light emission range of a screen of electronic device110, second object320-2has a better viewing angle if angle A is smaller. Correspondingly, electronic device110also detects that second object320-1is not within first threshold range310and is far from first threshold range310. Therefore, it can be determined that second object320-1does not pose a risk.

Returning toFIG.2, after the distance and the angle between second object320and electronic device110are determined, whether there is a risk of leaking displayed contents to second object320will be determined. At210, electronic device110determines whether the determined distance is less than a first distance threshold. As discussed above, when the distance between second object320and electronic device110is less than the first distance threshold, it is considered that second object320can see electronic device110, and method200proceeds to212. If the distance is not less than the first distance threshold, the method proceeds to218.

At212, electronic device110determines whether the determined angle is less than an angle threshold. If the determined angle is less than the angle threshold value, it means that second object320faces to electronic device110, and can see the displayed contents on electronic device110. Method200proceeds to214. If the angle is not less than the angle threshold, the method proceeds to218. At214, electronic device110determines, based on a biological feature signal in the detected signal, whether second object320is trustworthy.

In some implementations, biological recognition may be performed using a radar-based heartbeat pattern recognition technology or a radar-based respiratory pattern recognition technology. For example, the radar-based respiratory pattern recognition technology can use stored information of respiratory patterns (or other biometric features, such as a heart profile) of detected historical objects, and use the stored respiratory patterns to recognize objects. In some embodiments, a biological feature may be a biometric feature of a person, such as a person's height, the size or length of a particular body part or bone (for example, from the tibia to the thigh, or from the upper arm to the lower arm), an arm or leg length, a hip width, or a shoulder width ratio.

In some embodiments, electronic device110may determine a biological feature of the second object based on the biological feature signal of the second object. Then, the determined biological feature is compared with reference biological features in a feature library. The second object is labeled as trustworthy if the determined biological feature is matched with a reference biological feature in a trusted list in the feature library. The second object is labeled as untrustworthy if the biological feature is not matched with any reference biological feature in the trusted list in the feature library. Later, the feature library will be described in detail with reference toFIG.6.

If second object320is determined to be untrustworthy, the method proceeds to216. At216, electronic device110deauthenticates the first object. For example, after the deauthentication of the first object, electronic device110is in a logout state. At this time, the user can return to the authentication interface or darken the screen. In some implementations, radar apparatus120may prompt the user and ask whether electronic device110should be locked (for example, via a voice or an on-screen prompt). Alternatively, when second object320leaves the first threshold range of electronic device110, electronic device110may automatically re-authenticate the first object, that is, user140.

In this way, by means of recognizing whether the second object can see the displayed contents and is trustworthy and performing corresponding operations on the authentication according to corresponding determining results, it can be ensured that the displayed contents on electronic device110are not leaked to an untrustworthy object, thereby improving the confidentiality of user access and operations.

In contrast, at218, electronic device110maintains the authentication of the first object based on the determination results. In this way, it can be ensured that the normal access of the user will not be interrupted, thus ensuring the user experience.

The biological feature detection function based on radar apparatus120can not only achieve biological recognition, but also determine the mood of the user of electronic device110according to the detected biological feature. The solution for monitoring mood based on a radar will be described in detail below with reference toFIG.4toFIG.5.

FIG.4is a flow chart of example method400for monitoring mood based on a radar according to an embodiment of the present disclosure. Method400may be performed by, for example, electronic device110inFIG.1.

At402, electronic device110determines that the distance between the second object and electronic device110is less than a second distance threshold. Relative position relationships between various objects and electronic device110in this application scenario will be described below with reference toFIG.5.

FIG.5illustrates a schematic diagram of example scenario500for monitoring mood based on a radar according to an embodiment of the present disclosure. As shown inFIG.5, user140is accessing electronic device110. At this time, radar apparatus120of electronic device110detects that there is also second object510behind user140within its detection range130. Second threshold range520corresponding to the second distance threshold is also configured within detection range130. When second object510enters second threshold range520, it is considered that second object510is too close to electronic device110and user140. In the embodiment shown inFIG.5, it is detected, according to an azimuth signal in a detected signal for second object510, that second object510has entered second threshold range520.

Returning toFIG.4, at404, electronic device110detects the first object using a radar to obtain a biological feature signal of user140. In some embodiments, the biological feature signal is, for example, a signal of a biological feature related to the mood, for example, a heart rate and a breathing rate.

At406, electronic device110determines a mood state of user140based on the biological feature signal and with a trained mood prediction model. The mood prediction model can determine, based on a pattern of the biological feature signal, a mood state corresponding to the pattern. An elevated heartbeat or a sharp change to a respiratory pattern/heartbeat can be a sign of a stress response and other mood states. Therefore, the mood prediction model can predict the mood of the first object through training.

At408, electronic device110determines whether the mood state of user140is a nervous state. When electronic device110acquires a current biological feature signal of user140, electronic device110can determine the mood state of user140based on a difference between the current biological feature signal and a stored biological feature signal. Referring toFIG.5again, electronic device110stores biological feature signals, i.e., heart rate chart530, of user140when the mood state is positive. The current biological feature signal acquired by electronic device110is heart rate chart540. By the difference between heart rate chart540and heart rate chart530, electronic device110can determine that heart rate chart540indicates that user140feels nervous. Therefore, it can be determined that user140currently feels nervous. At this time, it can be considered that user140may be threatened by second object510.

If user140is nervous, method400proceeds to410. If user140is not nervous, the method ends. At410, electronic device110sends an alarm signal. For example, electronic device110makes sounds to broadcast around that user140is threatened. In addition, electronic device110can also deauthenticate user140and lock electronic device110. Alternatively, electronic device110may send a distress signal to other electronic devices.

In this way, when other persons are too close to the user, the current mood of the user can be determined through the detected biological feature signal, and the electronic device can recognize that the user is under a threat and take corresponding measures, thus improving the safety of the user.

FIG.6illustrates a schematic diagram of feature library600according to an embodiment of the present disclosure. In some embodiments, feature library600is established in electronic device110. Feature library600stores detected signals of all detected objects. For example, when electronic device110receives a detected signal of a detected object from radar apparatus120, electronic device110sets a random name for the detected signal for recognition, and stores the detected signal in the form of a profile. In addition, other information is recorded in feature library600, such as the number of times of detections and a duration of stay in a predetermined range as shown inFIG.6. The number of times of detections and the duration of stay can reflect a relationship between the detected person and the user who is accessing electronic device110to a certain extent.

Trusted list610, untrusted list620, and to-be-confirmed list630are established in feature library600. When detecting an object for the first time, electronic device110will put the object into to-be-confirmed list630for subsequent placement in trusted list610or in untrusted list620. Trusted list610includes all objects labeled as trustworthy. For example, an object that has been detected for a large number of times or stayed for a long time can be considered to be closely related to the user and trusted by the user, and therefore put in trusted list610. In some embodiments, when user140is being authenticated, there is a second object in proximity. This shows that the first object allows the second object to see the displayed contents on electronic device110. Therefore, the second object is automatically put in the trusted list, for example, object C2inFIG.6. Untrusted list620includes all objects labeled as untrustworthy. In some embodiments, when electronic device110detects an object in untrusted list620, electronic device110can take more stringent protection measures.

In this way, more information can be provided for the authentication monitoring or mood monitoring function by means of preventing and maintaining all the objects detected within detection range130of radar apparatus120, which is conducive to the implementation of the two functions.

FIG.7illustrates a schematic block diagram of example device700that may be used to implement embodiments of the present disclosure. As shownFIG.7, device700includes central processing unit (CPU)701, which may execute various appropriate actions and processing in accordance with computer program instructions stored in read-only memory (ROM)702or computer program instructions loaded from storage unit708onto random access memory (RAM)703. Various programs and data required for the operation of device700may also be stored in RAM703. CPU701, ROM702, and RAM703are connected to each other through bus704. Input/Output (I/O) interface705is also connected to bus704.

A plurality of components in device700are connected to I/O interface705, including: input unit706, such as a keyboard and a mouse; output unit707, such as various types of displays and speakers; storage unit708, such as a magnetic disk and an optical disc; and communication unit709, such as a network card, a modem, and a wireless communication transceiver. Communication unit709allows device700to exchange information/data with other devices via a computer network, such as the Internet, and/or various telecommunication networks.

The various methods and processes described above, such as methods200and400, may be performed by CPU701. For example, in some embodiments, methods200and400may be implemented as a computer software program that is tangibly included in a machine-readable medium such as storage unit708. In some embodiments, part or all of the computer program may be loaded and/or installed onto device700via ROM702and/or communication unit709. When the computer program is loaded into RAM703and executed by CPU701, one or more steps of methods200and400described above can be implemented.

Illustrative embodiments of the present disclosure include a method, an apparatus, a system, and/or a computer program product. The computer program product may include a computer-readable storage medium on which computer-readable program instructions for performing various aspects of the present disclosure are loaded.

The computer-readable storage medium may be a tangible device that may retain and store instructions used by an instruction-executing device. For example, the computer-readable storage medium may be, but is not limited to, an electric storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium include: a portable computer disk, a hard disk, a RAM, a ROM, an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disc (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a raised structure in a groove with instructions stored thereon, and any suitable combination of the foregoing. The computer-readable storage medium used herein is not to be interpreted as transient signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber-optic cables), or electrical signals transmitted through electrical wires.

The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to various computing/processing devices or downloaded to an external computer or external storage device over a network, such as the Internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and/or edge servers. A network adapter card or network interface in each computing/processing device receives computer-readable program instructions from a network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing/processing device.

The computer program instructions for executing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or a plurality of programming languages, the programming languages including object-oriented programming languages such as Smalltalk and C++, and conventional procedural programming languages such as the C language or similar programming languages. The computer-readable program instructions may be executed entirely on a user computer, partly on a user computer, as a stand-alone software package, partly on a user computer and partly on a remote computer, or entirely on a remote computer or a server. In a case where a remote computer is involved, the remote computer can be connected to a user computer through any kind of networks, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, connected through the Internet using an Internet service provider). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), is customized by utilizing status information of the computer-readable program instructions. The electronic circuit may execute the computer-readable program instructions so as to implement various aspects of the present disclosure.

Various aspects of the present disclosure are described herein with reference to flow charts and/or block diagrams of the method, the apparatus (system), and the computer program product according to embodiments of the present disclosure. It should be understood that each block of the flow charts and/or the block diagrams and combinations of blocks in the flow charts and/or the block diagrams may be implemented by computer-readable program instructions.

These computer-readable program instructions may be provided to a processing unit of a general-purpose computer, a special-purpose computer, or a further programmable data processing apparatus, thereby producing a machine, such that these instructions, when executed by the processing unit of the computer or the further programmable data processing apparatus, produce means for implementing functions/actions specified in one or a plurality of blocks in the flow charts and/or block diagrams. These computer-readable program instructions may also be stored in a computer-readable storage medium, and these instructions cause a computer, a programmable data processing apparatus, and/or other devices to operate in a specific manner; and thus the computer-readable medium having instructions stored includes an article of manufacture that includes instructions that implement various aspects of the functions/actions specified in one or a plurality of blocks in the flow charts and/or block diagrams.

The computer-readable program instructions may also be loaded to a computer, a further programmable data processing apparatus, or a further device, so that a series of operating steps may be performed on the computer, the further programmable data processing apparatus, or the further device to produce a computer-implemented process, such that the instructions executed on the computer, the further programmable data processing apparatus, or the further device may implement the functions/actions specified in one or a plurality of blocks in the flow charts and/or block diagrams.

The flow charts and block diagrams in the drawings illustrate the architectures, functions, and operations of possible implementations of the systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flow charts or block diagrams may represent a module, a program segment, or part of an instruction, the module, program segment, or part of an instruction including one or a plurality of executable instructions for implementing specified logical functions. In some alternative implementations, functions marked in the blocks may also occur in an order different from that marked in the accompanying drawings. For example, two successive blocks may actually be executed in parallel substantially, and sometimes they may also be executed in a reverse order, which depends on involved functions. It should be further noted that each block in the block diagrams and/or flow charts as well as a combination of blocks in the block diagrams and/or flow charts may be implemented using a dedicated hardware-based system that executes specified functions or actions, or using a combination of special hardware and computer instructions.

Various embodiments of the present disclosure have been described above. The above description is illustrative, rather than exhaustive, and is not limited to the disclosed various embodiments. Numerous modifications and alterations will be apparent to persons of ordinary skill in the art without departing from the scope and spirit of the illustrated embodiments. The selection of terms used herein is intended to best explain the principles and practical applications of the various embodiments and their associated improvements, so as to enable persons of ordinary skill in the art to understand the embodiments disclosed herein.