Masking biometric markers by sensor path control

In accordance with some embodiments, an apparatus that controls sensor paths for privacy protection is provided. The apparatus includes a housing arranged to hold a second device. The apparatus obtains first sensor data that includes a biometric marker associated with a user. The apparatus controls sensor paths by obtaining the first sensor data using sensors on the second device, on the apparatus, and/or on a supplemental functional device. The apparatus further generates second sensor data by masking the biometric marker associated with the user in the first sensor data. The apparatus additionally controls the sensor paths by providing the second sensor data from the first apparatus to the second device.

TECHNICAL FIELD

This relates generally to the field of mobile device communication, and more specifically to an apparatus for controlling sensor paths on a personal communication device and masking biometric makers in sensor data collected by the personal communication device.

BACKGROUND

Various techniques allow extracting biometric information from sensor data (e.g., audio, image, vibration, IMU, etc.). One can then use the extracted biometric information to uniquely identify an individual, and in some cases, derive sensitive information about that particular individual. For example, voice data is typically considered unstructured data. Applying various techniques, telltale biometric markers (including health conditions of a user) can be extracted from voice utterances and/or speech samples. In another example, one can use images captured by cameras for body language mining, e.g., tracking habits or mood based on postures or facial expression in the image. As such, unbeknownst to the user, based on the information collected by sensors on personal communication devices, businesses can gain an unfair advantage over the individual. Moreover, in case malicious users obtain the derived biometric markers, the biometric markers can be used to defeat authentication methods in systems that utilize biometric authentication (e.g., iris scan, voice recognition, fingerprints).

In accordance with common practice the various features illustrated in the drawings cannot be drawn to scale. Accordingly, the dimensions of the various features can be arbitrarily expanded or reduced for clarity. In addition, some of the drawings cannot depict all of the components of a given system, method or device. Finally, like reference numerals can be used to denote like features throughout the specification and figures.

DETAILED DESCRIPTION

Systems, devices, and methods in accordance with embodiments disclosed herein protect individuals from private data mining. As explained above, a third party (e.g., a malicious user or a business) can derive biometric information from unstructured data and gain an unfair advantage over individuals. For example, an insurance company can use health conditions derived from voice utterance for determining insurance rate. Through an apparatus (also known as an active case, an active base, a smart case, or a safe case) disclosed herein, private individuals have more control over data captured by sensors on a personal communication device (e.g., a smartphone, a wearable device, or a tablet, etc.). Such data includes but is not limited to audio data captured by microphones on a smart phone, video data captured by cameras on a tablet, location data captured by GPS on a smart watch, etc.

For instance, in the case of audio data, the apparatus can obscure signals from a sound source, modify the background noise, morph the signals, and/or encrypt the signals before allowing transmission of such signals to a remote source. The morphed/modified voice of a user and/or masked ambient sound can change the biometric markers (e.g., age, gender, health, location, etc.) embedded in the audio data. In other words, the biometric markers embedded in sensor data can be obscured, e.g., modified acoustic data, generating a blurred image, etc. As such, the apparatus disclosed herein in accordance with embodiments protects user privacy and prevents misappropriation of private information. Moreover, through the apparatus disclosed herein, individuals have more control over private information revealed through sensor data and data paths connecting the sensors to the third party.

In accordance with some embodiments, a method is performed at a first apparatus that includes a housing arranged to hold a second device. The method includes obtaining first sensor data that includes a biometric marker associated with a user; generating second sensor data by obscuring the biometric marker associated with the user in the first sensor data; and providing the second sensor data from the first apparatus to the second device.

In accordance with some embodiments, a device includes one or more processors, non-transitory memory, and one or more programs; the one or more programs are stored in the non-transitory memory and configured to be executed by the one or more processors and the one or more programs include instructions for performing or causing performance of the operations of any of the methods described herein. In accordance with some embodiments, a non-transitory computer readable storage medium has stored therein instructions which when executed by one or more processors of a device, cause the device to perform or cause performance of the operations of any of the methods described herein. In accordance with some embodiments, a device includes means for performing or causing performance of the operations of any of the methods described herein.

In accordance with some embodiments, a device includes one or more processors, non-transitory memory, and one or more programs; the one or more programs are stored in the non-transitory memory and configured to be executed by the one or more processors and the one or more programs include instructions for performing or causing performance of the operations of any of the methods described herein. In accordance with some embodiments, a non-transitory computer readable storage medium has stored therein instructions which when executed by one or more processors of a device, cause the device to perform or cause performance of the operations of any of the methods described herein. In accordance with some embodiments, a device includes means for performing or causing performance of the operations of any of the methods described herein.

It should be appreciated that in the development of any actual embodiment (as in any development project), numerous decisions must be made to achieve the developers' specific goals (e.g., compliance with system and business-related constraints), and that these goals will vary from one embodiment to another. It will also be appreciated that such development efforts might be complex and time consuming, but would nevertheless be a routine undertaking for those of ordinary skill in the art of image capture having the benefit of this disclosure.

Referring toFIG.1, an exemplary operating environment100in which an active case120(also known as an active base, a smart case, or a safe case) controls sensor paths for privacy protection, in accordance with some embodiments. As will be explained below with reference toFIG.2, the active case120includes a housing arranged to hold a user equipment110. Further, the active case120includes a peripheral interface to connect to a supplemental functional device130(also known as a backpack). The sensors on the user equipment110, the active case120, and/or the backpack130can collect data associated with a user of the user equipment110. Such data reflect, for example, heart and/or pulse patterns101, gait102, fingerprints103, voice104, odor/scent105, facial image106of the user, among others. In some embodiments, biometric markers can be derived from the data collected by the sensors.

Biometric markers (or biometric identifiers) typically refer to the distinctive, measurable characteristics used to label and describe individuals. Biometric markers can reflect physiological and/or behavioral characteristics of individuals. Physiological characteristics are related to the function or shape of human body. Examples include, but are not limited to brain signal patterns, heart patterns, fingerprint, palm veins, face recognition, DNA, palm print, hand geometry, iris recognition, retina, and/or odor/scent. Behavioral characteristics are related to the pattern of behavior of a person, including but not limited to typing rhythm, gait, voice, RF emission pattern, and/or GPS location pattern of the personal.

In some embodiments, the active case120controls the sensor paths such that data obtained by the sensors are processed by the active case120. Biometric markers embedded in the sensor data are masked by the active case120. Further, as will be described below with reference toFIG.2, the active case120also controls communication paths. For instance, as shown inFIG.1, the active case120controls the transmission of the sensor data, such that the sensor data with biometric markers can be transmitted to a first remote source107-1, e.g., a secure server for authentication and/or access control. On the other hand, for privacy protection of the user, the sensor data with obscured biometric markers can be transmitted to a second remote source107-2, e.g., an unknown server.

Turning toFIG.2,FIG.2illustrates a block diagram200of the active case120holding the user equipment110and controlling sensor paths is depicted, in accordance with some embodiments. As will be described in further detail below, different from a conventional base or case that merely holds a user equipment, the active case120monitors and analyzes activities on the user equipment110and actively controls sensor paths and/or communication paths on the user equipment110. In some embodiments, the active case120includes a housing125arranged to hold the user equipment110(e.g., smartphone, wearable, tablet, etc.). In some embodiments, the housing125includes a plurality of components mateable with one another. In other words, the plurality of components, once mated with one another, form an assembly to hold and/or providing structural support of the user equipment110. The housing125allows a user to insert the user equipment110into the active case120for more protection of sensitive information or take the user equipment110out of the active case120for less monitoring of the user equipment110.

The active case120can have one or more moveable components (e.g., a hood) operable to slide to one or more positions (e.g., up or down) as well as non-moveable components. In such embodiments, the one or more moveable components, when in a first position (e.g., hood pushed down), are mateable (e.g., mechanically and/or electrically) with the non-moving components to form a housing assembly. The housing assembly forms an enclosure that at least partially support and hold the user equipment110. When in the first position, the housing125, along with other components of the active case120, protects the user equipment110against data mining, tracking, and/or spying, e.g., by audio jamming, camera covering, and/or RF shielding, etc. When the one or more moveable components of the housing assembly are in a second position (e.g., a hood slid up), a user can take the user equipment110out of the housing125and place the user equipment110in a non-protected mode.

In some embodiments, the active case120includes a plurality of sensors230. The plurality of sensors230include, for example, as one or more accelerometers, gyroscopes, and/or magnetometers (e.g., as part of an inertial measurement unit (IMU)202) for obtaining information concerning the position (e.g., altitude) of the user equipment110, light sensors204, acoustic sensors206(also known as audio sensors), touch sensors208, odor/scent sensors212, and/or heart/pulse sensors214, among others. The plurality of sensors230can be used independent of sensors on the user equipment110for collecting sensor data.

In some embodiments, the active case120includes memory225, which further includes one or more memory devices, including fixed and/or removable memory devices. In some embodiments, the memory225provides a non-transitory computer-readable storage medium for storing computer program instructions (e.g., a masking engine227) to be executed by the controller220. In some embodiments, the memory225stores sensor data, such as audio data, image data, location data, gait data, chemical data, health data. In some embodiments, when executed by the controller220, the masking engine227obscures the sensor data collected by the sensors230and provide obscured sensor data for user privacy protection.

In some embodiments, the active case120includes a peripheral interface150(e.g., a backpack interface or a backpack buss) to connect to the supplemental functional device160(e.g., a backpack). A supplemental functional device, as described herein, is a device connectable to the user equipment110through the active case120and provides supplemental functional functions to the user equipment110. The peripheral interface150connects the supplemental functional device160to the active case120. In some embodiments, the active case120also includes communication devices240, including one or more local communication devices242and/or one or more remote communication devices244. In some embodiments, the one or more local communication devices242relay messages from the peripheral interface150to the user equipment110and vice versa. As such, the peripheral interface150is a modular interface for the backpack160, which is a detachable device that allows supplemental hardware and software functionalities to be provided to the user.

In some embodiments, the housing125at least partially supports the peripheral interface150. For example, the peripheral interface150can include a number of connectors (e.g., contact pins or contact pads as indicated by the dots) connectable to the supplemental functional device160. In some embodiments, the connectors are affixed to the housing125and at least partially supported by the housing125. The connectors are mateable to an interface of the supplemental functional device160. In some embodiments, the peripheral interface150is wholly supported by the housing125, such that the peripheral interface150is integrated with or embedded in the housing125. In such embodiments, connectors from the supplemental functional device160can be plugged into the peripheral interface150in order to connect the supplemental functional device160to the active case120. In some embodiments, the peripheral interface150is operable to communicate with the supplemental functional device160via a physical wired channel, including communication connectors. The physical channel forms a secure communication path155between the active case120and the supplemental functional device160.

It should be noted that the peripheral interface150is not limited to physical connectors that can provide a wired connection. In some embodiments, the peripheral interface includes a wireless modem operable to wirelessly communicate with the supplemental functional device160. In some embodiments, the peripheral interface150is coupled to the communication devices240and leverages the wireless communication capability of the communication devices240to communicate with the supplemental functional device160. For example, the active case120can connect to a wireless communication enabled backpack device160through a wireless peripheral interface or through a wireless modem of the communication devices240. As such, a wireless communication enabled supplemental functional device160can communicate with the active case120without being in contact with the housing125or physically connected to the peripheral interface.

In some embodiments, the local communication device242includes a personal communication device interface modem (e.g., a WiFi modem, a BT/BLE radio, an infrared radio, an NFC radio, a Lightning® (a registered trademark of Apple Inc., Cupertino, Calif.) connector, etc.), among others. In some embodiments, the local communication device242is operable to provide a communication path (e.g., wirelessly or via physical connection) between the supplemental functional device160and the user equipment110. As such, in one direction, the communication path carries information from the user equipment110to the active case120for examination and masking in accordance with some embodiments. In the other direction, the communication path carries information from the active case120and/or the supplemental functional device160to the user equipment110in order to protect the user equipment110and/or supplement the functionality of the user equipment110. Additionally, in some embodiments, the communication path extends to include one or more remote communication paths with the remote source(s)107.

In some embodiments, the one or more remote communication devices244connect the active case120and the remote source(s)107wirelessly or through a wired connection. Wireless connection protocol can be, for example, Wi-Fi (e.g., IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, IEEE 802.11n, and/or IEEE 802.11ac), Bluetooth (BT), Bluetooth Low Energy (BLE), Near Field Communication (NFC), Global Positioning System (GPS), and/or cellular communication, including but not limited to long term evolution (LTE), wideband code division multiple access (W-CDMA), code division multiple access (CDMA), time division multiple access (TDMA), or Global System for Mobile Communications (GSM). The wired connection can be, for example, a Universal Serial Bus (USB) connector, a High Definition Multimedia Interface (HDMI) connector, and/or a Lightning® (a registered trademark of Apple Inc. of Cupertino, Calif.) connector.

In some embodiments, the active case120includes a controller220coupled to the peripheral interface150and the communication devices240. Embodiments of the controller220include hardware, software, firmware, or a combination thereof. In some embodiments, the controller220is operable to manage the communication channel between the user equipment110and the supplemental functional device160and through the communication devices240and the peripheral interface150. In other words, the controller220manages a segment of the communication path between the user equipment110and the active case120through the management of the one or more local communication devices242; and the controller220manages a segment of the communication path between the active case120and the supplemental functional device160through the management of the peripheral interface150. Additionally, in some embodiments, the controller220manages the extended communication path(s) associated with the remote source(s)107.

For example, when one remote source107(e.g., the second remote source107-2inFIG.1) attempts to communicate with the user equipment110held by the active case, the controller220can manage the communication path such that the RF signals transmitted or received by the user equipment110are degraded, e.g., jamming the RF signals. As a result, in case the remote source107is malicious, the degraded RF signals would be illegible. In another example, the controller220can also re-route the communication path, such that instead of allowing direct communication between the user equipment110and the remote source107, the controller220directs the one or more remote communication devices244to communicate with the remote source107on behalf of the user equipment110, e.g., providing obscured sensor data to the remote source107for user privacy protection.

In some embodiments, the active case120includes a power supply124. The power supply124supplies power to the peripheral interface, the communication devices240, and/or the controller220. In some embodiments, the power supply124can also supply power to the supplemental functional device160, e.g., passing energy through the wired or wireless connection with the supplemental functional device160. In some embodiments, the power supply124includes at least one of a battery, a charging socket, a USB connector, a power plug, and/or a power socket. In some embodiments, the power supply124includes a connector for a battery. In some embodiments, the power supply124includes a plurality of power supplying components, e.g., one battery providing power to the peripheral interface150, a power plug providing power to the communication devices240and/or the controller220, etc. The plurality of power supply124components can be connected to be charged together, charged separately, aggregating power to supply to one or more hardware electronic components of the active case120, or separately providing power to one or more hardware electronic components of the active case120.

In some embodiments, the supplemental functional device160includes a processing element250, such as an ASIC (Application Specific Integrated Circuit), portions or circuits of individual processor cores, entire processor cores, individual processors, programmable hardware devices such as a field programmable gate array (FPGA), and/or larger portions of systems that include multiple processors. In some embodiments, the supplemental functional device160also includes sensors252to collect information, such as sound, light, temperature, chemicals, odor/scent, drug, and/or biometrics measurement of a user, etc. In some embodiments, the supplemental functional device160additionally includes a radio frequency (RF) detection device256for detecting RF energy emission and transmission.

The information collected by the sensor(s)252and/or the RF detection device256are processed by the processing element250and communicated to the peripheral interface150of the active case120(e.g., a backpack bus) via the secure channel155, e.g., through wired connection between the peripheral interface150and an interface254(e.g., a backpack bus) on the supplemental functional device160. Upon receiving the information, the peripheral interface150of the active case120sends the information to the communication devices240under the management of the controller220, and the controller220further directs the information to the user equipment110in some embodiments. The additional information gathered by the supplemental functional device160supplements the functionality of the user equipment110. Moreover, in some embodiments, the active base120analyzes the additional information gathered by the supplemental functional device160and uses the information to further determine whether to obscure sensor data in order to protect the user equipment110.

For example, the sensors252can be biosensors for environmental monitoring, clinical diagnostics, and/or food analysis. The processing element250conducts preprocessing of the data gathered by the sensors252and prepares a summary of the data. The processing element250directs the summary data to the user equipment110through the secure channel155comprising the interface (e.g., the backpack bus)254, the peripheral interface150, and the one or more local communication devices242. Further, the active base120determines, based on the environmental monitoring (e.g., the presence of chemical, RF energy, infrared wave, a different set of user biometrics data, etc.), whether the user equipment110has been compromised. Thus, the additional data provided by the supplemental functional device160not only supplements the functionality of the user equipment110, but also enhances the function of the active case120.

Turning toFIGS.3A-3C,FIGS.3A-3Care block diagrams300A-300C illustrating embodiments of the active case120controlling sensor path and communication path associated with the user equipment for sensor data masking. In some embodiments, the user equipment110held by the active case120includes a processor, communication devices114, an input/output interface, sensors118, and memory for storing applications and instructions associated with the application. In some embodiments, the user equipment110is a portable communications device, such as a mobile phone, a wearable device, a tablet, a laptop computer, a digital media player, an electronic reader, or the like. In some embodiments, the user equipment110is a non-portable device, such as a desktop computer, a data storage device, a smart TV, a video game console, a smart home appliance or the like that is capable of storing, transmitting, and receiving data. It will be appreciated that the components, devices or elements illustrated in and described with respect toFIGS.3A-3Cmay not be mandatory and thus some may be omitted in certain embodiments. Additionally, some embodiments can include further or different components, devices or elements beyond those illustrated in and described with respect toFIGS.3A and3B.

In some embodiments, the communication devices114connect the user equipment110and an external electronic device wirelessly or through a wired connection. In some embodiments, the external electronic device is the active case120, such that the one or more communication devices114connect to the active case120wirelessly or through a wired communication. In some embodiments, the external electronic device is part of the remote source107. The wireless communication includes at least one of, for example, Wi-Fi (e.g., IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, IEEE 802.11n, and/or IEEE 802.11ac), Bluetooth (BT), Bluetooth Low Energy (BLE), Near Field Communication (NFC), Global Positioning System (GPS), and/or cellular communication, including but not limited to long term evolution (LTE), wideband code division multiple access (W-CDMA), code division multiple access (CDMA), time division multiple access (TDMA), or Global System for Mobile Communications (GSM). The wired connections include at least one of, for example, a Universal Serial Bus (USB) connector, a High Definition Multimedia Interface (HDMI) connector, and/or a Lightning® (a registered trademark of Apple Inc. of Cupertino, Calif.) connector.

In some embodiments, the user equipment110includes the sensors118, such as one or more accelerometers, gyroscopes, and/or magnetometers (e.g., as part of an inertial measurement unit (IMU)), light sensors, acoustic sensors, fingerprint sensors, touch sensors, heart/pulse sensors, gait sensors, among others. In some embodiments, the sensors118are coupled to the input/output interface, such that the information collected by the sensors118are passed to the processor by the input/output interface for further processing. For example, the input device camera uses light sensors for light sensing. In some embodiments, the sensors118are coupled to the one or more communication devices114, such that the information collected by the sensors118is transmitted to another device (e.g., the active case120and/or the remote source107).

In some embodiments, as shown inFIGS.3A and3B, the active case120allows collection of data by the sensors118. However, the active case120controls the sensor path, such the sensor data obtained by the sensors118are obtained by the active case120, e.g., via the communication devices114of the user equipment110and the communication devices240of the active case120. In some embodiments, upon obtaining the sensor data, the masking engine227of the active case120obscures the sensor data and generates obscured sensor data. In some embodiments, the obscured sensor data are generated by removing biometric markers in the sensor data. The obscured sensor data are then sent to the user equipment110, e.g., via the communication devices240of the active case120and the communication devices114of the user equipment110.

In some embodiments, as shown inFIG.3A, the active case120controls the communication path associated with the user equipment110such that a segment of the communication path between the user equipment110and the one or more remote resources107is allowed. In such embodiments, the active case120directs the communication devices114of the user equipment110to transmit obscured sensor data to the one or more remote sources107. In some embodiments, as shown inFIG.3B, the active case120controls the communication path associated with the user equipment110such that a segment of the communication path between the user equipment110and the one or more resources107is not allowed. In such embodiments, the communication devices240of the active case120transmit obscured sensor data to the one or more remote sources107. As such, in case the one or more remote sources107are insecure, by controlling the sensor path and the communication path, the active case120guards the sensor data with biometric markers for user privacy protection.

In some embodiments, as shown inFIG.3C, instead of allowing the sensors118on the user equipment110to collect sensor data, the active case120attenuates or blocks the ability of the sensors118on the user equipment110from collecting sensor data. Instead of obtaining the sensor data from the user equipment110, the active case120utilizes the sensors230on the active case120and/or the sensors252on the backpack for collecting the sensor data. For instance, the active case120may not have certain types of sensors or the backpack160has more sensitive sensors252that are capable of obtaining higher quality sensor data. In such embodiments, the backpack160can be attached to the active case120and the active case120can request the backpack160to obtain sensor data. Upon obtaining the sensor data, in some embodiments, the active case160can utilize the communication devices114on the user equipment and/or the communication device240on the active case120to communication with the remote sources107. As shown inFIG.3C, the sensor data with biometric markers are transmitted to a secure remote source, such as the first remote source107-1; while the sensor data with masked biometric makers are transmitted to an unknown remote source, such as the second remote source107-2.

For example,FIGS.4A and4Bare schematic diagrams400A and400B illustrating the active case120controlling light sensor path in accordance with some embodiments. As explained above, the active case120includes the housing125that receives and holds the user equipment110, which can include a camera420comprising light sensors. In some embodiments, the housing125is a housing assembly that further includes sub-assemblies, e.g., a plurality of both moveable parts and non-moveable parts that can form an enclosure when assembled together. For example, the sub-assemblies can include a base and a hood assembly410that is moveable. In some embodiments, as shown inFIG.4A, when the hood assembly410is moved/slid up or unclamped, the camera420on the user equipment110is unblocked and can record image data. When the hood assembly410is pushed down or lowered (as shown by the arrow inFIG.4A), the hood assembly410can block the camera on the user equipment110, as shown inFIG.4B. InFIG.4B, because the hood assembly410blocks the light, the light sensors on the user equipment110cannot detect light. As a result, the camera on the user equipment110cannot capture images to reveal user private information. In order to obtain image data for authentication purposes, the active case120can utilize sensors on the active case120and/or the backpack. For example, even in the dark, the active case120can use infrared light sensors on the active case120and/or the backpack to detect facial images for facial recognition purposes.

In another example,FIG.5Ais a block diagram500A andFIG.5Bis a cross-sectional view500B of a seal134, where both the block diagram500A and the cross-sectional view500B illustrate audio path control in accordance with some embodiments. In some embodiments, the user equipment110includes one or more input devices, e.g., the microphones142-1,142-2,142-3, and142-4. In some embodiments, the one or more input devices142include sensors that can obtain data from the surroundings. For example, the microphones142include acoustic or audio sensors that can record voice and/or ambient sound. As explained above the active case120includes the housing125that receives and holds the user equipment110. In some embodiments, the housing125also at least partially supports output devices (e.g., speakers130), the sensors230(not shown inFIG.3) and the communication devices240of the active case120. Further as explained above, in some embodiments, the housing125includes sub-assemblies, e.g., a plurality of both moveable parts and non-moveable parts that can form an enclosure when assembled together. In some embodiments, the sub-assemblies can include a base and a hood assembly that is moveable. For example, when the active case120in the privacy protection mode of operation, the hood assembly coordinated with the base engages audio seals134to mate the speakers130with the microphones142. The audio seals134provide sealing paths between the speakers130and the microphones142. Also in the privacy protection mode of operation, in some embodiments, the active case120generates masking signals. The masking signals are outputted from the speakers130, directed at the microphones142, and passed through the sealing paths.

In some embodiments, the seals134can be made of audio seals, structures, baffles, and/or sound isolating techniques known in the art to help reduce audio energy from outside the sealing paths reaching the speakers130. For example,FIG.5Bshows a cross-section of the audio seal134that forms part of an audio path. The audio seal134mates the speaker130with a microphone opening510, behind which a microphone142of the user equipment110is mounted. In some embodiments, the audio seal134is shaped to optimize the acoustical coupling to a targeted microphone of the user equipment110. This can be achieved by taking into account various factors including, but not limited to, the space available for the audio seal134, the surface material of the user equipment110or the speaker130, texture and form of an interface to which the audio seal134can mate, the acoustical path by which the targeted microphone detects audio content, and/or the level of sealing specified to meet the desired level of attenuation.

InFIG.5B, as a non-limiting example, the audio seal134is positioned between the microphone opening510and the speaker130. In order to cover the round opening of the microphone opening510, the audio seal134has a cut-out surrounded by wall. The cut-out forms a cavity or a chamber inside the wall in a shape of pipe, tube, or tunnel, and the cavity serves as part of the audio-sealing pathway for the audio signal from the speaker130to the microphone142. In some embodiments, the cavity is in the shape of cone, horn, or trumpet so that it amplifies the audio signal directed at the microphone142. In some embodiments, the audio seal134is made of foam material (e.g., polymer foam), flexible or compliant flexible material (e.g., elastomer, neoprene etc.), so that it seals the area surrounding the microphone opening510. The sealing provided by the audio seal134attenuates sound from entering the cavity and attenuates sound from leaking out of the cavity.

In some embodiments, the active case120includes the controller220that is at least partially supported by the housing125and coupled to the speakers130. In some embodiments, the controller220executes instructions stored in non-transitory memory (e.g., part of the memory225inFIG.2) to perform at least certain functions of audio path control, including sound masking. In some embodiments, in order to provide adaptive sound masking, the active case120also includes a plurality of input devices, e.g., microphones530-1,330-2,530-3, and530-4. The microphones530are at least partially supported by the housing125. In some embodiments, the microphones530record sound independent of the sound recorded by the microphones142on the user equipment110. In some embodiments, the active case120transmits the independently recorded sound to an external electronic device through a secure channel for secure communication and signal processing. In some embodiments, the sound recorded by the microphone(s)530is used by an envelope detector522included in the active case120to facilitate noise shaping.

In some embodiments, the envelope detector522is coupled to the microphones530and the controller220. In some embodiments, the envelope detector522includes an electronic circuit that takes audio signals (e.g., the ambient sound recorded by one or more of the microphones530) as an input and provides an output as an envelope associated with the input. The envelope detector522thus detects the amplitude variations of the incoming audible signals. In some embodiments, the envelope detector522outputs the envelope information to the controller220. Based on the envelope information, the controller220directs the speakers130to adjust the volume of the output audio signals from the speakers130appropriate for the level of ambient sound. Further, based on the envelope information, the controller220(e.g., the masking engine227inFIG.227) can direct the shaping of the audio signal, so that the shaped audio signal has frequency spectrum characterized by the current operating condition of the active case120.

By controlling the sensor path, sensor data with biometric markers are modified and obscured so that biometric markers are masked. For example,FIG.6Ais an audio signal power and frequency diagram600A illustrating a segment of exemplary sensor data. The segment includes a first portion610and a second portion620that may be associated with certain biometric markers. InFIG.6B, another audio signal power and frequency diagram600B, the sensor data are obscured to comprise the first portion610and a third portion630that is different from the second portion620(FIG.6A). As such, the biometric markers embedded in the segment of exemplary sensor data are masked. InFIG.6C, yet another audio signal power and frequency diagram600C, the exemplary sensor data have been further obscured, e.g., replaced by a different segment of sensor data. Accordingly, private information would not be revealed when the third party obtains the obscured sensor data.

FIG.7is a simplified block diagram700of biometric marker extraction and masking identity and authentication performed on an active case120-k, in accordance with some embodiments. In some embodiments, each of the active case120has an identifier and neural network with trained neural network parameters (e.g., NN parameters815,FIG.8) as well as feature vectors (e.g., the feature vectors816,FIG.8) specific to the active case120installed, e.g., receiving neural network parameters and a set of feature vectors [k] from a server as indicated by the dotted line and storing the received parameters and feature vectors [k]. The generation of the neural network parameters and the sent of feature vectors [k] is described in further detail below with reference toFIG.8.

While the user111accesses the user equipment110-kheld by the active case120-k, the sensors230(not shown) on the active case120and/or the sensors118(not shown) on the user equipment110(not shown) record the sensor data710, e.g., IMU data710-1, location data710-2, audio data (including voice data)710-3, Mth sensor data710-M, etc. After obtaining the sensor data710, in some embodiments, the active case120(e.g., a conditioning unit of the active case120) conditions the sensor data710, e.g., IMU signal conditioning720-1, location signal conditioning720-2, voice signal conditioning720-3, Mth feature signal conditioning720-M, etc. Further, the active case120(e.g., a feature extraction unit730of the active case120) extracts feature vectors from the conditioned sensor data using neural network parameters740received from the cloud in accordance with some embodiments.

ThoughFIG.7illustrates using the neural network parameters740for feature extraction, it should be appreciated that the embodiments are not limited to neural network described herein. A variety of feature extraction techniques can be used. For the sake of brevity, conventional techniques related to the signal processing and data transmission for obtaining the input data for feature extraction and the individual operating components of the machine learning may not be described in detail herein.

In some embodiments, using feature vector [k]750received from the cloud, the masking engine227of the active case120obscures sensor data, e.g., by replacing or removing certain biometric markers corresponding to feature vector [k]750from the sensor data. In some embodiments, the extracted feature vectors can be further used for authentication, e.g., by comparing the extracted feature vectors with feature vector [k]750. For example, the active case120-kcan generate an authentication score reflecting the similarities between the extracted feature vectors by the feature extraction unit730and the feature vector [k]750.

FIG.8is a simplified block diagram of a platform800for biometric authentication of users111of user equipment110, in accordance with some embodiments. In some embodiments, the platform800for biometric authentication includes a server810(e.g., a secure server providing Software as a service (SaaS) and/or the secure remote source107-1inFIG.1). In some embodiments, the server810further includes a neural network814for machine learning of sensor data obtained from a plurality of active cases120. ThoughFIG.8illustrates a neural network814for machine learning, it should be appreciated that the embodiments are not limited to the neural network described herein. A variety of machine learning techniques can be used. For the sake of brevity, conventional techniques related to the signal processing and data transmission for obtaining the input data for machine learning and the individual operating components of the machine learning may not be described in detail herein.

In the exemplary platform800, each user equipment110can be slid or inserted into a housing (e.g., the housing125inFIG.2) of the corresponding active case120. This action is indicated inFIG.8by the arrow depicting a movement from the user equipment110-kto the active case120-k. In some embodiments, during the enrollment phase, the user111accesses the user equipment110as usual, while the sensors230of the active case120and/or the sensors118of the user equipment110collect the sensor data to be communicated to the server810. In some embodiments, the server810stores the sensor data as feature data812in preparation for feature vector generation. After observing the user for a period of time, the server810learns characteristics from the sensor data by machine learning (e.g., by setting neural network parameters815in connection with neurons in the neural network814), and the server810stores the learned patterns in feature vectors816for future reference, e.g., for authentication or biometric marker masking as described above. During the authentication, the active case120compares the extracted feature vectors with the learned user model stored in the feature vectors136to make an authentication decision and gates access (e.g., sending electronic signal or sending password) to the user equipment110, e.g., locking or unlocking the user equipment110held by the active case120. In some embodiments, the active case120gates the access to another device different from the user equipment110. For example, based on the authentication decision, the active case120gates the access to a door or another remote device.

FIG.9is a flowchart representation of a method900for masking biometric markers, in accordance with some embodiments. In some embodiments, the method900is performed at a first apparatus (e.g., the active case120,FIG.1) with a housing (e.g., the housing125,FIG.2) arranged to hold a second device (e.g., the user equipment,FIG.2). In some embodiments, the first device also includes a controller (e.g., the controller220,FIG.2) for controlling sensor paths and communication paths and a non-transitory memory storing instructions for execution by the controller. In some embodiments, the biometric marker masking method900is performed by a masking engine (e.g., the masking engine227,FIG.2). Briefly, the method900includes obtaining first sensor data that includes a biometric marker associated with a user; generating second sensor data by masking the biometric marker associated with the user in the first sensor data; and providing the second sensor data from the first apparatus to the second device.

To that end, as represented by block910, the method900includes obtaining first sensor data that includes a biometric marker associated with a user. In some embodiments, as represented by block912, the biometric marker associated with the user identifies one or more of characteristics or status of the user. For example, as shown inFIG.1, the biometric marker can be a unique pattern of heart and/or pulse patterns101, gait102, fingerprints103, voice104, odor/scent105, and/or facial image106of a user. In other words, the biometric marker can be used to identify the user, reveal private information such as the health information, traits, behaviors, habits, or whereabouts of the user. As such, the biometric marker can reveal the user's private information.

The method900further includes, as represented by block920, generating second sensor data by masking the biometric marker associated with the user in the first sensor data. In some embodiments, as presented by block922, the first sensor data or the second sensor data includes one or more of sound (e.g., voice and/or ambient sound from the surrounding), image (e.g., facial image, fingerprint, and/or body pose, etc.), motion (e.g., gaits, gesture, body language, lip movements, and/or finger movement patterns on a touch sensitive surface, etc.), biometry (e.g., heart rate, pulse rhythm, and/or blood pressure patterns, etc.), chemical (e.g., odor, smell, scent, and/or drug composition, etc.), location (e.g., GPS), or telemetry (e.g., wireless and/or wired network connection) data. For example, as shown inFIG.7, sensor data are analyzed, features are identified and/or extracted, so that known features associated with the user of the active case k120-k, e.g., feature vector [k], are obscured or removed in order to mask the biometric markers associated with the user. In another example, as shown inFIGS.6A-6C, data representing a pattern of heart rhythm, gait, voice can be modified so that the pattern is no longer unique (e.g.,FIG.6Cshows a known pattern) and/or associated with the user (e.g., the pattern inFIG.6Bis different fromFIG.6A).

In some embodiments, as represented by block924, masking the biometric marker associated with the user in the first sensor data includes determining an appropriate level of obfuscation and masking the biometric marker in accordance with the appropriate level of obfuscation. For example, as shown inFIG.5A, the envelope detector522can be used to detect the level of ambient sound. Using the information detected by the envelope detector522, the controller220can direct the speakers130to output appropriate volume of masking sound and/or direct the masking engine to shape the output sound to the appropriate shape in order to mask the biometric marker in the audio signal captured by the microphones142of the user equipment110.

In some embodiments, as represented by block926, masking the biometric marker associated with the user in the first sensor data includes degrading one or more of reception by or transmission of the first sensor data from the second device. For example, inFIG.4B, when the hood assembly410blocks the light sensors of the camera420, the reception of image data is degraded. In another example, in case the first sensor data is location data, e.g., obtained through RF signal exchanges, jamming the RF signal can mask the biometric markers in the RF signals.

The method900continues, as presented by block930, with the first apparatus providing the second sensor data from the active case to the second device in accordance with some embodiments. As represented by block932, the active case controls the sensor paths so that the first sensor data can be obtained in accordance with various embodiments.

In one embodiment, as represented by block934, the active case can obtain the first sensor data by receiving from the second device, using a local communication channel, the first sensor data recorded by a sensor on the second device. For example, as shown inFIGS.3A and3B, the sensors118on the user equipment110obtains the sensor data, and the active case120obtains the sensor data from the user equipment110via the coupling of the communication devices240of the active case and the communication devices114of the user equipment110.

In another embodiment, as represented by block936, the active case utilizes the sensors on the active case for sensor data collection. In such embodiments, as represented by block937, the method900further includes establishing a first channel between the first apparatus and the second device, where the first channel includes a seal that at least partially block data collection by the second device from outside the first channel. Further, in such embodiments, obtaining the first sensor data includes obtaining the first sensor data using a sensor on the first apparatus; and providing the second sensor data from the first apparatus to the second device includes providing the second sensor data from the first apparatus to the second device through the first channel. For example, inFIGS.5A and5B, the seal134at least partially blocks the microphones142from receiving acoustic energy from outside the seal134. Thus, as shown inFIG.3C, the greyed-out sensors118(e.g., the microphones) on the user equipment110cannot obtain sensor data from outside the seal134. Inside the seal134and through the audio path formed by the seal134, the active case120passes the masking sound to the microphones142. In some embodiments, the masking sound is generated based in part on the audio data recorded by the microphones530of the active case120.

In yet another embodiment, as represented by block938, the active case utilizes the sensors on a third device (e.g., the backpack160,FIG.1) for sensor data collection. In such embodiment, the method900further includes establishing a second channel between the first apparatus and a third apparatus. Also in such embodiments, obtaining the first sensor data includes obtaining through the second channel the first sensor data, which is collected using a sensor on the third apparatus. For example, when the camera420of the user equipment110is blocked, the backpack with infrared camera can record infrared image data and pass the infrared image data to the active case.

Still referring toFIG.9, in some embodiments, as represented by block940, the method900further includes transmitting the first sensor data to a first remote source through a secure channel; and facilitating transmitting the second sensor data to a second remote source different from the first remote source. For example, as shown inFIGS.1and3C, the first sensor data with biometric markers embedded can be shared with the secure server107-1through a secure channel. On the other hand, an unknown remote source107-2receives the obscured sensor data with masked biometric markers embedded.

In some embodiments, as represented by block950, the method900further includes authenticating the user based on the first sensor data; and gating electronic access (e.g., allowing or denying the usage of the active case120and/or the user equipment110) to the second device based on whether or not the user is authenticated. For example, using an authentication system shown inFIG.8, sensor data from a plurality of users111are analyzed and features are extracted for authentication purpose.