Patent Description:
Proximity sensing is an important feature for touchscreen based mobile devices to detect the close presence of a part of the body of the user to, in the case of a user face, switch off the touchscreen display to prevent false touch events during voice calls, or, in the case of a user hand, to switch on the touchscreen display to enable touch events during device operation. Current mobile devices use infrared (IR) sensors to perform proximity sensing, which require area(s) adjacent the touchscreen on the front of the device. With devices trending toward significantly less frontal area available for a bezel, loss of space for IR sensors will require an alternative proximity sensing technique.

<CIT> discloses a method and a device for detecting if an object is in proximity to the device, wherein sound (audio) transducers already found in the device are used to realize the proximity detection function, along with digital signal processing, or equivalent means.

<CIT> discloses a driver circuitry for driving an electroacoustic transducer to provide an output comprising both ultrasonic and audio signal components.

<CIT> discloses a method and circuit for accurately determining the leading edge timing of a pulse of high frequency carrier signal at a receiver which substantially simultaneously receives the same pulse over multiple transmission paths resulting in distortion of the pulse envelope.

Advantageous, optional features of the invention are then set out in the appended dependent claims. In the following description, any embodiment referred to and not falling within the scope of the claims is merely an example useful to the understanding of the invention.

With respect to the discussion to follow and in particular to the drawings, it is stressed that the particulars shown represent examples for purposes of illustrative discussion and are presented in the cause of providing a description of principles and conceptual aspects of the present disclosure. In this regard, no attempt is made to show implementation details beyond what is needed for a fundamental understanding of the present disclosure. The discussion to follow, in conjunction with the drawings, makes apparent to those of skill in the art how embodiments in accordance with the present disclosure may be practiced. In the accompanying drawings:.

In the following description, for purposes of explanation, numerous examples and specific details are set forth in order to provide a thorough understanding of the present disclosure. It will be evident, however, to one skilled in the art that the present disclosure as expressed in the claims may include some or all of the features in these examples, alone or in combination with other features described below, and may further include modifications and equivalents of the features and concepts described herein.

As discussed in more detail below, proximity detection may be achieved using near-ultrasonic sound signals, which are inaudible in a frequency range from <NUM> to <NUM>. This technique avoids need for dedicated area(s) on the face of the host device for IR sensors since the built-in ear piece (EP) speaker may be used to emit the inaudible near-ultrasonic frequency sound waves along with normal speech and/or music.

<FIG> illustrates signal paths for incident <NUM> and reflected <NUM> sound waves according to one embodiment. The direction of the signal <NUM> emitted from the EP speaker <NUM> of the device <NUM> may be directed (e.g., beam formed) toward the head or ear of the user <NUM>. The high frequency (HF) sound waves <NUM> reflect <NUM> from the user <NUM> and are captured by the built-in microphone <NUM>. The cumulative strength of the reflected waves <NUM> is proportional to the proximity of the head of the user <NUM>, and the energy of the reflected HF sound may be used is a cue for detecting that proximity.

As discussed in more detail below, proximity of a head/object may be detected using near ultrasonic sound waves along with an adaptive acoustic sensing algorithm that determines the echo reflection profile of the surroundings (e.g., changes based on sound-absorption characteristics of surrounding objects) and adapts the detection and its parameters (e.g., idle signal floor, sensitivity, etc.). This adaptive acoustic sensing also enables detection of high intensity environmental noise that can interfere with the near ultrasonic excitation signal (ES) and provide a corrective path to minimize false triggers.

<FIG> illustrates block diagram <NUM> of near-ultrasonic excitation <NUM> and proximity computation <NUM> engines according to one embodiment. The excitation engine <NUM> includes a near-ultrasonic tone generator <NUM>, an adaptive gain controller <NUM> and a tone mixer <NUM>. The tone generator <NUM> produces an inaudible near-ultrasonic tone (e.g., <NUM>) <NUM> having a signal power or gain that may be adaptively controlled by the gain controller <NUM> to produce a controlled near-ultrasonic tone <NUM> that may be combined in the mixer <NUM> with an audible signal <NUM> to produce a drive signal <NUM> for the EP speaker <NUM>. Other elements of the host device include a mixer <NUM> for combining outgoing speech 217a and/or music 217b streams, and a band limiter (e.g., a lowpass filter) <NUM> for limiting the upper frequency range of the mixed signal(s) <NUM> to produce the audible signal <NUM>.

The near-ultrasonic tone generation may be adaptively controlled by received signal energy of the echo waves <NUM> received and converted by the microphone <NUM> to signals <NUM> processed by the proximity computation engine <NUM>. The waves <NUM> emitted from the EP speaker <NUM> are reflected by various obstacles in its path (including head/face of the user <NUM>) and are captured by the microphone <NUM> as echoes <NUM>. These echoes are further processed to detect the proximity of the head/face.

The near-ultrasonic proximity computation engine <NUM> includes a preprocessing engine <NUM>, a proximity sensing engine <NUM> and a proximity detection engine <NUM>. Together, these perform adaptive acoustic sensing of the echo reflection profile as affected by acoustic properties of the operating environment (which may vary in real time based on sound absorption characteristics of surrounding materials, size of head, contours of face, etc.) and dynamically adapt the detection parameters (e.g., idle signal floor, sensitivity, thresholds, etc.). Ultimately, a detection algorithm is used with configurable attack-hold-decay profiles to determine proximity accurately while minimizing false triggers.

Audio data captured by any of the front facing microphones <NUM> may be segmented or divided into frames of appropriate frame size (e.g., 10msec). Each audio frame may be processed by a sequence of signal preprocessing modules to pre-amplify and remove noise and other low frequency signals (e.g., below <NUM>). This may be advantageously done using a sixth order infinite impulse response (IIR) high pass filter (HPF) 232a along with a pre-amplifier to produce a resultant signal from having primarily echoes of the transmitted (and reflected) near ultrasonic tone (NUT). The echo signal from the HPF 232a may be further processed by an envelope generator 232b to extract an overall echo profile of the reflected signals <NUM>. This envelope generation may be done using a signal squarer and low pass filter (LPF). The resultant envelop signal may be down-sampled to <NUM> by a decimator 232c, which advantageously reduces computation complexity by a factor of <NUM> and enables low power computations.

The decimated echo envelope signal <NUM> may be further processed by the proximity sensing engine <NUM> using dynamic energy estimation (DEE). This DEE advantageously estimates mean energy of the echo envelope signal <NUM> for a predetermined time window (e.g., 100msec) to determine an absolute standard deviation of the echo envelope signal <NUM> over the estimated mean energy. This beneficially ensures that the mean energy of the echo envelope signal <NUM> may be detected accurately notwithstanding variances that can be experienced due to the operating environment. (For example, the mean energy may vary depending upon whether the device is held in a hand of the user, placed upon a wooden tabletop or steel tabletop, placed in a pocket of the user, and so on, due to differences in sound absorption among such disparate materials. ) Once the mean energy for the desired time window has been derived, the absolute standard deviation (ASD) of the echo envelope signal over the mean energy is generated as the output processed signal 235a. Additionally, a feedback signal 235b may be provided to the adaptive gain controller <NUM> in the NUT engine <NUM> to help ensure that drive signal <NUM> of a sufficient magnitude is provided in the EP speaker <NUM> based on the estimated mean energy.

The resultant absolute standard deviation (ASD) signal 235a may be further processed by the proximity detection (PD) engine <NUM> to determine (e.g., compute) the true proximities. This PD engine <NUM> advantageously employs a detection technique based on a configurable attack-hold-decay algorithm to detect the proximity accurately while minimizing false triggers. Such configurable attack-hold-decay algorithm employs an integrator core that accumulates envelope signal energy when the ASD is higher than a predetermined threshold and decays otherwise, and triggers final proximity events based on its internal states.

<FIG> illustrates operation flow for a near-ultrasonic proximity computation engine according to one embodiment. As indicated, and discussed in more detail hereinabove, the microphone <NUM> captures echoes <NUM> along with noise, and the high pass filter 232a (e.g., a <NUM>th order IIR filter with a cutoff frequency of <NUM>) removes noise and the lower frequency audible signals. The filtered signal may then be squared and filtered with a low pass filter (e.g., having a cutoff frequency of <NUM>) for performing envelope detection 232b, followed by down-sampling with a decimator 232c by a factor M=<NUM>. The software platform of the host device may then be employed to perform the operations of the proximity sensing engine <NUM> and proximity detection engine <NUM>.

<FIG> illustrates an attack-hold-decay algorithm for proximity computation according to one embodiment. This may be used for detecting and validating a computed proximity while effectively rejecting false detections. An integration of the ASD signal 235a during time interval I1 produces an inclining ramp which ramps up by a factor of PXDT_INC_FACTOR if the standard deviation of the envelope signal <NUM> transcends beyond a predetermined AWAY Detection Threshold. Otherwise, the integration produces a declining ramp that decays by a factor of PXDT_DEC_FACTOR, e.g., as seen during time interval I3. Proximity is detected as TRUE if the integration reaches a higher predetermined STILL Detection Threshold, e.g., as seen during time interval I2. Conversely, proximity is detected as FALSE if the integration decays below the lower AWAY Detection Threshold, e.g., as seen during time interval I3. The difference between the higher STILL Detection Threshold and lower AWAY Detection Threshold maintains a degree of hysteresis to minimize false triggers. The ramping up time needed for the integration to reach the upper STILL Detection Threshold also helps to minimize false triggers.

Constants associated with this process include: PXDT_INC_FACTOR = <NUM> as the % Increment/Attack Factor; PXDT_DEC_FACTOR = <NUM> as the % Decay Factor; PXDT_INC_THRLD_LEVEL = <NUM> as the % Signal Level above which an Attack starts/continues and below which decay starts/continues; PXDT_MAX_STILL_DET_FLOOR = <NUM> as the % Higher the Value slower will result in a STILL detection; and PXDT_MAX_AWAY_DET_FLOOR = <NUM> as the % Away Detection Threshold.

<FIG> illustrates test cases for different detection capabilities, distances and object sizes according to one embodiment. Examples of test objects associated with a user were (as indicated) a head, a torso, a hand and two fingers. For one example test using a technique as described above, location relative to the EP speaker <NUM> of the device <NUM> was accurately validated as far away as <NUM> for all objects (head, torso, hand and fingers), <NUM> for larger objects (head, torso and hand), and <NUM> for only the largest object (torso).

<FIG> illustrates use of proximity sensing for wake-on gesturing with a mobile device according to one embodiment. Similar to the use case for a mobile communication device, such as telephone handset <NUM> (<FIG>), proximity sensing as described herein may be advantageously used for enabling wake-on gesturing of a computing device <NUM>, such a mobile or desktop computer (though perhaps more advantageously for a mobile device which, for purposes of preserving battery life, will enable dimming or closing of the display following prescribed intervals of inactivity). As with the previous example, a near ultrasonic signal <NUM> emitted from an EP speaker <NUM> of the device <NUM> may be directed (e.g., beam formed) toward a designated area in front of the display and in which a hand <NUM> of the user is expected to gesture. The high frequency (HF) sound waves <NUM> reflect from the hand <NUM> and are captured as echoes by the built-in microphone <NUM>. The strength of the echoes <NUM> is proportional to the proximity of the hand <NUM>, and the energy of the reflected HF sound may be used is a cue for detecting and confirming that proximity.

<FIG> illustrates an area and range for detection of wake-on gesturing with a mobile device according to one embodiment. As noted above, a near ultrasonic signal <NUM> emitted from an EP speaker <NUM> of the device <NUM> may be directed toward a designated area <NUM> with a predetermined range <NUM> in front of the display and in which a hand <NUM> of the user is expected to gesture.

In this use case, the inbuilt PC speakers <NUM> emit near ultrasonic pulses for use in determining the presence of an object within the detection area <NUM>. Similar to the tone example, the emitted pulses create echoes from nearby objects and may be detected by the inbuilt microphones <NUM>. The received signal contains echo profiles of all detected objects and may be processed further to extract the presence of a hand <NUM> (or virtually any other object as desired to perform a wake-on operation). The strength of the echo is proportional to the size of the object and inversely proportional to its distance from the speakers <NUM>. The presence of the hand <NUM> may be detected by evaluating features like sensed depth and strength of the reflected signal strength corresponding to a hand size with sufficient hold time.

Typical operation may begin when the device <NUM> has entered an IDLE state (e.g., following some prescribed interval of little or no user interaction with the device <NUM>). During this state, the device <NUM> emits the near ultrasonic pulses <NUM> via the speaker(s) <NUM> and listens for echoes <NUM> via the microphone(s) <NUM>. In response to reception of echoes, the device <NUM> performs proximity sensing and detection operations to detect and confirm the presence of a hand <NUM>. Upon a successful detection, the device <NUM> transitions from an IDLE state to a WAKE state, following which the user may perform further authentication procedures to access features and operations of the device <NUM>.

As discussed in more detail below, two major device subsystems may be employed: a near ultrasonic pulse excitation engine, and a depth sensing (DS) and proximity detection (PD) engine.

Near ultrasonic pulses <NUM> are emitted by the PC speakers <NUM>. The pulses <NUM> may be shaped in such a way that there are sufficient ramp up and ramp down times to ensure that they remain inaudible to the user. In accordance with one embodiment, the pulses <NUM> may have a <NUM> nominal center frequency, 400usec pulse-durations and eight repetitions per second for a repetition interval of 125msec. Such a low repetition interval ensures that the speaker <NUM> actively emits sound waves only <NUM>% and remains idle <NUM>% of the time, thereby minimizing uses of power from the host device <NUM>. However, the repetition rate may be increased, albeit with added power consumption and computations. While various pulse shaping techniques may be used, it was found that distortions created at the output may be minimized by using a pulse shaping technique based on auto-correlation. Such near ultrasonic pulse generation and shaping may be described as follows:
Base Tone: <MAT>.

<FIG> illustrates a magnitude versus time graph <NUM> for near-ultrasonic pulse as described above according to one embodiment.

<FIG> illustrates a block diagram of a near-ultrasonic proximity computation engine <NUM> according to another embodiment. A depth sensing (DS) engine and proximity detection (PD) engine analyzes the reflected near ultrasonic pulses that together form an echo profile signal which is the sum of all echoes. The inbuilt microphone receives the transmitted pulses as well as the echo profile signal. The microphone also listens to the environment sound including music, speech and noise which spans the complete audio spectrum. A depth sensing technique may be used to remove noise and detect the echo signals for processing. The DS engine further performs many checks to confirm that primarily echo signals are being analyzed and calculates the depth for the detected object.

The DS engine and PD engine work on captured audio data using any of the front facing microphones and will segment such data into frames of appropriate frame size (e.g., 10msec). Each audio frame is processed by a sequence of signal preprocessing modules to pre-amplify and remove noise and other low frequency signals (e.g., below <NUM>) from the data segments. This may be done using a sixth order IIR high pass filter (HPF) along with a pre-amplifier <NUM>. The resultant filtered signal <NUM> retains primarily echoes of transmitted pulses.

The filtered echo signal <NUM> is processed further by envelope extractor (EE) engine <NUM> to extract an overall echo profile of the reflected signals. This envelope generation may be done by a signal squarer [abs(x[n])] 904a and a low pass filter 904b (e.g., with a <NUM> cutoff frequency). Once the envelop of echoes is generated, the resultant signal <NUM> may be down-sampled to a <NUM> signal to reduce computation complexity by a factor of two and thereby enable low power computations.

Once the envelop signal has been derived from the EE engine <NUM>, the envelope signal <NUM> is processed by a peak extractor (PE) engine <NUM> to detect peaks in the echo profile signal which can be used for echo detection. A technique for finding the peaks in the echo profile may be implemented to analyze echo signal characteristics as follows:
<MAT>
<NUM>, Otherwise
Where: dt = Stride value
m = Amplitude difference across peak.

<FIG> illustrates block diagrams of near-ultrasonic excitation and proximity computation engines according to another embodiment. The peaks of the echo profile signal extracted from the PE engine <NUM> may be further processed by a depth profile calculation engine <NUM> that extracts and validates the echoes, and calculates depth information associated with the selected echo.

The input <NUM> of the depth profile calculation engine <NUM> contains peaks corresponding to the pulsed excitation signal, noise peaks and echo peaks. The depth profile calculation engine <NUM> locks to the periodic pulse excitation signal to find the relevant echoes corresponding to the particular pulse excitation. This operation of DPM may be described as follows: (<NUM>) The reference pulse excitation signal from near ultrasonic pulse excitation generator engine <NUM> is determined from among the remaining signals; (<NUM>) echoes with respect to the reference signal are determined; (<NUM>) the most prominent echo signal is selected; and (<NUM>) the time difference between the reference pulse signal and echo signal which is proportional to the depth of the object detected is derived.

Similar to the example discussed above, the proximity detection (PD) engine <NUM> performs proximity detection based on a configurable attack-hold-decay algorithm to detect the proximity accurately while minimizing false triggers. As previously discussed, this attack-hold-decay process performs an integration that accumulates envelope signal energy when the ASD is higher than a predetermined threshold and decays otherwise, and triggers proximity events based on its internal states.

<FIG> illustrates a block diagram of a mobile device in the form of a computing device <NUM> according to one embodiment. The computing device <NUM> may house a system board <NUM> that may include a number of components, including, without limitation, to a processor <NUM> and at least one communication package <NUM>. The communication package <NUM> may be coupled to one or more antennas <NUM>. The processor <NUM> may be physically as well as electrically coupled to the board <NUM>.

Depending on its applications, computing device <NUM> may include other components that may or may not be physically and electrically coupled to the board <NUM>. These other components include, without limitation, volatile memory (e.g., DRAM) <NUM>, non-volatile memory (e.g., ROM) <NUM>, flash memory (not shown), a graphics processor <NUM>, a digital signal processor (not shown), a crypto processor (not shown), a chipset <NUM>, an antenna <NUM>, a display <NUM> (e.g., a touchscreen), a touchscreen controller <NUM>, a battery <NUM>, an audio codec (not shown), a video codec (not shown), a power amplifier <NUM>, a global positioning system (GPS) device <NUM>, a compass <NUM>, an accelerometer (not shown), a gyroscope (not shown), a speaker <NUM>, a camera <NUM>, a lamp <NUM>, a microphone array <NUM>, and a mass storage device (such as a hard disk drive) <NUM>, compact disk (CD) drive (not shown), digital versatile disk (DVD) drive (not shown), and so forth. These components may be connected to the system board <NUM>, mounted to the system board, or combined with any of the other components.

The communication package <NUM> enables wireless and/or wired communications for the transfer of data to and from the computing device <NUM>. The communication package <NUM> may implement any of a number of wireless or wired standards or protocols, including but not limited to Wi-Fi (IEEE <NUM> family), WiMAX (IEEE <NUM> family), IEEE <NUM>, long term evolution (LTE), Ev-DO, HSPA+, HSDPA+, HSUPA+, EDGE, GSM, GPRS, CDMA, TDMA, DECT, Bluetooth, Ethernet derivatives thereof, as well as any other wireless and wired protocols that are designated as <NUM>, <NUM>, <NUM>, and beyond. The computing device <NUM> may include multiple communication packages <NUM>. For instance, a first communication package <NUM> may be dedicated to shorter range wireless communications such as Wi-Fi and Bluetooth and a second communication package <NUM> may be dedicated to longer range wireless communications such as GPS, EDGE, GPRS, CDMA, WiMAX, LTE, Ev-DO, and others.

The cameras <NUM> may contain image sensors with pixels or photodetectors, and may use resources of an image processing chip <NUM> to read values and also to perform exposure control, depth map determination, format conversion, coding and decoding, noise reduction and 3D mapping, etc. The processor <NUM> is coupled to the image processing chip to drive the processes, set parameters, etc..

In various implementations, the computing device <NUM> may be eyewear, a laptop, a netbook, a notebook, an ultrabook, a smartphone, a tablet, a personal digital assistant (PDA), an ultra mobile PC, a mobile phone, a desktop computer, a server, a set-top box, an entertainment control unit, a digital camera, a portable music player, a digital video recorder, wearables or drones. The computing device may be fixed, portable, or wearable. In further implementations, the computing device <NUM> may be any other electronic device that processes data. Embodiments may be further implemented as a part of one or more memory chips, controllers, CPUs (Central Processing Unit), microchips or integrated circuits interconnected using a motherboard, an application specific integrated circuit (ASIC), and/or a field programmable gate array (FPGA).

References to "one embodiment", "an embodiment", "example embodiment", "various embodiments", etc., indicate that the embodiment(s) so described may include particular features, structures, or characteristics, but not every embodiment necessarily includes the particular features, structures, or characteristics. Further, some embodiments may have some, all, or none of the features described for other embodiments.

In the foregoing and following description and the following claims, the term "coupled" along with its derivatives, may be used. "Coupled" is used to indicate that two or more elements cooperate or interact with each other, but they may or may not have intervening physical or electrical components between them.

As used in the claims, unless otherwise specified, the use of the ordinal adjectives "first", "second", "third", etc., to describe a similar element, merely indicate that different instances of such elements are being recited, and are not intended to imply that the elements so described must be in a given sequence, either temporally, spatially, in ranking, or in any other manner.

The drawings and the forgoing description give examples of embodiments. Those skilled in the art will appreciate that one or more of the described elements may well be combined into a single functional element. Alternatively, certain elements may be split into multiple functional elements. Elements from one embodiment may be added to another embodiment. For example, orders of operation described herein may be changed and are not limited to the manner described herein, but only by the appended claims. Moreover, actions of any operation flow need not be implemented in the order described, nor do all actions necessarily need to be performed. Also, those actions that are not dependent on other actions may be performed in parallel with the other actions.

Method examples described herein may be implemented, at least in part, with nor or more machines or computing devices. Some examples may include a computer-readable medium or machine-readable medium encoded with instructions operable to configure an electronic device to perform methods as described in the examples disclosed herein. An example implementation of such methods may include code, such as microcode, assembly language code, a higher-level language code, or the like. Such code may include computer readable instructions for performing various methods. Further, in an example, the code may be tangibly stored on one or more volatile, non-transitory, or non-volatile tangible computer-readable media, such as during execution or at other times. Examples of these tangible computer-readable media may include, without limitation, hard disks, removable magnetic disks, removable optical disks (e.g., compact disks and digital video disks), magnetic cassettes, memory cards or sticks, random access memory (RAM), read only memory (ROM), and the like.

Claim 1:
A mobile device (<NUM>), comprising:
an acoustic signal excitation engine (<NUM>) configured to produce a plurality of acoustic signals (<NUM>) including a near ultrasonic acoustic signal (<NUM>, <NUM>), wherein said near ultrasonic acoustic signal comprises a tone having a nominal frequency within a frequency range of <NUM>-<NUM> kilohertz, or wherein said near ultrasonic acoustic signal comprises a pulse stream having a center frequency within a frequency range of <NUM>-<NUM> kilohertz, a pulse duration and a pulse repetition interval, said pulse duration being substantially less than said pulse repetition interval;
a first electromechanical transducer (<NUM>) coupled to said acoustic signal excitation engine and responsive to said plurality of acoustic signals by producing a plurality of outgoing acoustic waves (<NUM>) including an outgoing near ultrasonic acoustic wave related to said near ultrasonic acoustic signal;
a second electromechanical transducer (<NUM>) responsive to a plurality of incoming acoustic waves (<NUM>) including an incoming near ultrasonic acoustic wave related to an echo of said outgoing near ultrasonic acoustic wave by producing a plurality of incoming acoustic signals; and
a proximity computing engine (<NUM>) coupled to said second electromechanical transducer and responsive to said plurality of incoming acoustic signals by producing at least one status signal related to said incoming near ultrasonic acoustic wave and a proximity of said second electromechanical transducer to an object from which said incoming near ultrasonic acoustic wave was received,
wherein the proximity computing engine performs an attack-hold-decay integration of a signal corresponding to one or more echoes of the outgoing near ultrasonic acoustic wave to produce an integrated signal having multiple ramp portions respectively spanning multiple time periods, the multiple ramp portions including a level portion at a still detection threshold over a time period; and
wherein proximity is reported when the attack-hold-decay integration to produce the integrated signal reaches the level portion at the still detection threshold during the time period.