Patent ID: 12219344

DETAILED DESCRIPTION

Example Systems

FIG.1Aillustrates a system for implementing a spatial audio application, according to an embodiment. In the example scenario shown, user100is wearing auxiliary device101while watching audio/visual (A/V) content displayed on a screen of source device102. A/V content can include but is not limited to: movies, television shows, video conference calls, video games, augmented reality (AR) applications, virtual reality (VR) applications and any other content that utilizes spatial or immersive audio. Auxiliary device101includes any type of audio output device that is worn by the user, including on-ear, in-ear and over-ear headphones, earbuds, earphones, Airpods®, VR or AR headsets, smart glasses, and any other audio output device that is capable of producing spatial audio. In the description that follows, the auxiliary device101will hereinafter be referred to as headset101. That term is used for convenience and should not be construed as limiting the types of auxiliary devices that are applicable to the embodiments disclosed herein.

Source device102is any device capable of presenting any type of A/V content or only spatial audio content, including but not limited to: a smartphone, tablet computer, notebook computer, wearable computer, table computer, surface tablet, etc. A spatial audio application is an application that runs on source device102and generates surround sound or 3D audio through headset101. Spatial Audio is any audio which generates a 3D virtual auditory space that gives a listener a sense of space beyond conventional stereo, allowing the listener to pinpoint where sound is coming from, whether above, below, or a full 360 degrees around the listener. Spatial audio as used herein includes any channel-based, binaural or object-based audio technology, protocol, standard or format, including but not limited to: surround sound, Ambisonics, 3D audio, binaural audio, immersive audio, wave field synthesis, 360 audio, game audio and any other audio rendering technology, protocol or format that provides a 3D virtual auditory space.

FIG.1Billustrates a centered 3D virtual auditory space103, according to an embodiment. The 3D virtual auditory space103includes virtual sound sources or “virtual speakers” (e.g., center (C), Left (L), Right (R), left-surround (L-S) and right-surround (R-S)) that are rendered in ambience bed105using known spatial audio techniques, such as binaural rendering. Note that the example ambience bed105is for a 5.1 audio format, where all audio channels are located in an XAYAplane of ambience bed105(ZA=0), where XAis forward towards the center channel (C), YAis right and ZAis down, following the “right-hand” rule. Other embodiments, can have more or fewer audio channels, and the audio channels can be placed at different locations in the 3D virtual auditory space arbitrarily in any plane.

To maintain the desired 3D spatial audio effect, it is desired that the center channel (C) be aligned with a boresight vector104. The boresight vector104originates from a headset reference frame and terminates at a source device reference frame. When the virtual auditory environment103is first initialized, the center channel (C) is aligned with boresight vector104by rotating a reference frame for the ambience bed105(XA, YA, ZA) to align the center channel (C) with boresight vector104, as shown inFIG.1B. This alignment process causes the spatial audio to be “centered.” When the spatial audio is centered, the user perceives audio from the center channel (e.g., spoken dialogue) as coming directly from the display of source device102.

In an embodiment, the centering is accomplished by tracking boresight vector104to the location of source device102from the head reference frame using a tracking algorithm, such as an Extended Kalman Filter (EKF) which provides an estimate of boresight104that requires periodic correction. In an embodiment, boresight vector104determines the location of the center channel (C), and a second tracker takes as input the boresight vector104and provides an output orientation of ambience bed105, which determines the location of the L/L-S and R/R-S surround channels around the user in addition to the center channel (C). Aligning the center channel (C) of ambience bed105with boresight vector104allows rendering of the center channel (C) at the estimated location of source device102for the user's audio perception. If boresight vector104is not centered on source device102when the user is looking at the screen of source device102straight on (e.g., due to tracking error), then aligning the center channel (C) of ambience bed105will not “center” the audio, since the center channel will still be rendered at the erroneous estimate of the location of source device102. Note that boresight vector104changes whenever the user's head rotates with respect to source device102, such as when source device102is stationary in front of the user and the user's head is rotating. In this case, the motion of the user's head is accurately tracked as the head rotates, so that even when boresight vector104changes, the audio stays centered on the estimated location of source device102because the head tracker is providing accurate tracking of how the true boresight vector104is changing over time. To track the head relative to source device102, the IMU sensor data is integrated, which can introduce drift over time due to sensor errors (e.g., bias, scale factor error, etc.). It is desirable to take any good opportunity to correct such accumulated tracking errors using the BTZ process described in reference toFIGS.2-4.FIG.1Cillustrates boresight correction using bleed-to-zero (BTZ), according to an embodiment. Due to tracking error, boresight104becomes uncentered from screen of source device102, resulting in a correction angle θ that needs to zeroed out using BTZ, as described in reference toFIG.2.

Static Correlated Mutual Quiescence and BTZ

FIG.2is a flow diagram of process200for audio re-centering based on mutual quiescence, according to an embodiment. There are two types of mutual quiescence that are monitored by process200: strict or static mutual quiescence and loose or correlated mutual quiescence. Strict/static mutual quiescence is when both source device102and headset101are static, as determined by source device inertial measurement unit (IMU)1007and headset IMU1111. In an embodiment, IMU1007includes a 3-axis MEMS accelerometer and a 3-axis MEMS gyroscope, which measure 3D accelerations and 3D rotation rates, respectively, in a source device body frame. Similarly, IMU1111includes a 3-axis MEMS accelerometer and a 3-axis MEMS gyroscope, which measure 3D accelerations and 3D rotation rates, respectively, in a headset body frame. Source device102and headset101each have a wireless transceiver (e.g., Bluetooth chipset) that allow bi-directional communication of sensor data and other data. The sharing of sensor data allows for relative motion tracking of boresight vector104by source device102and/or headset101using a head tracking filter.

Loose/correlated mutual quiescence is detected by process200when correlated motion between source device102and headset101is detected. Correlated motion is detected when source device102and headset move together (e.g., rotate or translate) relative to a global inertial reference frame. For example, correlated motion occurs when user100is viewing content on source device102while walking with source device102in hand, riding, flying in an airplane, walking on a treadmill, etc. For example, if user100is viewing content on a screen of source device102while riding in a car and the car makes a left or right turn, both source device IMU1007and headset IMU1111will sense that rotation. A correlated motion detector in source device102and/or headset101compares the two rotation motions (e.g., compares rotation rates and accelerations) and uses logic to determine correlated motion. When strict/static or loose/correlated mutual quiescence is detected by process200, process200assumes that user is consuming content on source device102, and the spatial audio is re-centered to correct the tracking error, as described in reference toFIG.1B.

FIG.3illustrates the use of different audio re-centering rates for different size correction angles, according to an embodiment. The re-centering of spatial audio (also referred to herein as bleed-to-zero (BTZ)), removes a boresight correction angle adaptively based on the size of the correction angle. For a small correction angle θ1, a slow re-centering rate is applied. For a large correction angle θ3, a fast re-centering rate is applied. For a correction angle θ2between the small correction angle θ2and the large correction angle θ3, a medium re-centering rate is applied. The correction angles (θ1, θ2, θ3) can be computed by taking a dot product of boresight vector104and a reference direction vector (XA) determined during initialization when boresight vector104is aligned with the center channel (C) of ambience bed105, as described in reference toFIG.1B. The correction angles (θ1, θ2, θ3) are compared to specified threshold angles (θth1, θth2, θth3) to determine the rate of re-centering.

FIG.4is a plot of BTZ rate versus correction angle, according to an embodiment. As shown in this example, a minimum rate (rate_1) is used to re-center boresight correction angles below a first angle threshold, θth1, and a maximum rate (rate2) is used to re-center boresight correction angles greater than a second threshold angle, θth2. Boresight correction angles between the first and second thresholds are re-centered at a rate determined by a linear function, as shown inFIG.4. In other embodiments, a different rate curve can be used (e.g., quadratic).

FIG.5illustrates tracking mutual quiescence using timers, according to an embodiment. To track static and correlated mutual quiescence, three quiescence timers are used: a static mutual quiescence timer, a correlated mutual quiescence timer and a disturbance timer. It is desired to accumulate quiescent time using the static or correlated mutual quiescence timers and also allow for temporary disturbances such that the onset of BTZ is not terminated. The timers are configured so that as the static or correlated mutual quiescence timers increment the disturbance timer decrements. The timers are used to determine when a BTZ horizon time is reached to invoke the BTZ process to re-center the spatial audio in the 3D virtual auditory space.

When static or correlated mutual quiescence is detected the static mutual quiescence timer or the correlated mutual quiescence timer begin to accumulate time. Assuming correlated mutual quiescence (used when correlated motion is detected), if no disturbance is detected, the correlated mutual quiescence timer continues to accumulate time until a threshold time τ (BTZ horizon time) is exceeded at which time BTZ is invoked. If after a first period of correlated mutual quiescence (Δtq0), a temporary disturbance is detected, then the disturbance timer begins to accumulate disturbance time for the duration of the temporary disturbance (δtd) and the correlated mutual quiescence timer is stopped but not reset. If the temporary disturbance ends before a threshold disturbance time (τ0), the accumulated disturbance time (δtd) is subtracted from the first period of mutual quiescence time (Δtq0−δtd), the disturbance timer is set to zero and a second period of mutual quiescence time (Δtq1) starts until a total accumulated mutual quiescence time (Δtq0−δtd+Δtq1) exceeds the threshold time (i), after which time the BTZ process is invoked.

Once the boresight is moved to the center position during the BTZ process, the boresight is kept at the center position as long as mutual quiescence continues. If the disturbance lasts longer than a threshold time (τ0) during the mutual quiescence period, the mutual quiescence and disturbance timers are both reset to zero (which aborts the BTZ process), as illustrated inFIG.6. The static mutual quiescence timer can operate in a similar manner as the mutual quiescence timer described above. In an embodiment, the time threshold for triggering BTZ based on static mutual quiescence is less than the time threshold for triggering BTZ based correlated mutual quiescence.

FIG.7Aillustrates logic for triggering BTZ based on static or correlated mutual quiescence, according to an embodiment. The logic can be implemented using one or both of the architectures described in reference toFIGS.10and11. The logic described below can be performed on each sample of sensor data or on multiple samples (e.g., windows of samples).

In an embodiment, a first logic701is evaluated to determine whether to invoke the BTZ process to re-center the spatial audio: hasBTZQuiescence=currentlyQuisescent && (timeOfStatic>τ∥timeOfCorrelatedQuiescence>2τ). The value “hasBTZQuiescence” can be a Boolean value that is TRUE if the subsequent conditions are met and is otherwise FALSE. The variable “currentlyQuiescent” can be a Boolean value that is TRUE if mutual quiescence is currently detected. The variable “timeOfStatic” is a static quiescence timer value and “timeOfCorrelatedQuiescence” is a correlated quiescence timer value, as described in reference toFIG.5. When there is current mutual quiescence AND either a static quiescence time OR a correlated quiescence time exceeds its respective threshold value, the BTZ process is invoked. Otherwise, “hasBTZQuiescence” is FALSE and the BTZ process is not invoked.

During a mutual quiescence period, as second logic702is evaluated to determine if the current mutual quiescence is static or correlated (e.g., correlated motion). In an embodiment, this determination is made using a buffer of sensor data samples, such as N seconds (e.g., N=0.25 s) of buffered acceleration samples. An example of second logic702is: srcIMU.isStatic && auxIMU.iStatic && Max(∥auxOmega∥, ∥srcOmega∥)>ε0, where “srcIMU.isStatic” can be a Boolean value indicating that the source device is static and “auxIMU.iStatic” can be a Boolean value indicating that the headset is static, ∥auxOmega∥ is the magnitude of the 3D rotation rate vector from the Headset IMU1111and ∥srcOmega∥ is the magnitude of the 3D rotation rate vector from the source device IMU1007. Second logic702can be verbalized as follows. When source device IMU1007indicates that source device102is static AND the Headset IMU1111indicates that headset101is static AND the greater of the maximum source device rotation rate magnitude or the maximum headset device rotation rate magnitude is less than a threshold rotation rate, then static mutual quiescence is detected. When static mutual quiescence is detected, a static quiescence timer “timeOfStatic” is incremented, a correlated mutual quiescence timer “timeOfCorrelatedQuiescent” is incremented and a disturbance timer “timeOfDisturbance” is decremented. Otherwise, third logic703is evaluated.

In an embodiment, third logic703is evaluated to determine if a temporary disturbance is present: Max(∥srcAccLPF∥, ∥auxAccLPF∥)<δ0&& Max(∥srcOmegaBufferAvg∥, ∥auxOmegaBufferAvg∥)<ε1&& Max(∥auxOmega∥)<2ε0&& Max(∥auxAcc∥, ∥srcAcc∥)<δ1, where “srcAccLPF is low-pass filtered acceleration vector from the source device IMU1007, “auxAccLPF” is low-pass filtered acceleration vector from the headset IMU1111, “srcOmegaBufferAvg” is the average of a number of buffered rotation rate samples from the source device IMU1007, “auxOmegaBufferAvg” is the average of a number of buffered rotation rate samples (e.g., 2 s of rotation rate samples) from the headset IN/U1111, “srcOmega” is the current rotation rate vector from the source device IMU1007, “auxOmega” is the current rotation rate vector from the headset IMU1111, “srcAcc” is the current acceleration vector from the source device IMU1007and “auxAcc” is the current acceleration vector from the headset IMU1111.

If logic703is TRUE, a temporary disturbance is not detected and the timeOfCorrelatedQuiescent timer is incremented and the timeOfDisturbance timer is decremented. Otherwise, if logic703is FALSE, a temporary disturbance is detected, the timeOfStatic timer is decremented, the timeOfCorrelatedQuiescent timer is decremented and the timeOfDisturbence is incremented, as previously described in reference toFIG.5. If timeOfDisturbance exceeds a maximum threshold, τ0, then timeOfStatic=0, timeOfCorrelatedQuiescent=0 and timeOfDisturbance=0. The logic described above allows for temporary disturbances and loosens quiescence bounds when there is accumulated historical quiescence and absence of BTZ.

FIG.7Billustrates correlated-activity mutual quiescence704, according to an embodiment. In an embodiment, mutual quiescence is also detected when source device102and auxiliary device101are in a correlated activity state, such as, for example, during walking, in a vehicle or on treadmill. As used herein, “correlated,” means that source device102and auxiliary device101are aligned in yaw rotation only, and the up and down bounces in the sensor data from walking or riding in vehicle are ignored, because these noises should not affect how the boresight is estimated. To isolate rotations in yaw, the rotations are projected into the inertial gravity directions obtained from the attitude estimators on the respective source device and auxiliary devices, using the inertial gravity estimates previously described.

In an embodiment, there are two processing paths (paths1and2) used to detect correlated activity. In path1, an activity classifier705that runs on source device102(CMMotionActivity) is used to detect an activity type (e.g., walking, driving) If the output of the activity classifier indicates the detection of long-term moving activities like walking or vehicular activities, yaw correlation is checked706. If yaw rotations are correlated706, the system enters a correlated activity state709and the isInCorrelatedActivity=TRUE. Otherwise, isInCorrelatedActivity=FALSE.

Path2is a special case of correlated activity for viewing content on source device102while working out on a treadmill or other gym machine (e.g., elliptical, rowing machine). The system assumes that source device102is stationary707on a tray of the workout machine. When the system detects that the user is constantly moving but not rotating in yaw708, the system enters a correlated activity state709. Not rotating in yaw is generally true for gym machine activities, such as a treadmill, elliptical or rowing machine. When using these types of gym machines, the user's head motion is mostly pitching and but little yawing. Once the system detects that source device102and auxiliary device101are in correlated activity, loose mutual quiescence is accumulated for BTZ.

FIG.7Cis a multi-panel timeline illustrating an example scenario where the user is walking while viewing content from her source device in hand, according to an embodiment. The first panel shows boresight yaw, the second panel shows boresight pitch. The lines710,711are truth data and the lines712,713are estimated boresight yaw and pitch angles. The third panel shows tracking status: tracking1IMU is single IMU tracking mode, where just headset IMU1111is used in tracking. Also shown is complex transition states and BTZ events. The fourth panel shows the magnitude of rotation rates on source device102and auxiliary device101. The rotation rates provides a sense of how much motion there is on both devices while user100is walking. Note that the BTZ process was triggered multiple times during walking, despite the noisy bounces in the sensor data due to the walking. Also when user100turns her head, the BTZ process exits immediately to capture the head turn so tracking is perceived as responsive by user100.

FIG.8illustrates extending a timeline for BTZ based on mutual quiescence, according to an embodiment. In some embodiments, the BTZ time horizon is extended, i.e., the time that elapses before the BTZ process is invoked. During a period of static or correlated mutual quiescence, a request is made by a head tracking filter to a face detector of a camera subsystem of source device102for a visual anchor, such as the face coordinates of user100in a camera frame. For example, if the head tracking filter is an extended Kalman filter (EKF) the visual anchor can be used as an observation in an update phase of the EKF. If the user turns her head away from the camera, the updates will no longer be available and the EKF will converge. If the user's head is turned away for an extended period of time and the static or correlated mutual quiescence timer is greater than zero (i.e., in a current mutual quiescence period), the conditions for extending the BTZ horizon are met.

FIG.9is a flow diagram of a process900for audio re-centering based on a static or correlated mutual quiescence in a spatial audio application, according to an embodiment. Process900can be implemented using one or both of the architectures described in reference toFIGS.10and11.

Process900begins by obtaining first motion data from an auxiliary device communicatively coupled to a source device (901), and obtaining second motion data from one or more motion sensors of the source device (902). The source device is configured to provide spatial audio content and the auxiliary device is configured to playback the spatial audio content. In an embodiment, the motion data includes 3D acceleration data obtained from a 3-axis MEMS accelerometers of the source device and auxiliary device and 3D rotation rate data obtained from 3-axis MEMS gyros of the source device and auxiliary device.

Process900continues by determining whether the source device and auxiliary device are in a period of mutual quiescence based on the first and second motion data (903), and in accordance with determining that the source device and the auxiliary device are in a period of mutual quiescence, re-centering the spatial audio in a three-dimensional virtual auditory space (904).

Process900continues by rendering (e.g., using binaural rendering) the 3D virtual auditory space for playback on the auxiliary device (905). In an embodiment, logic is used to determine a period of static or correlated mutual acquiescence, and timers are used to track the periods of mutual quiescence and also temporary disturbances, as described in reference to FIGS.5-7. Mutual quiescent time is accumulated using the static or correlated mutual quiescence timers. The accumulated quiescence time is then used to determine when to invoke the BTZ process to re-center the spatial audio in the 3D virtual auditory space. The disturbance timer measures the duration of a temporary disturbance which is subtracted from the accumulated mutual quiescence time such that the onset of BTZ is not disturbed. The BTZ process can be adaptive in that different rates for removing tracking error can be used based on the size of the tracking error.

Example Software Hardware Architectures

FIG.10is a conceptual block diagram of source device software/hardware architecture1000implementing the features and operations described in reference toFIGS.1-9. Architecture1000can include memory interface1021, one or more data processors, digital signal processors (DSPs), image processors and/or central processing units (CPUs)1022and peripherals interface1020. Memory interface1023, one or more processors1022and/or peripherals interface1020can be separate components or can be integrated in one or more integrated circuits.

Sensors, devices and subsystems can be coupled to peripherals interface1020to provide multiple functionalities. For example, IMU1007, light sensor1008and proximity sensor1009can be coupled to peripherals interface1020to facilitate motion sensing (e.g., acceleration, rotation rates), lighting and proximity functions of the wearable computer. Location processor1010can be connected to peripherals interface1020to provide geo-positioning. In some implementations, location processor1010can be a GNSS receiver, such as the Global Positioning System (GPS) receiver. Electronic magnetometer1011(e.g., an integrated circuit chip) can also be connected to peripherals interface1020to provide data that can be used to determine the direction of magnetic North. Electronic magnetometer1011can provide data to an electronic compass application. IMU1007can be an IMU that includes one or more accelerometers and/or gyros (e.g., 3-axis MEMS accelerometer and 3-axis MEMS gyro) configured to determine change of speed and direction of movement of the source device. Barometer1006can be configured to measure atmospheric pressure around the mobile device.

Camera/3D depth sensor1002captures digital images and video and can include both front-facing and rear-facing cameras. The 3D depth sensor can be any sensor capable of capturing 3D data or point clouds, such as a time of flight (TOF) sensor or LiDAR.

Communication functions can be facilitated through wireless communication subsystems1012, which can include radio frequency (RF) receivers and transmitters (or transceivers) and/or optical (e.g., infrared) receivers and transmitters. The specific design and implementation of the wireless communication subsystem1024can depend on the communication network(s) over which a mobile device is intended to operate. For example, architecture1000can include communication subsystems1024designed to operate over a GSM network, a GPRS network, an EDGE network, a Wi-Fi™ network and a Bluetooth™ network. In particular, the wireless communication subsystems1024can include hosting protocols, such that the mobile device can be configured as a base station for other wireless devices.

Audio subsystem1005can be coupled to a speaker1003and one or more microphones1004to facilitate voice-enabled functions, such as voice recognition, voice replication, digital recording and telephony functions. Audio subsystem1005can be configured to receive voice commands from the user.

I/O subsystem1013can include touch surface controller1017and/or other input controller(s)1015. Touch surface controller1017can be coupled to a touch surface1018. Touch surface1018and touch surface controller1017can, for example, detect contact and movement or break thereof using any of a plurality of touch sensitivity technologies, including but not limited to capacitive, resistive, infrared and surface acoustic wave technologies, as well as other proximity sensor arrays or other elements for determining one or more points of contact with touch surface1046. Touch surface1018can include, for example, a touch screen or the digital crown of a smart watch. I/O subsystem1013can include a haptic engine or device for providing haptic feedback (e.g., vibration) in response to commands from processor or a digital signal processor (DSP)1022. In an embodiment, touch surface1018can be a pressure-sensitive surface.

Other input controller(s)1015can be coupled to other input/control devices1016, such as one or more buttons, rocker switches, thumb-wheel, infrared port and USB port. The one or more buttons (not shown) can include an up/down button for volume control of speaker1003and/or microphones1004. Touch surface1018or other input control devices1016(e.g., a button) can include, or be coupled to, fingerprint identification circuitry for use with a fingerprint authentication application to authenticate a user based on their fingerprint(s).

In one implementation, a pressing of the button for a first duration may disengage a lock of the touch surface1018; and a pressing of the button for a second duration that is longer than the first duration may turn power to the mobile device on or off. The user may be able to customize a functionality of one or more of the buttons. The touch surface1018can, for example, also be used to implement virtual or soft buttons.

In some implementations, the mobile device can present recorded audio and/or video files, such as MP3, AAC and MPEG files. In some implementations, the mobile device can include the functionality of an MP3 player. Other input/output and control devices can also be used.

Memory interface1021can be coupled to memory1023. Memory1023can include high-speed random access memory and/or non-volatile memory, such as one or more magnetic disk storage devices, one or more optical storage devices and/or flash memory (e.g., NAND, NOR). Memory1023can store operating system1024, such as the iOS operating system developed by Apple Inc. of Cupertino, California Operating system1024may include instructions for handling basic system services and for performing hardware dependent tasks. In some implementations, operating system1024can include a kernel (e.g., UNIX kernel).

Memory1023may also store communication instructions1025to facilitate communicating with one or more additional devices, one or more computers and/or one or more servers, such as, for example, instructions for implementing a software stack for wired or wireless communications with other devices. Memory1023may include graphical user interface instructions1026to facilitate graphic user interface processing; sensor processing instructions1027to facilitate sensor-related processing and functions; phone instructions1028to facilitate phone-related processes and functions; electronic messaging instructions1029to facilitate electronic-messaging related processes and functions; web browsing instructions1030to facilitate web browsing-related processes and functions; media processing instructions1031to facilitate media processing-related processes and functions; GNSS/Location instructions1032to facilitate generic GNSS and location-related processes; and camera/3D depth sensor instructions1033for capturing images (e.g., video, still images) and depth data (e.g., a point cloud). Memory1023further includes spatial audio instructions1034for use in spatial audio applications, including but not limited AR and immersive video applications. Instructions1034include head tracking instructions and implements the user pose change detection features and processes, described in reference toFIGS.1-9.

Each of the above identified instructions and applications can correspond to a set of instructions for performing one or more functions described above. These instructions need not be implemented as separate software programs, procedures, or modules. Memory1023can include additional instructions or fewer instructions. Furthermore, various functions of the mobile device may be implemented in hardware and/or in software, including in one or more signal processing and/or application specific integrated circuits.

FIG.11is a conceptual block diagram of headset software/hardware architecture1110implementing the features and operations described in reference toFIGS.1-9. In an embodiment, architecture1100can includes system-on-chip (SoC)1101, stereo loudspeakers1102a,1102b(e.g., ear buds, headphones, earphones), battery protector1103, rechargeable battery1104, antenna1105, filter1106, LEDs1107, microphones1108, memory1109(e.g., flash memory), I/O/Charge port1110, IMU1111and pushbuttons1112for turning the headset on and off, adjusting volume, muting, etc. IMU1111was previously described in reference toFIGS.1-9, and includes, for example, a 3-axis MEMS gyro and a 3-axis MEMS accelerometer.

SoC1101further includes various modules, such as a radio frequency (RF) radio (wireless transceiver) for wireless bi-directional communication with other devices, such as a source device102, as described in reference toFIGS.1-9. SoC1101further includes an application processor (AP) for running specific applications, memory (e.g., flash memory), central processing unit (CPU) for managing various functions of the headsets, audio codec for encoding/decoding audio, battery charger for charging/recharging rechargeable battery1104, I/O driver for driving I/O and charge port (e.g., a micro USB port), digital to analog converter (DAC) converting digital audio into analog audio and LED driver for driving LEDs1107. Other embodiments can have more or fewer components.

The described features can be implemented advantageously in one or more computer programs that are executable on a programmable system including at least one programmable processor coupled to receive data and instructions from, and to transmit data and instructions to, a data storage system, at least one input device, and at least one output device. A computer program is a set of instructions that can be used, directly or indirectly, in a computer to perform a certain activity or bring about a certain result. A computer program can be written in any form of programming language (e.g., SWIFT, Objective-C, C#, Java), including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, a browser-based web application, or other unit suitable for use in a computing environment.

While this specification contains many specific implementation details, these should not be construed as limitations on the scope of any inventions or of what may be claimed, but rather as descriptions of features specific to particular embodiments of particular inventions. Certain features that are described in this specification in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable sub combination. Moreover, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a sub combination or variation of a sub combination.

Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multitasking and parallel processing may be advantageous. Moreover, the separation of various system components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

As described above, some aspects of the subject matter of this specification include gathering and use of data available from various sources to improve services a mobile device can provide to a user. The present disclosure contemplates that in some instances, this gathered data may identify a particular location or an address based on device usage. Such personal information data can include location-based data, addresses, subscriber account identifiers, or other identifying information.

The present disclosure further contemplates that the entities responsible for the collection, analysis, disclosure, transfer, storage, or other use of such personal information data will comply with well-established privacy policies and/or privacy practices. In particular, such entities should implement and consistently use privacy policies and practices that are generally recognized as meeting or exceeding industry or governmental requirements for maintaining personal information data private and secure. For example, personal information from users should be collected for legitimate and reasonable uses of the entity and not shared or sold outside of those legitimate uses. Further, such collection should occur only after receiving the informed consent of the users. Additionally, such entities would take any needed steps for safeguarding and securing access to such personal information data and ensuring that others with access to the personal information data adhere to their privacy policies and procedures. Further, such entities can subject themselves to evaluation by third parties to certify their adherence to widely accepted privacy policies and practices.

In the case of advertisement delivery services, the present disclosure also contemplates embodiments in which users selectively block the use of, or access to, personal information data. That is, the present disclosure contemplates that hardware and/or software elements can be provided to prevent or block access to such personal information data. For example, in the case of advertisement delivery services, the present technology can be configured to allow users to select to “opt in” or “opt out” of participation in the collection of personal information data during registration for services.

Therefore, although the present disclosure broadly covers use of personal information data to implement one or more various disclosed embodiments, the present disclosure also contemplates that the various embodiments can also be implemented without the need for accessing such personal information data. That is, the various embodiments of the present technology are not rendered inoperable due to the lack of all or a portion of such personal information data. For example, content can be selected and delivered to users by inferring preferences based on non-personal information data or a bare minimum amount of personal information, such as the content being requested by the device associated with a user, other non-personal information available to the content delivery services, or publicly available information.