Patent Description:
Head-mounted displays in an artificial reality system often include features such as speakers or personal audio devices to provide audio content to users of the head-mounted displays. The audio systems in head-mounted displays can include microphones positioned at or near the entrances of a user's ears to measure the sound produced by the speakers and calibrate the audio system. Current microphones for use in head-mounted displays, such as binaural microphones or microphone arrays embedded in frames of head-mounted devices, have limited sensitivity. For example, typical microphones used in head-mounted devices have difficulty detecting audio pressure waves produced by bone conduction transducers, which generate particle displacements outside the ear in the nanometer or picometer range. To generate pressure waves that can be detected by existing microphones, bone conduction transducers must produce a very loud volume, which is unpleasant for the user.

From <CIT>, it is known to provide a behind the ear hearing device having an external, optical microphone.

<CIT> discloses an ear canal microphone configured for placement in the ear canal to detect high frequency sound localization cues.

From <CIT>, it is known to provide a pair of glasses with an audio transceiver.

<CIT> discloses a hearing assistance system incorporated into a frame of eyeglasses.

<CIT> discusses an optical fiber acoustical sensor for detecting sound waves in a fluid medium.

From <CIT>, it is known to provide a hearing prosthesis including an implantable housing containing a detector, a light source, a fiber optical waveguide extending from the implantable housing in light communication with the light source and the detector, and an interferometer connected to the fiber optical waveguide and located outside of the implantable housing.

This present disclosure describes an audio system that includes an optical microphone for detecting audio waves with a higher sensitivity than previous microphones. The audio system may be a component of an eyewear device that is a component of an artificial reality head-mounted display (HMD). The audio system includes at least one transducer that produces acoustic pressure waves, and an optical microphone to detect the acoustic pressure waves. The optical microphone can be positioned at the entrance to the user's ear canal or in the vicinity of the user's ear. The optical microphone includes a laser that emits light that is separated into a sensing beam and a reference beam, e.g., using a beam splitter. The sensing beam travels through an optical sensing pathway, such as an optical fiber. The acoustic wave interacts with the sensing beam while it is in the optical sensing pathway by altering the optical path length of the sensing beam. A detector assembly receives the sensing beam from the optical sensing pathway, and also receives the reference beam. The detector measures the detected acoustic pressure wave based on the change in optical path length of the sensing beam. The audio system may adjust the acoustic pressure waves produced by the transducer based on the measurement of the detected acoustic pressure wave.

The invention is directed to an audio system according to claim <NUM> and to an eyewear device according to claim <NUM>. Further aspects of the invention are defined in the dependent claims.

According to an aspect of the present invention, there is provided an audio system comprising: a transducer assembly configured to be coupled to an ear of a user and to produce an acoustic pressure wave based on an audio instruction; an optical fiber wherein an end of the optical fiber is configured to be suspended in air and positioned at an entrance of an ear canal of the ear; a laser configured to emit light that is separated into a reference beam and a sensing beam, and the sensing beam is coupled into the optical fiber, wherein movement of the optical fiber caused by the acoustic pressure wave interacts with the sensing beam in the optical fiber to alter an optical path length of the sensing beam; a detector assembly configured to: detect the reference beam and detect the sensing beam from the optical fiber, and measure a detected acoustic pressure wave based in part on changes in optical path length between the reference beam and the sensing beam; and a controller configured to adjust the audio instruction based on the measurement of the detected acoustic pressure wave.

According to a further aspect of the present invention there is provided an eyewear device comprising: a frame; an audio system coupled to the frame, the audio system comprising: a transducer assembly configured to be coupled to an ear of a user and to produce an acoustic pressure wave based on an audio instruction, an optical fiber wherein an end of the optical fiber is configured to be suspended in air and positioned at an entrance of an ear canal of the ear, a laser configured to emit light that is separated into a reference beam and a sensing beam, and the sensing beam is coupled into the optical fiber, wherein movement of the optical fiber caused by the acoustic pressure wave interacts with the sensing beam in the optical fiber to alter an optical path length of the sensing beam, a detector assembly configured to: detect the reference beam and detect the sensing beam from the optical fiber, and measure a detected acoustic pressure wave based in part on changes in optical path length between the reference beam and the sensing beam; and a controller configured to adjust the audio instruction based on the measurement of the detected acoustic pressure wave.

Embodiments of the invention may include or be implemented in conjunction with an artificial reality system. Artificial reality is a form of reality that has been adjusted in some manner before presentation to a user, which may include, e.g., a virtual reality, an augmented reality, a mixed reality, a hybrid reality, or some combination and/or derivatives thereof. Artificial reality content may include completely generated content or generated content combined with captured (e.g., real-world) content. The artificial reality content may include video, audio, haptic sensation, or some combination thereof, and any of which may be presented in a single channel or in multiple channels (such as stereo video that produces a three-dimensional effect to the viewer). Additionally, in some embodiments, artificial reality may also be associated with applications, products, accessories, services, or some combination thereof, that are used to, e.g., create content in an artificial reality and/or are otherwise used in (e.g., perform activities in) an artificial reality. The artificial reality system that provides the artificial reality content may be implemented on various platforms, including an eyewear device, a head-mounted display (HMD) assembly with the eyewear device as a component, a HMD connected to a host computer system, a standalone HMD, a mobile device or computing system, or any other hardware platform capable of providing artificial reality content to one or more viewers.

An audio system includes an optical microphone for measuring sound provided to an ear of a user. The audio system comprises one or more transducers, such as cartilage conduction transducers, air conduction transducers, or bone conduction transducers. The transducers produce acoustic pressure waves sensed by a user's ear. Because ear shape and configuration varies between users, transducers produce acoustic pressure waves that vary from user to user. The acoustic pressure waves may be airborne pressure waves or tissue borne pressure waves (e.g., an acoustic pressure wave that propagates through bone, cartilage, or one or more other tissues), depending on the transducer used. For example, a cartilage conduction transducer vibrates an auricle of the user's ear, which creates an airborne acoustic pressure wave at an entrance of the ear that travels down an ear canal to an eardrum where it is perceived as sound by the user. In response to a given vibration of a cartilage conduction transducer, different ear geometries produce different airborne acoustic pressure waves. The optical microphone measures the acoustic pressure waves generated by the transducers, and provides the measurement to a controller that adjusts audio instructions to the transducers according to the measurement.

The optical microphone disclosed herein includes an optical sensing pathway that moves with a detected acoustic pressure wave. The movement of the optical sensing pathway alters an optical path length of a sensing beam that travels through the optical sensing pathway. Measuring the change in optical path length provides a measurement of the detected acoustic pressure wave. The optical microphone configuration described herein is highly sensitive. For example, the optical microphone can detect particle deflections in the nanometer or picometer range, which enables measurement of airborne pressure waves generated by a bone conduction transducer at the outside of a user's ear, even at a low volumes. Thus, the optical microphone can be used to calibrate the audio instructions to the transducers without the need for unpleasant, high-volume sounds.

<FIG> is a perspective view of an eyewear device <NUM> including an audio system, in accordance with one or more embodiments. The eyewear device <NUM> presents media to a user. In one embodiment, the eyewear device <NUM> may be a component of a head-mounted display (HMD). In some embodiments, the eyewear device <NUM> is a near-eye display. Examples of media presented by the eyewear device <NUM> include one or more images, video, audio, or some combination thereof. The eyewear device <NUM> may include, among other components, a frame <NUM>, a lens <NUM>, a sensor device <NUM>, a transducer assembly <NUM>, an optical microphone assembly <NUM>, and a controller <NUM>.

The eyewear device <NUM> may correct or enhance the vision of a user, protect the eye of a user, or provide images to a user. The eyewear device <NUM> may be eyeglasses which correct for defects in a user's eyesight. The eyewear device <NUM> may be sunglasses which protect a user's eye from the sun. The eyewear device <NUM> may be safety glasses which protect a user's eye from impact. The eyewear device <NUM> may be a night vision device or infrared goggles to enhance a user's vision at night. The eyewear device <NUM> may be a HMD that produces artificial reality content for the user. Alternatively, the eyewear device <NUM> may not include a lens <NUM> and may be a frame <NUM> with an audio system that provides audio (e.g., music, radio, podcasts) to a user.

The frame <NUM> includes a front part that holds the lens <NUM> and end pieces to attach to the user. The front part of the frame <NUM> bridges the top of a nose of the user. The end pieces (e.g., temples) are portions of the frame <NUM> to which the temples of a user are attached. The length of the end piece may be adjustable (e.g., adjustable temple length) to fit different users. The end piece may also include a portion that curls behind the ear of the user (e.g., temple tip, ear piece).

The lens <NUM> provides or transmits light to a user wearing the eyewear device <NUM>. The lens <NUM> is held by a front part of the frame <NUM> of the eyewear device <NUM>. The lens <NUM> may be prescription lens (e.g., single vision, bifocal and trifocal, or progressive) to help correct for defects in a user's eyesight. The prescription lens transmits ambient light to the user wearing the eyewear device <NUM>. The transmitted ambient light may be altered by the prescription lens to correct for defects in the user's eyesight. The lens <NUM> may be a polarized lens or a tinted lens to protect the user's eyes from the sun. The lens <NUM> may be one or more waveguides as part of a waveguide display in which image light is coupled through an end or edge of the waveguide to the eye of the user. The lens <NUM> may include an electronic display for providing image light and may also include an optics block for magnifying image light from the electronic display. Additional detail regarding the lens <NUM> can be found in the detailed description of <FIG>.

The sensor device <NUM> estimates a current position of the eyewear device <NUM> relative to an initial position of the eyewear device <NUM>. The sensor device <NUM> may be located on a portion of the frame <NUM> of the eyewear device <NUM>. In other embodiments, the sensor device <NUM> may be located in a different location from the location shown in <FIG>. The sensor device <NUM> includes a position sensor and an inertial measurement unit. Additional details about the sensor device <NUM> can be found in the detailed description of <FIG>.

The audio system of the eyewear device <NUM> comprises a transducer assembly <NUM> configured to provide audio content to a user of the eyewear device <NUM> and an optical microphone assembly <NUM> configured to detect acoustic pressure waves produced by the transducer assembly <NUM>. In the illustrated embodiment of <FIG>, the audio system of the eyewear device <NUM> includes the transducer assembly <NUM>, the optical microphone assembly <NUM>, and the controller <NUM>. The audio system provides audio content to a user by utilizing the transducer assembly <NUM>. The audio system also uses feedback from the optical microphone assembly <NUM> to create a similar audio experience across different users. The controller <NUM> manages operation of the transducer assembly <NUM> by generating audio instructions. The controller <NUM> also receives feedback as monitored by the microphone assembly <NUM>, e.g., for updating the audio instructions. Additional detail regarding the audio system can be found in the detailed description of <FIG>.

Various types of transducers are available for outputting audio content to a user's ear. The transducer assembly <NUM> can include a single type of transducer, such as a cartilage conduction transducer, a bone conduction transducer, or an air conduction transducer. Alternatively, the transducer assembly <NUM> is a hybrid transducer that includes two or more types of transducers. For example, the transducer assembly <NUM> includes two transducers configured to vibrate over two different frequency ranges, which may or may not overlap. The transducer assembly <NUM> operates according to audio instructions, which may include a content signal, a control signal, and a gain signal. The content signal may be based on audio content for presentation to the user. The control signal may be used to enable or disable the transducer assembly <NUM> or one or more transducers of the transducer assembly. The gain signal may be used to adjust an amplitude of the content signal.

In some embodiments, the transducer assembly <NUM> includes a cartilage conduction transducer that produces sound by vibrating cartilage in the ear of the user. In an embodiment, a cartilage conduction transducer is coupled to an end piece of the frame <NUM> and is configured to be coupled to the back of an auricle of the ear of the user. The auricle is a portion of the outer ear that projects out of a head of the user. The cartilage conduction transducer receives audio instructions from the controller <NUM> and vibrates the auricle to generate an airborne acoustic pressure wave at an entrance of the user's ear according to the audio instructions.

In some embodiments, the transducer assembly <NUM> includes an air conduction transducer that produces sound by generating an airborne acoustic pressure wave in the ear of the user. In an embodiment, the air conduction transducer is coupled to an end piece of the frame <NUM> and is placed in front of an entrance to the ear of the user. The air conduction transducer receives audio instructions from the controller <NUM>.

In some embodiments, the transducer assembly <NUM> includes a bone conduction transducer that produces sound by vibrating bone in the user's head. In an embodiment, the bone conduction transducer is coupled to an end piece of the frame <NUM> and is configured to be behind the auricle and coupled to a portion of the user's bone. The bone conduction transducer receives audio instructions from the controller <NUM> and vibrates the portion of the user's bone according to the audio instructions. The bone vibration generates a tissue borne acoustic pressure wave that propagates toward the user's cochlea, thereby bypassing the eardrum.

The optical microphone assembly <NUM> detects an acoustic pressure wave at the entrance of the ear of the user. The optical microphone assembly <NUM> is coupled to an end piece of the frame <NUM>. The optical microphone assembly <NUM>, as shown in <FIG>, includes an optical sensing pathway, such as an optical fiber, that is positioned at the entrance of the user's ear. The optical microphone assembly <NUM> also includes a laser and a detector assembly, which are coupled to or housed in the frame <NUM>. For example, the laser and/or detector assembly may be housed in the frame <NUM> at or near the controller <NUM>, or housed in the end piece of the frame <NUM> to which the optical sensing pathway is coupled. The laser is configured to emit light into the optical sensing pathway, and the detector assembly is configured to detect light that has traveled through the optical sensing pathway. The detector measures the acoustic pressure wave in the vicinity of the user's ear based on an optical path length of the detected light.

In the embodiment shown in <FIG>, the optical fiber is configured so that the optical microphone assembly <NUM> directly measures an acoustic pressure wave at the entrance of the ear of the user. In other embodiments, the optical fiber is located in a different location in the vicinity of the user's ear. In still other embodiments, the optical microphone assembly <NUM> includes an optical fiber coupled to a flexible membrane that is configured to be coupled to the back of the auricle of the user, and the optical microphone assembly <NUM> indirectly measures the acoustic pressure wave at the entrance of the ear. For example, the optical microphone assembly <NUM> measures a vibration that is a reflection of the acoustic pressure wave at the entrance of the ear and/or measure a vibration created by the transducer assembly <NUM> on the auricle of the ear of the user, which is used to estimate the acoustic pressure wave at the entrance of the ear. In other embodiments, the flexible membrane with the optical fiber is coupled to a bone in the user's head or other tissue. Additional detail regarding the optical microphone assembly <NUM> can be found in the detailed description of <FIG>.

The controller <NUM> provides audio instructions to the transducer assembly <NUM> and receives information from the optical microphone assembly <NUM> regarding the produced sound, and updates the audio instructions based on the received information. The audio instructions may be generated by the controller <NUM>. The controller <NUM> may receive audio content (e.g., music, calibration signal) from a console for presentation to a user and generate audio instructions based on the received audio content. Audio instructions instruct the transducer assembly <NUM> or each transducer of the transducer assembly <NUM> how to produce vibrations. For example, audio instructions may include a content signal (e.g., a target waveform based on the audio content to be provided), a control signal (e.g., to enable or disable the transducer assembly), and a gain signal (e.g., to scale the content signal by increasing or decreasing an amplitude of the target waveform). If multiple transducers are included in the transducer assembly <NUM>, the controller <NUM> tailors different audio instructions for different transducers. For example, an acoustic pressure wave generated by a bone conduction transducer generally has a smaller magnitude than the acoustic pressure waves generated by cartilage or air conduction transducers. In addition, the frequency responses of different transducers may be different, so the controller <NUM> adjusts the instructions for each transducer based on their frequency responses.

The controller <NUM> also receives information from the optical microphone assembly <NUM> that describes the produced sound at an ear of the user. The controller <NUM> uses the received information as feedback to compare to the produced sound to a target sound (e.g., audio content) and updates the audio instructions to make the produced sound closer to the target sound. For example, the controller <NUM> updates audio instructions for a cartilage conduction transducer assembly to adjust vibration of the auricle of the user's ear to come closer to the target sound. The controller <NUM> is embedded into the frame <NUM> of the eyewear device <NUM>. In other embodiments, the controller <NUM> may be located in a different location. For example, the controller <NUM> may be part of the transducer assembly <NUM> or the optical microphone assembly <NUM>, or located external to the eyewear device <NUM>. Additional detail regarding the controller <NUM> and the controller's <NUM> operation with other components of the audio system can be found in the detailed description of <FIG>.

<FIG> is a profile view <NUM> of a portion of an audio system including an optical fiber microphone as a component of an eyewear device (e.g., the eyewear device <NUM>), in accordance with one or more embodiments. In this embodiment, the transducer assembly <NUM> includes a cartilage conduction transducer <NUM>, an air conduction transducer <NUM>, and a bone conduction transducer <NUM>. The optical sensing pathway <NUM> is a component of the optical microphone assembly <NUM>. The optical sensing pathway <NUM> detects audio pressure waves produced by one or more of the cartilage conduction transducer <NUM>, the air conduction transducer <NUM>, or the bone conduction transducer <NUM>.

In the embodiment shown in <FIG>, the optical sensing pathway <NUM> is an optical fiber through which light travels to detect acoustic pressure near the entrance to an ear <NUM> of the user. The light traveling through the optical fiber is a sensing beam that is transmitted by a laser housed in the frame <NUM>. The sensing beam travels through the optical fiber in a direction away from the frame <NUM>. The sensing beam is reflected at the end of the optical fiber and travels back through the optical fiber towards a detector, which is also be housed in the frame <NUM>. For example, the optical sensing pathway <NUM> may include a Fabry-Perot interferometer at the end near the entrance to the ear <NUM>. The Fabry-Perot interferometer includes a half mirror and a full mirror pointed towards each other, so that the sensing beam passes back and forth between these two mirrors. The mirrors may be separated by air or another medium. The acoustic pressure waves modulate the sensing beam as is passes between the two mirrors. In other embodiments, other types of interferometer configurations may be used. In some embodiments, light travels in a single direction through the optical fiber, and the optical sensing pathway <NUM> includes both a forward and return pathway (i.e., the optical sensing pathway <NUM> forms a loop). An acoustic pressure wave generated, either directly or indirectly, by one or more of the transducers <NUM>, <NUM>, or <NUM> interacts with the sensing beam as the sensing beam travels through the optical fiber (e.g., within the Fabry-Perot interferometer at the end of the optical fiber) such that the acoustic pressure wave alters an optical path length of the sensing beam. The optical microphone assembly <NUM> determines the optical path length of the sensing beam that traveled through the optical fiber, and measures the acoustic pressure wave based on the detected optical path length of the sensing beam. The components of the optical microphone assembly <NUM> are described in greater detail with respect to <FIG>.

As depicted in <FIG>, the optical sensing pathway <NUM> is an optical fiber that is suspended from the frame <NUM>, which is a housing of the audio system. In this case, the optical sensing pathway <NUM> extends directly from the frame <NUM> towards the entrance of the ear <NUM>. The optical sensing pathway <NUM> measures airborne acoustic waves produced by the transducers <NUM>, <NUM>, or <NUM>. For example, the optical sensing pathway <NUM> measures an airborne pressure wave directly produced by the air conduction transducer <NUM> and conducted through the air in the vicinity of the ear <NUM>. The optical sensing pathway <NUM> measures an airborne pressure wave indirectly produced by the cartilage conduction transducer <NUM> or the bone conduction transducer <NUM>, i.e., an airborne pressure wave that is produced from a tissue borne pressure wave. The length of the optical fiber may be either longer or shorter than it is depicted in <FIG>. A longer optical fiber may increase sensitivity of the optical microphone, while a shorter optical fiber may be less distracting to a user. In some embodiments, the optical sensing pathway <NUM> includes a rigid component extending from the frame <NUM> and a flexible optical fiber extending from the rigid component and positioned near the entrance of the ear <NUM>. In an embodiment, the optical sensing pathway <NUM> suspended from the housing of the audio system is configured to be coupled to tissue of the user.

The cartilage conduction transducer <NUM> is coupled to a portion of the back of an auricle of an ear <NUM> of a user. The cartilage conduction transducer <NUM> vibrates the back of auricle of the ear <NUM> of a user at first range of frequencies to generate a first range of airborne acoustic pressure waves at an entrance of the ear <NUM> based on audio instructions (e.g., from the controller). The air conduction transducer <NUM> is a speaker (e.g., a voice coil transducer) that vibrates over a second range of frequencies to generate a second range of airborne acoustic pressure waves at the entrance of the ear. The first and second ranges of frequencies may be different or may have some overlap. The first range of airborne acoustic pressure waves and the second range of airborne acoustic pressure waves travel from the entrance of the ear <NUM> down an ear canal <NUM> where an eardrum is located. The eardrum vibrates due to fluctuations of the airborne acoustic pressure waves which are then detected as sound by a cochlea of the user (not shown in <FIG>). The optical sensing pathway <NUM> and other components of the optical microphone assembly <NUM> are positioned at the entrance of the ear <NUM> of the user to detect the acoustic pressure waves produced by the cartilage conduction transducer <NUM> and the air conduction transducer <NUM>.

The bone conduction transducer <NUM> is coupled to a portion of the user's bone behind the user's ear <NUM>. The bone conduction transducer <NUM> vibrates over a third range of frequencies. The bone conduction transducer <NUM> vibrates the portion of the bone to which it is coupled. The portion of the bone conducts the vibrations to create a third range of tissue borne acoustic pressure waves at the cochlea which is then perceived by the user as sound. The vibration within the inner ear created by the bone conduction transducer <NUM> results in a weak airborne acoustic pressure wave outside the user's ear. The optical sensing pathway <NUM> and other components of the optical microphone assembly <NUM> are configured to detect the airborne acoustic pressure waves produced by the bone conduction transducer <NUM>.

More particularly, the bone conduction transducer <NUM> generates tissue borne pressure waves that travel through the user's bone (e.g., the mastoid) to the inner ear, which contains the cochlea. When the tissue borne pressure waves reach the inner ear, the waves within the inner ear vibrate the ear drum from the inside, which generates weak airborne pressure waves on the outside of the user's ear drum. For example, the airborne pressure waves outside the user's ear may result in particle displacements on the order of nanometers or picometers. These airborne pressure waves are too weak to be detected by typical binaural microphones or microphone arrays. However, the optical sensing pathway <NUM> is sensitive enough to detect particle displacements on the order of nanometers or picometers, and therefore can detect acoustic pressure waves generated by the bone conduction transducer <NUM>.

Although the portion of the audio system, as shown in <FIG>, illustrates one cartilage conduction transducer <NUM>, one air conduction transducer <NUM>, one bone conduction transducer <NUM>, and one optical sensing pathway <NUM> configured to produce and detect audio content for one ear <NUM> of the user, other embodiments include an identical setup to produce audio content for the other ear of the user. Other embodiments of the audio system comprise any combination of one or more cartilage conduction transducers, one or more air conduction transducers, and one or more bone conduction transducers. Examples of the audio system include a combination of cartilage conduction and bone conduction, another combination of air conduction and bone conduction, another combination of air conduction and cartilage conduction, etc..

<FIG> is a profile view <NUM> of a portion of an audio system including an optical microphone with a flexible membrane as a component of an eyewear device (e.g., the eyewear device <NUM>), in accordance with one or more embodiments. The transducer assembly <NUM> includes a cartilage conduction transducer <NUM>, an air conduction transducer <NUM>, and a bone conduction transducer <NUM>, which are similar to the cartilage conduction transducer <NUM>, air conduction transducer <NUM>, and bone conduction transducer <NUM> described with respect to <FIG>. The optical sensing pathway <NUM> is a component of an alternative embodiment of the optical microphone assembly <NUM>. The optical sensing pathway <NUM> detects airborne audio pressure waves produced by one or more of the cartilage conduction transducer <NUM>, the air conduction transducer <NUM>, or the bone conduction transducer <NUM>.

In the embodiment shown in <FIG>, the optical sensing pathway <NUM> includes an optical fiber <NUM> to which a membrane <NUM> is coupled. The membrane <NUM> is flexible, and the optical fiber <NUM> is attached to the membrane <NUM> in such a way that when the membrane <NUM> moves (e.g., in response to an acoustic pressure wave), the length of the optical sensing pathway <NUM> changes. The optical fiber <NUM> may be rigid, so that the changes in optical path length are generated by movement of the membrane <NUM>, rather than movement of the optical fiber <NUM>. The membrane <NUM> and optical fiber <NUM> are connected to the frame <NUM> and positioned in the vicinity of the ear canal <NUM>. As with the optical fiber in <FIG>, a sensing beam emitted by a laser housed in the frame <NUM> travels into and through the optical fiber <NUM>. The sensing beam is reflected by the membrane <NUM> and travels back through the optical fiber <NUM> towards a detector. The sensing beam output by the optical fiber <NUM> is directed towards the detector.

An acoustic pressure wave generated by one or more of the transducers <NUM>, <NUM>, or <NUM> interacts with the sensing beam as the sensing beam travels through the optical fiber such that the acoustic pressure wave alters an optical path length of the sensing beam. In particular, the membrane <NUM> moves with the detected acoustic pressure wave, and the movement of the membrane <NUM> causes a change in the optical path length of the optical fiber <NUM>. For example, when the acoustic pressure wave pushes the membrane <NUM> in the direction of the frame <NUM>, this shortens the optical path length compared to a neutral position of the membrane position <NUM>. The optical microphone assembly <NUM> determines the optical path length of the sensing beam that traveled through the optical fiber <NUM>, and measures the acoustic pressure wave based on the detected optical path length of the sensing beam. For example, the membrane <NUM> may vibrate with an acoustic pressure wave, and a detected amplitude of the vibrations, as measured by an amount of variation of the optical path length, may be correlated to an amplitude of the detected acoustic pressure wave. The coupled optical fiber <NUM> is sensitive to acoustic pressure waves on the order of nanometers or even picometers, allowing the detection at low volumes and detection of pressure waves generated by the bone conduction transducer <NUM>.

While in <FIG> the membrane <NUM> of the optical sensing pathway <NUM> is positioned near the entrance to the ear <NUM>, in other embodiments, the optical sensing pathway <NUM> and/or membrane <NUM> is located at a different position. For example, the optical fiber <NUM> and attached membrane <NUM> may be mounted directly on the frame <NUM>, rather than the optical fiber <NUM> extending from the frame <NUM> towards the entrance of the user's ear as shown in <FIG>. In other embodiments, the membrane <NUM> and coupled optical fiber <NUM> are coupled to tissue of a user's head. For example, the membrane <NUM> is coupled to the auricle of the ear <NUM> or to a bone in the user's head. Coupling the membrane <NUM> to a bone in the user's head may further improve detection of acoustic pressure waves generated by the bone conduction transducer <NUM>. In this example, the membrane <NUM> measures a tissue borne pressure wave, rather than an airborne pressure wave resulting from a tissue borne pressure wave. In some embodiments, the audio system includes multiple optical sensing pathways, e.g., one optical sensing pathway near the ear canal for detecting airborne acoustic pressure waves, and a second optical sensing pathways coupled to tissue for detecting tissue borne acoustic pressure waves.

As shown in <FIG>, the optical sensing pathway <NUM> has an optical fiber <NUM> that is suspended from the housing of the audio system (e.g., the frame <NUM>). In one embodiment, the optical sensing pathway <NUM> (e.g., the membrane <NUM>) is configured to be coupled to tissue of the user. In another embodiment, an end of the optical sensing pathway <NUM> (e.g., the membrane <NUM>) is configured to be suspended in air and positioned at an entrance to the ear of the user (e.g., the entrance to the ear canal <NUM>, as shown in <FIG>).

<FIG> is a block diagram of an audio system <NUM>, in accordance with one or more embodiments. The audio system in <FIG> is an embodiment of the audio system <NUM>. The audio system <NUM> includes one or more transducers <NUM>, an acoustic assembly <NUM>, and a controller <NUM>. In one embodiment, the audio system <NUM> further comprises an input interface. In other embodiments, the audio system <NUM> can have any combination of the components listed with any additional components. Similarly, the functions can be distributed among the components in a different manner than is described here.

The transducers <NUM> comprise any combination of one or more cartilage conduction transducers, one or more air conduction transducers, and one or more bone conduction transducers, in accordance with one or more embodiments. The transducers <NUM> provide sound to a user over a total range of frequencies. For example, the total range of frequencies is <NUM> - <NUM>, generally around the average range of human hearing. Each of the transducers <NUM> is configured to vibrate over various ranges of frequencies. In one embodiment, each of the transducers <NUM> operates over the total range of frequencies. In other embodiments, each transducer operates over a subrange of the total range of frequencies. In one embodiment, one or more transducers operate over a first subrange and one or more transducers operate over a second subrange. For example, a first transducer is configured to operate over a low subrange (e.g., <NUM> - <NUM>) while a second transducer is configured to operate over a medium subrange (e.g., <NUM> - <NUM>) and a third transducer is configured to operate over a high subrange (e.g., <NUM> - <NUM>). In another embodiment, subranges for the transducers <NUM> partially overlap with one or more other subranges.

In some embodiments, the transducers <NUM> include a cartilage conduction transducer. A cartilage conduction transducer is configured to vibrate a cartilage of a user's ear in accordance with audio instructions (e.g., received from the controller <NUM>). The cartilage conduction transducer is coupled to a portion of a back of an auricle of an ear of a user. The cartilage conduction transducer includes at least one transducer to vibrate the auricle over a first frequency range to cause the auricle to create an acoustic pressure wave in accordance with the audio instructions. Over the first frequency range, the cartilage conduction transducer can vary amplitude of vibration to affect amplitude of acoustic pressure waves produced. For example, the cartilage conduction transducer is configured to vibrate the auricle over a first frequency subrange of <NUM> - <NUM>. In one embodiment, the cartilage conduction transducer maintains good surface contact with the back of the user's ear and maintains a steady amount of application force (e.g., <NUM> Newton) to the user's ear. Good surface contact provides maximal translation of vibrations from the transducers to the user's cartilage.

In one embodiment, a transducer is a single piezoelectric transducer. A piezoelectric transducer can generate frequencies up to <NUM> using a range of voltages around +/- 100V. The range of voltages may include lower voltages as well (e.g., +/- 10V). The piezoelectric transducer may be a stacked piezoelectric actuator. The stacked piezoelectric actuator includes multiple piezoelectric elements that are stacked (e.g. mechanically connected in series). The stacked piezoelectric actuator may have a lower range of voltages because the movement of a stacked piezoelectric actuator can be a product of the movement of a single piezoelectric element with the number of elements in the stack. A piezoelectric transducer is made of a piezoelectric material that can generate a strain (e.g., deformation in the material) in the presence of an electric field. The piezoelectric material may be a polymer (e.g., polyvinyl chloride (PVC), polyvinylidene fluoroide (PVDF)), a polymer-based composite, ceramic, or crystal (e.g., quartz (silicon dioxide or SiO<NUM>), lead zirconate-titanate (PZT)). By applying an electric field or a voltage across a polymer which is a polarized material, the polymer changes in polarization and may compress or expand depending on the polarity and magnitude of the applied electric field. The piezoelectric transducer may be coupled to a material (e.g., silicone) that attaches well to an ear of a user.

In another embodiment, a transducer is a moving coil transducer. A typical moving coil transducer includes a coil of wire and a permanent magnet to produce a permanent magnetic field. Applying a current to the wire while it is placed in the permanent magnetic field produces a force on the coil based on the amplitude and the polarity of the current that can move the coil towards or away from the permanent magnet. The moving coil transducer may be made of a more rigid material. The moving coil transducer may also be coupled to a material (e.g., silicone) that attaches well to an ear of a user.

In some embodiments, the transducers <NUM> include an air conduction transducer. An air conduction transducer is configured to vibrate to generate acoustic pressure waves at an entrance of the user's ear in accordance with audio instructions (e.g., received from the controller <NUM>). The air conduction transducer is in front of an entrance of the user's ear. Optimally, the air conduction transducer is unobstructed, being able to generate acoustic pressure waves directly at the entrance of the ear. The air conduction transducer includes at least one transducer (substantially similar to the transducer described in conjunction with the cartilage conduction transducer) to vibrate over a second frequency range to create an acoustic pressure wave in accordance with the audio instructions. Over the second frequency range, the air conduction transducer can vary amplitude of vibration to affect amplitude of acoustic pressure waves produced. For example, the air conduction transducer is configured to vibrate over a second frequency subrange of <NUM> - <NUM> (or a higher frequency that is hearable by humans).

In some embodiments, the transducers <NUM> include a bone conduction transducer. A bone conduction transducer is configured to vibrate the user's bone to be detected directly by the cochlea in accordance with audio instructions (e.g., received from the controller <NUM>). The bone conduction transducer may be coupled to a portion of the user's bone. In one implementation, the bone conduction transducer is coupled to the user's skull behind the user's ear. In another implementation, the bone conduction transducer is coupled to the user's jaw. The bone conduction transducer includes at least one transducer (substantially similar to the transducer described in conjunction with the cartilage conduction transducer) to vibrate over a third frequency range in accordance with the audio instructions. Over the third frequency range, the bone conduction transducer can vary amplitude of vibration. For example, the bone conduction transducer assembly is configured to vibrate over a third frequency subrange of <NUM> (or a lower frequency that is hearable by humans) - <NUM>.

The microphone assembly <NUM> detects acoustic pressure waves at the entrance of the user's ear. The microphone assembly <NUM> is an optical microphone that includes an optical sensing pathway, such as one of the optical sensing pathways described with respect to <FIG> and <FIG>. One or more optical microphones may be positioned at an entrance of each ear of a user. The microphone assembly <NUM> is configured to detect the airborne acoustic pressure waves formed at an entrance of the user's ears. Alternatively or additionally, the microphone assembly <NUM> is configured to detect the airborne acoustic pressure waves formed at an entrance of the user's ears. In one embodiment, the microphone assembly <NUM> provides information regarding the produced sound to the controller <NUM>. The microphone assembly <NUM> transmits feedback information of the detected acoustic pressure waves to the controller <NUM>. An example of the microphone assembly <NUM> is described in greater detail with respect to <FIG>.

The controller <NUM> controls components of the audio system <NUM>. The controller <NUM> generates audio instructions to instruct the transducers <NUM> how to produce vibrations based on feedback from the microphone assembly <NUM>. For example, audio instructions may include a content signal (e.g., signal applied to any one of the transducers <NUM> to produce a vibration), a control signal to enable or disable any of the transducers <NUM>, and a gain signal to scale the content signal (e.g., increase or decrease amplitude of vibrations produced by any of the transducers <NUM>). For example, the controller <NUM> subdivides the audio instructions into different sets of audio instructions for different transducers <NUM>. A set of audio instructions controls a specific transducer. In some embodiments, the controller <NUM> subdivides the audio instructions for each transducer based on a frequency range for each transducer, based on a received selection of an audio source option from the user (e.g., via an input interface), or based on both the frequency range of each transducer and the received selection of an audio source option.

For example, the audio system <NUM> may comprise a cartilage conduction transducer, an air conduction transducer, and a bone conduction transducer. Following this example, the controller <NUM> may designate a first set of audio instructions for dictating vibration over a medium range of frequencies for the cartilage conduction transducer, a second set of audio instructions for dictating vibration over a high range of frequencies for the air conduction transducer, and a third set of audio instructions for dictating vibration over a low range of frequencies for the bone conduction transducer. In additional embodiments, the sets of audio instructions instruct the transducers <NUM> such that a frequency range of one transducer partially overlaps a frequency range of another transducer.

The controller <NUM> generates the content signal of the audio instructions based on portions of audio content and a frequency response model. The audio content to be provided may include sounds over the entire range of human hearing. The controller <NUM> takes the audio content and determines portions of the audio content to be provided by each of the transducers <NUM>. In one embodiment, the controller <NUM> determines portions of the audio content for each transducer based on the operable frequency range of that transducer. For example, the controller <NUM> determines a portion of the audio content within a range of <NUM> - <NUM> which may be the range of operation for a bone conduction transducer. The content signal may comprise a target waveform for vibrating of each of the transducers <NUM>. A frequency response model describes the response of audio system <NUM> to inputs at certain frequencies and may indicate how an output is shifted in amplitude and phase based on the input. With the frequency response model, the controller <NUM> may adjust the content signal so as to account for the shifted output. Thus, the controller <NUM> may generate a content signal of the audio instructions with the audio content (e.g., target output) and the frequency response model (e.g., relationship of the input to the output). In one embodiment, the controller <NUM> may generate the content signal of the audio instructions by applying an inverse of the frequency response to the audio content.

The controller <NUM> receives feedback from the microphone assembly <NUM>. The microphone assembly <NUM> provides information about the detected acoustic pressure waves produced by one or more of the transducers <NUM>. The controller <NUM> may compare the detected acoustic pressure waves with a target waveform based on audio content to be provided to the user. The controller <NUM> can then compute an inverse function to apply to the detected acoustic pressure waves such that the detected acoustic pressure waves match the target waveform. Thus, the controller <NUM> can update the frequency response model of the audio system using the computed inverse function specific to each user. The adjustment of the frequency model may be performed while the user is listening to audio content. The adjustment of the frequency model may also be conducted during a calibration of the audio system <NUM> for a user. The controller <NUM> can then generate updated audio instructions using the adjusted frequency response model. By updating audio instructions based on feedback from the microphone assembly <NUM>, the controller <NUM> can better provide a similar audio experience across different users of the audio system <NUM>.

In some embodiments of the audio system <NUM> with any combination of a cartilage conduction transducer, an air conduction transducer, and a bone conduction transducer, the controller <NUM> updates the audio instructions so as to affect varying changes of operation to each of the transducers <NUM>. As each auricle of a user is different (e.g., shape and size), the frequency response model will vary from user to user. By adjusting the frequency response model for each user based on audio feedback captured by the microphone assembly <NUM>, the audio system can maintain the same type of produced sound (e.g., neutral listening) regardless of the user. Neutral listening is having similar listening experience across different users. In other words, the listening experience is impartial or neutral to the user (e.g., does not change from user to user).

In another embodiment, the audio system uses a flat spectrum broadband signal to generate the adjusted frequency response model. For example, the controller <NUM> provides audio instructions to the transducers <NUM> based on a flat spectrum broadband signal. The microphone assembly <NUM> detects acoustic pressure waves at the entrance of user's ear. The controller <NUM> compares the detected acoustic pressure waves with the target waveform based on the flat spectrum broadband signal and adjusts the frequency model of the audio system accordingly. In this embodiment, the flat spectrum broadband signal may be used while performing calibration of the audio system for a particular user. Thus, the audio system may perform an initial calibration for a user instead of continuously monitoring the audio system. In this embodiment, the microphone assembly <NUM> may be temporarily coupled to the audio system <NUM> for calibration of the user. For example, after calibration, the optical sensing pathway <NUM> or <NUM> can be removed from the eyewear device to improve comfort to the user.

In some embodiments, the controller <NUM> manages calibration of the audio system <NUM>. The controller <NUM> generates calibration instructions for each of the transducers <NUM>. Calibration instructions may instruct one or more transducers to generate an acoustic pressure wave that corresponds to a target waveform. In some embodiments, the acoustic pressure wave may correspond to, e.g., a tone or a set of tones. In other embodiments, the acoustic pressure wave may correspond to audio content (e.g., music) that is being presented to the user. The controller <NUM> may send the calibration instructions to the transducers <NUM> one at a time or multiple at a time. As a transducer receives the calibration content, the transducer generates acoustic pressure waves in accordance with the calibration instructions. The microphone assembly <NUM> detects the acoustic pressure waves and sends the detected acoustic pressure waves to the controller <NUM>. The controller <NUM> compares the detected acoustic pressure waves to the target waveform. The controller <NUM> can then modify the calibration instructions such that the transducers <NUM> emit an acoustic pressure wave that is closer to the target waveform. The controller <NUM> can repeat this process in until the difference between the target waveform and the detected acoustic pressure waves is within some threshold value. In one embodiment where each transducer is calibrated individually, the controller <NUM> compares the calibration content sent to the transducer against the detected acoustic pressure waves by the microphone assembly <NUM>. The controller <NUM> may generate a frequency response model based on the calibration for that transducer assembly. Responsive to completing calibration of the user, the microphone assembly <NUM> may be uncoupled from the audio system <NUM>. Advantages of removing the microphone assembly <NUM> include making the audio system <NUM> easier to wear, reducing volume and weight of the audio system <NUM> and potentially an eyewear device (e.g., eyewear device <NUM>, eyewear device <NUM>, or eyewear device <NUM>) of which the audio system <NUM> is a component, and reducing power consumption of the audio system <NUM>.

<FIG> is a block diagram of a microphone assembly <NUM> of the audio system, in accordance with one or more embodiments. The microphone assembly <NUM> shown in <FIG> is an optical Mach-Zehnder interferometer that includes a laser <NUM> for generating a beam of light, a beam splitter <NUM> for splitting the beam into a reference beam and a sensing beam, an optical sensor <NUM> through which the sensing beam passes, a reference beam modulator <NUM> through which the reference beam passes, and a detector assembly <NUM> for measuring the sensed acoustic pressure wave based on the sensing beam and the reference beam.

The laser <NUM> emits a beam of light. The laser <NUM> may be any coherent light source, such as a laser diode. The laser <NUM> is coupled into a beam splitter <NUM>. The beam splitter <NUM> is a device configured to separate the light beam emitted by the laser <NUM> into a first beam of light and a second beam of light. For example, the beam splitter <NUM> may be a half-silvered mirror, a pair of glass prisms, or a dichroic mirrored prism. The first beam is a sensing beam used to detect an acoustic pressure wave, and the second beam is a reference beam that is used to detect changes in the sensing beam.

The first beam (i.e., the sensing beam) is coupled into an optical sensor <NUM>. The optical sensor <NUM> includes an optical sensing pathway through which the sensing beam travels. In some embodiments, the optical sensing pathway is configured to move with a detected acoustic pressure wave so that the sensing beam can sense the acoustic pressure wave. For example, the optical sensor <NUM> may be an optical fiber with a Fabry-Perot interferometer as described with respect to <FIG>, or an optical fiber coupled to a flexible membrane, as described with respect to <FIG>.

A detected acoustic pressure wave interacts with the optical sensor <NUM> to alter an optical path length of the sensing beam. Optical path length is the product of the geometric length of the path that the sensing beam travels and the index of refraction of the material through which the sensing beam travels (e.g., the index of refraction of the optical fiber, or the index of refraction of the cavity between the mirrors in a Fabry-Perot interferometer). The detected acoustic pressure wave creates particle displacements in a transmission medium through which the acoustic pressure wave travels. For an airborne wave detected by an optical sensing pathway suspended in air and positioned at an entrance to the user's ear, such as the optical sensing pathways shown in <FIG> and <FIG>, the transmission medium is air. For a tissue borne pressure wave detected by an optical sensing pathway coupled to tissue of the user's head (e.g., in the vicinity of the bone conduction transducer shown in <FIG> and <FIG>), the transmission medium is tissue, such as bone or cartilage. In either case, the particle displacements created by the acoustic pressure wave vibrate the optical sensing pathway (e.g., the membrane or the Fabry-Perot interferometer), and this vibration alters the geometric path length of the sensing beam through the optical sensing pathway, and thus alters the optical path length. Measuring changes to the optical path length provides a measurement of the particle displacements, which corresponds to a measurement of acoustic pressure.

The second beam (i.e., the reference beam) is coupled to a reference beam modulator <NUM>. The microphone assembly <NUM> compares the sensing beam to the reference beam to measure the change to the optical path length of the sensing beam caused by the acoustic pressure wave. The reference beam modulator <NUM> modulates a parameter of the reference beam so that the detector assembly <NUM> can identify the reference beam based on the modified parameter and distinguish the reference beam from the sensing beam. For example, the reference beam modulator <NUM> can modulate the amplitude or frequency of the reference beam.

The modulated reference beam output by the reference beam modulator <NUM> and the sensing beam output by the optical sensor <NUM> are coupled into a detector assembly <NUM>. In some embodiments, the modulated reference beam and the sensing beam are recombined prior to entering the detector assembly <NUM>, as shown in <FIG>. For example, the reference beam and the sensing beam output by the reference beam modulator <NUM> and the optical sensor <NUM>, respectively, may enter a second beam splitter that outputs a combined reference beam and sensing beam. The change to the optical path length is observed in a change in the phase of the sensing beam after passing through the optical sensor <NUM>. The change in phase of the sensing beam can be determined by comparing the phase of the sensing beam to the phase of the reference beam.

In the example shown in <FIG>, the detector assembly <NUM> includes a photodetector <NUM> and a signal processor <NUM>. The combined modulated reference beam and the sensing beam are coupled into the photodetector <NUM>. The photodetector <NUM> is a device that receives light and converts the light into an electrical current. The combination of the reference beam and sensing beam yields constructive interference, and the amount of light detected at the photodetector <NUM> is related to the relative phases of the reference beam and the sensing beam. The signal processor <NUM> receives the current generated by the photodetector <NUM> and converts the current into a measurement of the detected acoustic pressure wave. In particular, the signal processor <NUM> determines changes in the optical path length of the sensing beam relative to the reference beam based on the relative phases of the sensing beam and the reference beam, and determines a measurement of the acoustic pressure wave based on the changes in optical path length. The signal processor <NUM> transmits the measurement of the acoustic pressure wave to the controller <NUM>, which adjusts an audio instruction for one or more of the transducers <NUM> based on the measurement, as discussed above.

The signal processor <NUM> may also distinguish between changes to optical path length caused by acoustic pressure waves and changes to optical path length due to other factors. For example, if the audio system <NUM> is worn by a user in motion, motion of the user's head may cause changes to the optical path length, e.g., due to motion of the optical fiber. As an example, the signal processor <NUM> processes the received signal to identify frequencies of detected changes to optical path length, and selects portions of the signal in a range of frequencies that corresponds to acoustic pressure waves (e.g., <NUM> - <NUM>). Changes to optical path length caused by physical motion are typically lower frequency, so the signal processor <NUM> can remove the portion of the received signal caused by physical motion as noise. In some embodiments, the controller <NUM> instructs a transducer <NUM> to produce acoustic pressure waves at a particular frequency, set of frequencies, or range of frequencies, and transmits this frequency information to the signal processor <NUM>. In such embodiments, the signal processor <NUM> measures the portion of the received signal that matches the frequencies of the produced acoustic pressure waves.

While <FIG> shows an optical microphone assembly <NUM> based on a Mach-Zehnder interferometer, in other embodiments, alternate detection devices can be used. For example, the optical microphone assembly may be based on a Michelson interferometer, a Fizeau interferometer, or another type of optical interferometer or optical detection apparatus.

<FIG> is a system environment <NUM> of an eyewear device including an audio system, in accordance with one or more embodiments. The system <NUM> may operate in an artificial reality environment, e.g., a virtual reality, an augmented reality, a mixed reality environment, or some combination thereof. The system <NUM> shown by <FIG> comprises an eyewear device <NUM> and an input/output (I/O) interface <NUM> that is coupled to a console <NUM>. The eyewear device <NUM> may be an embodiment of the eyewear device <NUM>. While <FIG> shows an example system <NUM> including one eyewear device <NUM> and one I/O interface <NUM>, in other embodiments, any number of these components may be included in the system <NUM>. For example, there may be multiple eyewear devices <NUM> each having an associated I/O interface <NUM> with each eyewear device <NUM> and I/O interface <NUM> communicating with the console <NUM>. In alternative configurations, different and/or additional components may be included in the system <NUM>. Additionally, functionality described in conjunction with one or more of the components shown in <FIG> may be distributed among the components in a different manner than described in conjunction with <FIG> in some embodiments. For example, some or all of the functionality of the console <NUM> is provided by the eyewear device <NUM>.

The eyewear device <NUM> may be a HMD that presents content to a user comprising augmented views of a physical, real-world environment with computer-generated elements (e.g., two dimensional (2D) or three dimensional (3D) images, 2D or 3D video, sound, etc.). In some embodiments, the presented content includes audio that is presented via an audio system <NUM> that receives audio information from the eyewear device <NUM>, the console <NUM>, or both, and presents audio data based on the audio information. In some embodiments, the eyewear device <NUM> presents virtual content to the user that is based in part on a real environment surrounding the user. For example, virtual content may be presented to a user of the eyewear device. The user physically may be in a room, and virtual walls and a virtual floor of the room are rendered as part of the virtual content.

The eyewear device <NUM> includes the audio system <NUM> of <FIG>. The audio system <NUM> includes one or more sound conduction methods and an optical microphone assembly for detecting the produced sound. As mentioned above, the audio system <NUM> may include any combination of one or more cartilage conduction transducers, one or more air conduction transducers, and one or more bone conduction transducers. The audio system <NUM> provides audio content to the user of the eyewear device <NUM>. The audio system <NUM> uses the optical microphone to monitor the produced sound so that it can compensate for a frequency response model for each ear of the user and can maintain consistency with produced sound across different individuals using the eyewear device <NUM>.

The eyewear device <NUM> may include a depth camera assembly (DCA) <NUM>, an electronic display <NUM>, an optics block <NUM>, one or more position sensors <NUM>, and an inertial measurement Unit (IMU) <NUM>. The electronic display <NUM> and the optics block <NUM> is one embodiment of a lens <NUM>. The position sensors <NUM> and the IMU <NUM> is one embodiment of sensor device <NUM>. Some embodiments of the eyewear device <NUM> have different components than those described in conjunction with <FIG>. Additionally, the functionality provided by various components described in conjunction with <FIG> may be differently distributed among the components of the eyewear device <NUM> in other embodiments, or be captured in separate assemblies remote from the eyewear device <NUM>.

The DCA <NUM> captures data describing depth information of a local area surrounding some or all of the eyewear device <NUM>. The DCA <NUM> may include a light generator, an imaging device, and a DCA controller that may be coupled to both the light generator and the imaging device. The light generator illuminates a local area with illumination light, e.g., in accordance with emission instructions generated by the DCA controller. The DCA controller is configured to control, based on the emission instructions, operation of certain components of the light generator, e.g., to adjust an intensity and a pattern of the illumination light illuminating the local area. In some embodiments, the illumination light may include a structured light pattern, e.g., dot pattern, line pattern, etc. The imaging device captures one or more images of one or more objects in the local area illuminated with the illumination light. The DCA <NUM> can compute the depth information using the data captured by the imaging device or the DCA <NUM> can send this information to another device such as the console <NUM> that can determine the depth information using the data from the DCA <NUM>.

The electronic display <NUM> displays 2D or 3D images to the user in accordance with data received from the console <NUM>. In various embodiments, the electronic display <NUM> comprises a single electronic display or multiple electronic displays (e.g., a display for each eye of a user). Examples of the electronic display <NUM> include: a liquid crystal display (LCD), an organic light emitting diode (OLED) display, an active-matrix organic light-emitting diode display (AMOLED), some other display, or some combination thereof. The electronic display <NUM> may be a waveguide display.

In some embodiments, the optics block <NUM> magnifies image light received from the electronic display <NUM>, corrects optical errors associated with the image light, and presents the corrected image light to a user of the eyewear device <NUM>. In various embodiments, the optics block <NUM> includes one or more optical elements. Example optical elements included in the optics block <NUM> include: a waveguide, an aperture, a Fresnel lens, a convex lens, a concave lens, a filter, a reflecting surface, or any other suitable optical element that affects image light. Moreover, the optics block <NUM> may include combinations of different optical elements. In some embodiments, one or more of the optical elements in the optics block <NUM> may have one or more coatings, such as partially reflective or anti-reflective coatings.

Magnification and focusing of the image light by the optics block <NUM> allows the electronic display <NUM> to be physically smaller, weigh less, and consume less power than larger displays. Additionally, magnification may increase the field of view of the content presented by the electronic display <NUM>. For example, the field of view of the displayed content is such that the displayed content is presented using almost all (e.g., approximately <NUM> degrees diagonal), and in some cases all, of the user's field of view. Additionally, in some embodiments, the amount of magnification may be adjusted by adding or removing optical elements.

In some embodiments, the optics block <NUM> may be designed to correct one or more types of optical error. Examples of optical error include barrel or pincushion distortion, longitudinal chromatic aberrations, or transverse chromatic aberrations. Other types of optical errors may further include spherical aberrations, chromatic aberrations, or errors due to the lens field curvature, astigmatisms, or any other type of optical error. In some embodiments, content provided to the electronic display <NUM> for display is pre-distorted, and the optics block <NUM> corrects the distortion when it receives image light from the electronic display <NUM> generated based on the content.

The IMU <NUM> is an electronic device that generates data indicating a position of the eyewear device <NUM> based on measurement signals received from one or more of the position sensors <NUM>. A position sensor <NUM> generates one or more measurement signals in response to motion of the eyewear device <NUM>. Examples of position sensors <NUM> include: one or more accelerometers, one or more gyroscopes, one or more magnetometers, another suitable type of sensor that detects motion, a type of sensor used for error correction of the IMU <NUM>, or some combination thereof. The position sensors <NUM> may be located external to the IMU <NUM>, internal to the IMU <NUM>, or some combination thereof.

Based on the one or more measurement signals from one or more position sensors <NUM>, the IMU <NUM> generates data indicating an estimated current position of the eyewear device <NUM> relative to an initial position of the eyewear device <NUM>. For example, the position sensors <NUM> include multiple accelerometers to measure translational motion (forward/back, up/down, left/right) and multiple gyroscopes to measure rotational motion (e.g., pitch, yaw, and roll). In some embodiments, the IMU <NUM> rapidly samples the measurement signals and calculates the estimated current position of the eyewear device <NUM> from the sampled data. For example, the IMU <NUM> integrates the measurement signals received from the accelerometers over time to estimate a velocity vector and integrates the velocity vector over time to determine an estimated current position of a reference point on the eyewear device <NUM>. Alternatively, the IMU <NUM> provides the sampled measurement signals to the console <NUM>, which interprets the data to reduce error. The reference point is a point that may be used to describe the position of the eyewear device <NUM>. The reference point may generally be defined as a point in space or a position related to the eyewear device's <NUM> orientation and position.

The I/O interface <NUM> is a device that allows a user to send action requests and receive responses from the console <NUM>. An action request is a request to perform a particular action. For example, an action request may be an instruction to start or end capture of image or video data, or an instruction to perform a particular action within an application. The I/O interface <NUM> may include one or more input devices. Example input devices include: a keyboard, a mouse, a game controller, or any other suitable device for receiving action requests and communicating the action requests to the console <NUM>. An action request received by the I/O interface <NUM> is communicated to the console <NUM>, which performs an action corresponding to the action request. In some embodiments, the I/O interface <NUM> includes an IMU <NUM>, as further described above, that captures calibration data indicating an estimated position of the I/O interface <NUM> relative to an initial position of the I/O interface <NUM>. In some embodiments, the I/O interface <NUM> may provide haptic feedback to the user in accordance with instructions received from the console <NUM>. For example, haptic feedback is provided when an action request is received, or the console <NUM> communicates instructions to the I/O interface <NUM> causing the I/O interface <NUM> to generate haptic feedback when the console <NUM> performs an action.

The console <NUM> provides content to the eyewear device <NUM> for processing in accordance with information received from one or more of: the eyewear device <NUM> and the I/O interface <NUM>. In the example shown in <FIG>, the console <NUM> includes an application store <NUM>, a tracking module <NUM> and an engine <NUM>. Some embodiments of the console <NUM> have different modules or components than those described in conjunction with <FIG>. Similarly, the functions further described below may be distributed among components of the console <NUM> in a different manner than described in conjunction with <FIG>.

The application store <NUM> stores one or more applications for execution by the console <NUM>. An application is a group of instructions, that when executed by a processor, generates content for presentation to the user. Content generated by an application may be in response to inputs received from the user via movement of the eyewear device <NUM> or the I/O interface <NUM>. Examples of applications include: gaming applications, conferencing applications, video playback applications, or other suitable applications.

The tracking module <NUM> calibrates the system environment <NUM> using one or more calibration parameters and may adjust one or more calibration parameters to reduce error in determination of the position of the eyewear device <NUM> or of the I/O interface <NUM>. Calibration performed by the tracking module <NUM> also accounts for information received from the IMU <NUM> in the eyewear device <NUM> and/or an IMU <NUM> included in the I/O interface <NUM>. Additionally, if tracking of the eyewear device <NUM> is lost, the tracking module <NUM> may re-calibrate some or all of the system environment <NUM>.

The tracking module <NUM> tracks movements of the eyewear device <NUM> or of the I/O interface <NUM> using information from the one or more position sensors <NUM>, the IMU <NUM>, the DCA <NUM>, or some combination thereof. For example, the tracking module <NUM> determines a position of a reference point of the eyewear device <NUM> in a mapping of a local area based on information from the eyewear device <NUM>. The tracking module <NUM> may also determine positions of the reference point of the eyewear device <NUM> or a reference point of the I/O interface <NUM> using data indicating a position of the eyewear device <NUM> from the IMU <NUM> or using data indicating a position of the I/O interface <NUM> from an IMU <NUM> included in the I/O interface <NUM>, respectively. Additionally, in some embodiments, the tracking module <NUM> may use portions of data indicating a position or the eyewear device <NUM> from the IMU <NUM> to predict a future location of the eyewear device <NUM>. The tracking module <NUM> provides the estimated or predicted future position of the eyewear device <NUM> or the I/O interface <NUM> to the engine <NUM>.

The engine <NUM> also executes applications within the system environment <NUM> and receives position information, acceleration information, velocity information, predicted future positions, or some combination thereof, of the eyewear device <NUM> from the tracking module <NUM>. Based on the received information, the engine <NUM> determines content to provide to the eyewear device <NUM> for presentation to the user. For example, if the received information indicates that the user has looked to the left, the engine <NUM> generates content for the eyewear device <NUM> that mirrors the user's movement in a virtual environment or in an environment augmenting the local area with additional content. Additionally, the engine <NUM> performs an action within an application executing on the console <NUM> in response to an action request received from the I/O interface <NUM> and provides feedback to the user that the action was performed. The provided feedback may be visual or audible feedback via the eyewear device <NUM> or haptic feedback via the I/O interface <NUM>.

The foregoing description of the embodiments of the disclosure has been presented for the purpose of illustration; it is not intended to be exhaustive or to limit the disclosure to the precise forms disclosed. Persons skilled in the relevant art can appreciate that many modifications and variations are possible in light of the above disclosure.

Some portions of this description describe the embodiments of the disclosure in terms of algorithms and symbolic representations of operations on information. These algorithmic descriptions and representations are commonly used by those skilled in the data processing arts to convey the substance of their work effectively to others skilled in the art. These operations, while described functionally, computationally, or logically, are understood to be implemented by computer programs or equivalent electrical circuits, microcode, or the like. Furthermore, it has also proven convenient at times, to refer to these arrangements of operations as modules, without loss of generality. The described operations and their associated modules may be embodied in software, firmware, hardware, or any combinations thereof.

Embodiments of the disclosure may also relate to an apparatus for performing the operations herein. This apparatus may be specially constructed for the required purposes, and/or it may comprise a general-purpose computing device selectively activated or reconfigured by a computer program stored in the computer. Such a computer program may be stored in a non-transitory, tangible computer readable storage medium, or any type of media suitable for storing electronic instructions, which may be coupled to a computer system bus. Furthermore, any computing systems referred to in the specification may include a single processor or may be architectures employing multiple processor designs for increased computing capability.

Embodiments of the disclosure may also relate to a product that is produced by a computing process described herein. Such a product may comprise information resulting from a computing process, where the information is stored on a non-transitory, tangible computer readable storage medium and may include any embodiment of a computer program product or other data combination described herein.

Claim 1:
An audio system (<NUM>) comprising:
a transducer assembly (<NUM>) configured to be coupled to an ear of a user and to produce an acoustic pressure wave based on an audio instruction; an optical fiber (<NUM>;<NUM>) wherein an end of the optical fiber (<NUM>; <NUM>) is configured to be suspended in air and positioned at an entrance of an ear canal (<NUM>;<NUM>) of the ear (<NUM>;<NUM>);
a laser (<NUM>) configured to emit light that is separated into a reference beam and a sensing beam, and the sensing beam is coupled into the optical fiber, wherein movement of the optical fiber caused by the acoustic pressure wave interacts with the sensing beam in the optical fiber to alter an optical path length of the sensing beam;
a detector assembly (<NUM>) configured to:
detect the reference beam and detect the sensing beam from the optical fiber, and
measure a detected acoustic pressure wave based in part on changes in optical path length between the reference beam and the sensing beam; and
a controller (<NUM>) configured to adjust the audio instruction based on the measurement of the detected acoustic pressure wave.