Spatial audio warp compensator

Methods and devices for correcting warping in spatial audio may include identifying a geometric transform that defines a geometric warping between a first spatial geometric model that represents how sound is produced in a first volumetric space and a second spatial geometric model that represents how sound is produced in a second volumetric space different from the first volumetric space. The methods and devices may include determining an inverse of the geometric transform that compensates for the geometric transform. The methods and devices may include applying the inverse of the geometric transform to a first location in the first spatial geometric model by mapping the first location to a second location in the second spatial geometric model to correct for the geometric warping.

BACKGROUND

The present disclosure relates to spatial audio, and more particularly, to the use of spatial audio on a computer device.

Spatial audio provides the ability to place sounds about the listener using volumetric coordinates. For example, the listener may be placed at an origin point represented as (x, y, z), where x=0, y=0 and z=0 or (0, 0, 0). A sound can be placed at any position represented as a combination of values of (x, y, z).

There are various methods of rendering spatial audio such that the listener experiences the sound as originating from the position specified, however, some spatial rendering technologies cannot maintain relative geometric positioning and velocity due to geometric distortions (e.g., warping) introduced in the rendering process. For example, spatial rendering technologies that rely on physical speakers placed around the listener to achieve the specialization effect can exhibit spatial audio warping as a physical room geometry, e.g., the relative locations and number of the physical speakers, in which the spatial audio content is rendered can be different from the geometry used to create the spatial audio content. Because the physical room geometry is not known when content is authored, the spatial audio content must be authored to a standardized or normalized geometry that abstracts the physical room into a known layout or geometry (referred to as a normalized room or spatial geometry).

In particular, a spatial audio renderer maps the normalized room geometry into the physical room geometry. The conversion from the normalized room geometry to the physical room geometry can result in a warping in geometric space. For instance, if the author animated a sound in a perfect circle about the listener at a constant velocity based on the normalized room geometry, then the result generated by the spatial audio renderer in a differently configured physical room geometry would not be a perfect circle and the velocity would not be constant, resulting in a warping of both space and time.

Thus, there is a need in the art for improvements in spatial audio.

SUMMARY

One example implementation relates to a computer device. The computer device may include a memory to store data and instructions, a processor in communication with the memory, and an operating system in communication with the memory and processor. The operating system may be operable to identify a geometric transform that defines a geometric warping between a first spatial geometric model that represents how sound is produced in a first volumetric space and a second spatial geometric model that represents how sound is produced in a second volumetric space different from the first volumetric space, determine an inverse of the geometric transform that compensates for the geometric transform, and apply the inverse of the geometric transform to a first location in the first spatial geometric model by mapping the first location to a second location in the second spatial geometric model to correct for the geometric warping.

Another example implementation relates to a method for correcting warping in spatial audio. The method may include identifying, at an operating system executing on a computer device, a geometric transform that defines a geometric warping between a first spatial geometric model that represents how sound is produced in a first volumetric space and a second spatial geometric model that represents how sound is produced in a second volumetric space different from the first volumetric space. The method may also include determining, at the operating system, an inverse of the geometric transform that compensates for the geometric transform. The method may also include applying the inverse of the geometric transform to a first location in the first spatial geometric model by mapping the first location to a second location in the second spatial geometric model to correct for the geometric warping.

Another example implementation relates to computer-readable medium storing instructions executable by a computer device. The computer-readable medium may include at least one instruction for causing the computer device to identify a geometric transform that defines a geometric warping between a first spatial geometric model that represents how sound is produced in a first volumetric space and a second spatial geometric model that represents how sound is produced in a second volumetric space different from the first volumetric space. The computer-readable medium may include at least one instruction for causing the computer device to determine an inverse of the geometric transform that compensates for the geometric transform. The computer-readable medium may include at least one instruction for causing the computer device to apply the inverse of the geometric transform to a first location in the first spatial geometric model by mapping the first location to a second location in the second spatial geometric model to correct for the geometric warping.

Additional advantages and novel features relating to implementations of the present disclosure will be set forth in part in the description that follows, and in part will become more apparent to those skilled in the art upon examination of the following or upon learning by practice thereof.

DETAILED DESCRIPTION

The present disclosure provides systems and methods for compensating for warping, such as geometric and/or velocity (e.g., space and/or time) warping, during authoring of spatial audio content. The systems and methods provide a first spatial geometric model that represents how sound is produced relative to a fixed position within the first spatial geometric model. In addition, the systems and methods provide a differently-configured second spatial geometric model that represents how sound is produced relative to the fixed position within the second spatial geometric model. The models may represent a physical location and/or a virtual location. In an implementation, the systems and methods may allow a user to specify the spatial geometric models that represent the volumetric space where audio will be consumed. For example, a user may input a room size, a number of static audio objects to place in the room, a number of dynamic audio objects to place in the room, and/or locations of the static and dynamic audio objects in the room.

The systems and methods may compensate for geometric and velocity warping that may occur during rendering of spatial audio content authored due to a difference between the first spatial geometric model that the author uses to create the spatial audio content and the second spatial geometric model that represents how the sound will be produced when rendered. The systems and methods may compute a three dimensional (3D) transformation that defines a geometric warping resulting from the differences between spatial geometric models and may encode an inverse of the geometric warping into one of the spatial geometric models. Thus, based on applying of the inverse geometric model, during rendering of the spatial audio content in the second spatial geometric model, the original author-intended geometry of the spatial audio content associated with the first spatial geometric model is restored. The systems and methods may efficiently map between multiple spatial geometries in a single mapping pass.

The present solution may be utilized with any spatial audio rendering technology that uses physical speakers placed around the listener to achieve a specialization effect where the listener experiences the sound as originating from specified positions. Because the physical room geometry may not be known when the spatial audio content is authored, the content must be authored to a normalized room geometry that abstracts the room into a known layout.

When authoring spatial audio content, there are at least two types of audio objects defined within the volumetric space, a static audio object and a dynamic audio object. A static audio object represents a sound that will play back through a specific speaker (or audio channel). For example, spatial audio content rendered to a static audio object defined as Front-Right will result in audio coming out of the Front-Right speaker, and only that speaker, if that speaker is present at the rendering stage. If the speaker is not present, the renderer will balance the sound across adjacent speakers, referred to as panning.

A dynamic audio object represents a sound that will play back not through a specific speaker/channel, but instead will playback though any and all channels necessary to achieve the spatial illusion, based on its position in the volumetric space. A dynamic audio object may be placed anywhere in volumetric space and sound may be rendered out of the physical speakers represented by the static audio objects. Which static audio objects are needed to “project” the dynamic audio object into the volumetric space, and the corresponding amount of signal power needed, may be mathematically determined by, for example, a spatial renderer.

Generally, the following criteria may be associated with a normalized room geometry in order to work as a spatial audio volumetric coordinate system. First, every static audio object supported by the spatial encoding format must be represented in the normalized room geometry. Because the normalized room geometry has no knowledge of what physical speaker may or may not be present in the physical room geometry, all speakers are assumed to be present. Second, for each static audio object, there must be a specified position in the volumetric coordinate system (e.g., in x, y, z coordinates), such that any dynamic object positioned at the same location will render audio only though the corresponding speaker.

When spatial audio content is authored, there may be no knowledge of the physical room geometry in which sound may be rendered. As such, an author creates spatial audio content relative to the normalized room geometry (e.g., a known layout for the room). The spatial audio renderer (e.g., a spatial encoder) maps the normalized room geometry into the physical room geometry in which sound is to be rendered.

In some cases, the conversion from a normalized room geometry to the physical room geometry results in a warping in geometric space, which, in turn, results in temporal warping of animated dynamic objects traversing the warped geometry over time. For example, the normalized room geometry may specify a listener position at origin (0, 0, 0) and a front-left static object at position (−1, 1, 0). In the physical room, assume the listener is facing forward toward a front-center static object at position (0,−1, 0) and the relative angle to the listener of the front-left static object is 45 degrees to the left relative to the listener, in the same horizontal plane as the listener. For rendering spatial audio content in the physical room geometry, however, the front-left speaker corresponding to the front left static object in the normalized room geometry has a recommended placement of 30 degrees relative to the listener. As such, there may be a relative warping of 15 degrees from what was authored versus what will be heard by the listener. When applying the warping to each speaker separately, variable warping may occur. If the author animated a sound in a perfect circle about the listener at a constant velocity, the result may not be a perfect circle and the velocity may not be constant. Thus, a warping in both space and time may occur (e.g., a space-time warping). The present solution compensates for this warping by applying an inverse of the warping to enable the spatial audio content to be rendered in the physical room geometry at the intended position relative to the listener as defined in the normalized room geometry that was used to author the spatial audio content.

Referring now toFIGS. 1A-1C, example views of a normalized room geometry100(which may be referred to as an authoring model) represent a room which is 2 units wide, 2 units deep, and 2 units tall, and having a listener9in the center of the room at geometric position (0,0,0). A distance to all “walls” of the room from listener is 1 unit. For example, the model for the normalized room geometry100may include static audio objects such as one or more Front-Center (FC) speakers positioned at location (0,1,0), one or more Back-Center (BC) speakers positioned at location (0,−1,0), one or more Left-Side (LS) speakers positioned at (−1,0,0), and one or more Right-Side (RS) speakers positioned at (1,0,0). In addition, the model for normalized room geometry100may place static audio objects such as one or more Front-Left (FL) speakers in the corner at (−1,1,0) and one or more Front-Right (FR) speakers in the corner at (1,1,0), and so on. The model for the normalized room geometry100may also place static audio objects such as one or more upper and lower speakers (TopFrontLeft, TopFrontRight, TopBackLeft, TopBackRight, and corresponding four bottom speakers) at the center of each of the four ceiling and lower quadrants. The normalized room geometry100may vary from encoder to encoder.

Referring now toFIGS. 2A-2C, an example of a physical room geometry200(which may be referred to as a rendering model) in accordance with an implementation includes different positioning of each static audio object relative to the listener9, as compared to the normalized room geometry100ofFIGS. 1A-1C. The geometric configuration of physical room geometry200demonstrate one of many possible differences in layout of the geometry between the physical room geometry and the normalized room geometry. In particular, any difference between any of the static audio objects between the two geometries can cause space and time warping of spatial audio content. For example, as compared to the normalized room geometry100ofFIGS. 1A-1C, the physical room geometry200may have static audio objects such as one or more front center speakers relatively closer to the listener9, and one or more front left speakers and front right speakers at different relative angles (e.g., placed at +/−30 degrees, respectively, relative to the listener9), and one or more left side and right side speakers, respectively, in different relative positions with respect to the listener9. In addition, the physical room geometry200may have static audio objects such as one or more ceiling and floor speakers positioned closer to the listener9as compared to the normalized room geometry100ofFIGS. 1A-1C. Also, in the physical room geometry200, static audio objects such as one or more back speakers (e.g., back center (BC), back left, and back right) may be positioned physically further away from and/or at different angles relative to the listener9. The placement of each static audio object in the physical room geometry200may be based on, for example, recommended speaker positions and angles of speakers by a specific spatial audio encoder technology for an “optimal audio experience.”

Geometric warping in the rendering of spatial audio content may be caused by the difference between the normalized room geometry100ofFIGS. 1A-1C(e.g., the authoring model) and the physical room geometry200ofFIGS. 2A-2C(the rendering model).

For example, spatial audio content in the form of a sphere300about the listener9authored in the normalized room geometry100is illustrated inFIGS. 3A-3C.

In contrast,FIGS. 4A-4C, illustrate an example of a geometric warping400that may occur when rendering the sphere300authored in the normalized room geometry100in a room with physical room geometry200. For instance, when mapping the x, y, z coordinates from the normalized room geometry100to the physical room geometry200, geometry warping may occur in order to guarantee that a location of each static audio object in the normalized room geometry100maps perfectly onto the corresponding static audio object in the physical room geometry200. Thus, rather than producing the intended sphere300about the listener9, rendering of this example of the spatial audio content in the physical room geometry200will result in the geometric warping400.

The systems and methods of the present disclosure compensate for geometric and velocity warping (or, space and/or time warping) that may occur due to differences between an authoring model and a rendering model, by encoding an inverse of the warping into the normalized room geometry used for authoring the spatial audio content. As such, the author would not author spatial audio content to the normalized room geometry, but instead authors spatial audio content to an expected physical room geometry. An expected physical room geometry may be obtained, for example, via documentation provided by a spatial audio encoder, or computed based on received measurements of warping of a known output. In addition, an expected physical room geometry may be based on receiving, e.g., from an end user via a user interface, information that accurately describes the geometric room (e.g., the speaker geometry) in which the spatial audio content will be rendered.

Referring now toFIGS. 5A-5C,FIG. 5Aincludes an example of a room with the expected physical room geometry500and an intended spatial audio content rendering510, e.g., a rendered sphere, representing a result of the present solution.FIGS. 5B and 5Cincludes the normalized room geometry100modified according to the present disclosure to include an inverse projection520(e.g., the 3D transformation, or the inverse of the geometric warping) of the intended spatial audio content rendering510, e.g., the sphere, into the normalized geometry100. This modification enables the expected physical room geometry500to properly and accurately render the intended spatial audio content rendering510, e.g., with a substantial reduction or elimination of time and/or space warping of the spatial audio content.

The methods and systems of the present disclosure may compute the 3D transformation that defines the geometric warping that may occur between the expected physical room geometry and the normalized geometry and may encode the 3D transformation (e.g., an inverse of the geometric warping, or inverse geometric transform) into the normalized room geometry. Thus, geometric models for spatial audio content can be generated to enable rendering the original authored intended geometry in a given physical room geometry.

Referring now toFIG. 6, an example computer device602for compensating for warping of 3D spatial audio may include an operating system610executed by processor40and/or memory42of computer device602. As used herein, 3D spatial audio may include an ability to place audio sources about a listener in three dimensions, e.g., using x, y, z coordinates relative to the listener.

Computer device602may include one or more applications10executed or processed by processor40and/or memory42of computer device602. Memory42of computer device602may be configured for storing data and/or computer-executable instructions defining and/or associated with operating system610, and processor40may execute operating system610. An example of memory42can include, but is not limited to, a type of memory usable by a computer, such as random access memory (RAM), read only memory (ROM), tapes, magnetic discs, optical discs, volatile memory, non-volatile memory, and any combination thereof. An example of processor40can include, but is not limited to, any processor specially programmed as described herein, including a controller, microcontroller, application specific integrated circuit (ASIC), field programmable gate array (FPGA), system on chip (SoC), or other programmable logic or state machine.

Computer device602may include any mobile or fixed computer device, which may be connectable to a network. Computer device602may be, for example, a computer device such as a desktop or laptop or tablet computer, a cellular telephone, a gaming device, a mixed reality or virtual reality device, a music device, a television, a navigation system, a camera, a personal digital assistant (PDA), or a handheld device, or any other computer device having wired and/or wireless connection capability with one or more other devices.

Application10may allow a user to develop spatial audio content. For example, a user may develop spatial audio content for a location, such as a room, that may be a physical or virtual location. Application10may allow a user to specify the room, for example, by inputting a room size, a number of static audio objects to place in the room, a number of dynamic audio objects to place in the room, and/or locations of the static and dynamic audio objects in the room.

An example user may include a developer writing a game. The developer may want an immersive audio experience and may use application10to access spatial audio API20to place sounds about a listener (e.g., a gamer playing the game) to improve an experience of the game. The developer may want the gamer to hear a gunshot off to the left on the horizon, or to hear footsteps of someone walking up behind them, or hear a voice of someone up on a balcony, and may use audio as the only cue (or in combination with a visual cue) to the gamer to look in the direction of the sound. In each scenario, the sound may have a 3-Dimensional position relative to the listener (e.g., a point of view of the gamer or a character playing the game). Sound objects, usually mono-audio samples of the gunshot, the footsteps, or the spoken words, may be stored the memory42, e.g., on a hard drive. The sound itself is generally a sound file and has no spatial information. Since the position and orientation of the listener changes dynamically during gameplay, only at the moment of playback is the position of the sound relative to the listener determined. As such, sound is played by spatial audio API20based on the specified x, y, z coordinates specified. If the listener (e.g., point of view of the gamer or character playing the game) is moving while sounds are playing, or if the sound is moving, the coordinates of the playback for the sound are updated dynamically in real time or near real time. As such, by using spatial audio API20, the developer does not need to think about how the audio is spatialized. The developer may do pre-processing of the sound to provide distance decay, if not provided by the encoder in question, or if the developer wants to provide other effects like Doppler effect on a moving sound.

Application10may also allow a user to experience spatial audio. For example, a user of application10may include a listener who experiences the immersive 3D audio during gameplay when listening on a device and/or environment that supports spatial 3D audio. The user may hear the gunshot on the ridge to the left, the footsteps walking behind them, the bad guy talking up on the balcony, and know exactly where that sound is coming from.

Other uses of application10may include virtual surround sound (VSS) using a head-related transfer function (HRTF) spatial encoder to place virtual speakers about a listener over headphones. The static objects may be virtualized into HRTF so the user may experience surround sound over headphones while watching a movie that was rendered in surround sound, or even full spatial audio of a movie with authored spatial audio or a game with spatial audio.

Application10may communicate with at least one spatial audio application programming interface (API)20. Spatial audio API20may allow for the playback of audio via, for example, one or more speakers12, as either a static audio object assigned to any of the available static audio objects and/or dynamic audio objects generated by two or more static audio objects based on corresponding x, y, z position data.

Spatial audio API20may expose any spatial geometry, such as normalized spatial geometry100and/or physical room spatial geometry200. Spatial geometries may include a specific layout of static audio objects relative to a listener as defined either by vertical and horizontal angles and distance to the listener, or by providing a room width, depth and height, along with x, y, z coordinates of the static audio objects within the volumetric space relative to the listener. Spatial geometries may be defined by requirements from an encoder. As such, a spatial geometry may vary from encoder to encoder. In addition, spatial audio API20may expose a common spatial geometry independent of any underlying spatial geometry requirements from encoders. Spatial audio API20may also expose a spatial geometry specified by a user (e.g., specifying the physical spatial geometry of the volumetric space which the audio will be consumed). When a user specifies a spatial geometry, spatial audio API20may also continue to expose a common spatial geometry and spatial audio API20may perform additional mapping between the common spatial geometry and the user defined spatial geometry prior to mapping to an encoder spatial geometry.

Spatial audio API20may accept room geometry input from one or more applications10and may process the room geometry input into a specific spatial audio encoder38.

Spatial audio API20may communicate with a spatial audio encoder38that defines an encoder spatial geometry26. The encoder spatial geometry26may include a specific layout of a plurality of static audio objects28relative to a listener. The encoder spatial geometry26may be defined by vertical and horizontal angles of the static audio objects28and a relative distance from the static audio objects28to a listener. The encoder spatial geometry26may also be defined by providing a room width, depth, and height, and corresponding x, y, z coordinates of the static audio objects28within the volumetric space relative to a listener.

Spatial audio API20may also determine an expected spatial geometry30of the room specified by application10. For example, spatial audio API20may receive the geometry of the room from spatial audio API20.

Spatial audio API20may have a mapping component22that maps the positions of the dynamic audio objects32of the expected spatial geometry30directly to positions of static audio object28in the encoder spatial geometry26when the x, y, z positions of the dynamic audio objects32align with the documented x, y, z positions of the static audio objects28. The mapping occurs regardless of any differences in spatial geometries between the encoder spatial geometry26and the expected spatial geometry30. Differences in static audio objects positions between geometries may cause stretching and compression (e.g., warping) of coordinate space between any various static audio objects, which may occur in three dimensional space as well. As such, a geometric transform35may be identified that defines a geometric warping between the expected spatial geometry30and the encoder spatial geometry26. In one implementation, geometric transform35may be predicted by defining a geometry which corresponds to the encoder spatial geometry26.

For example, if two static objects are closer together, the space between them may be compressed during the mapping, however, those two static audio objects may also be further away from the listener, so the space may be stretched as well. Another example may include the distance remains the same where no stretching or compression occurs. Other examples may include one of the static audio objects is closer or further apart to the other, but at the same time one static object is closer to the listener while the other is further away. For example, in a two dimensional case, there may be a listener and two static objects with a potential for any combination of compression, stretching, and/or same on each of the line segments between the static objects and/or the static objects and the listener. In the 3D space, there may be six line segments forming a “pyramid” of a mesh between three static objects and a listener, where each segment may stretch, compress, and/or stay the same. As such, a complex 3D warping of the space contained within the mesh zone may be created.

When the relative positions (e.g., angles and/or distances) of the static audio objects in one model do not match the angles and positions of the corresponding static audio objects in a second model, but the requirement of volumetric coordinates x, y, z and corresponding static audio objects remain matched, a geometric warping of the coordinates between the two models occurs.

Warping may cause both spatial and temporal distortions when perceived over time. Spatial warping may be evidenced by a dynamic audio object placed at a specific distance and a specific angle from a listener in the first model and the dynamic audio object is also placed at the same x, y, z coordinates in the second model, but with different distance and/or angle than was originally authored in the first model. Temporal distortions may be evidenced by an animated path of a dynamic audio object over time, which in the first model may be authored on a path that traverses a perfect circle at a constant velocity and constant radius form a listener, but due to geometry warping, while still being placed at the same x, y, z positions over time, no longer has a constant velocity or a constant radius from the listener.

In addition, spatial audio API20may also include a calculator component24that calculates an inverse geometric transform34that compensates for a geometric warping that may occur during the mapping of the dynamic audio objects32in the expected spatial geometry30to static audio objects28in the encoder spatial geometry26.

Spatial audio API20may apply the inverse geometric transform34to the dynamic audio objects32and generate an expected rendered spatial geometry36. The expected rendered spatial geometry36may include new positions for the dynamic audio objects32in the expected rendered spatial geometry36so that the new positions are encoded based on the requirements of the encoder spatial geometry26. The expected rendered spatial geometry36may correct for geometric warping caused by differences between the encoder spatial geometry26and the expected spatial geometry30.

For example, a modeled sound may be created at static audio object33in an expected spatial geometry30. When the modeled sound is produced in the encoder spatial geometry26, the actual sound outputted by one or more speakers12may differ from the modeled sound due to geometric warping that may have occurred because of differences between the encoder spatial geometry26and the expected spatial geometry30. The expected rendered spatial geometry36may be used by spatial audio API20when outputting the modeled sound, for example, through one or more speakers12so that the actual sound outputted by the speakers12corrects for the geometric warping between the encoder spatial geometry26and the expected spatial geometry30.

Spatial audio API20may expose the expected rendered spatial geometry36to application10. Thus, application10may receive an expected rendered spatial geometry36that maintains an intent of the user while compensating for geometric warping based on the encoder spatial geometry26of the spatial audio encoder38. In an implementation, spatial audio rendering component16may render the spatial audio during playback of the rendered spatial geometry36. Spatial audio rendering component16may be on computer device602or another external device.

Referring now toFIG. 7, a method700of correcting warping in spatial audio may be executed by an operating system610(FIG. 1) on computer device602(FIG. 1).

At702, method700may include identifying a first spatial geometric model. The first spatial geometric model may represent how sound may be produced in a first volumetric space. In addition, the first spatial geometric model may include a first set of static audio objects positioned relative to a fixed point. For example, the front-left audio object might be at position −10, −15, 0, within a range of −15 to 15 for each x, y, z coordinate. In an implementation, the first spatial geometric model may be the expected spatial geometry30(FIG. 1) generated based on the received room geometry input from application10(FIG. 1). As such, a user of application10may define the range of the coordinate positions for the static audio objects and/or may define a geometry for the first spatial geometric model.

At704, method700may include identifying a second spatial geometric model. The second spatial geometric model may represent how sound may be produced in a second volumetric space that may be different from the first volumetric space. The second spatial geometric model may also include a second set of static audio objects positions relative to a fixed point. Each of the static audio objects within the second geometric model may be identified with coordinate positions within a second range of positions. For example, the fixed point may be a listener represented at location (0, 0, 0) and the second geometric model may identify a range of valid positions for the second set of static audio objects relative to the listener. The range of valid positions for the second set of static audio objects may be different from the range of valid positions for the static audio objects in the first geometric model. As such, the geometric relationship of the second geometric model may be different from the geometric relationship of the first geometric model. For example, the fixed point may be a listener represented at location (0,0,0) and the first geometric model may identify a range of valid positions for the static audio objects relative to the listener. An example range may include −1 to 1 for each x, y, z position for the static audio object, where one unit equals one meter. In an implementation, the second spatial geometric model may be the encoder spatial geometry26(FIG. 1) specified by spatial audio encoder38(FIG. 1).

At706, method700may include identify a geometric transform that defines a geometric warping between the first geometric model and the second geometric model. Spatial audio API20(FIG. 1) may identify geometric transform35(FIG. 1). For example, a front-left speaker in the first geometric model may be positioned at −1, −1, 0, with a range of −1 to 1 for each x, y, z coordinate and a front-left speaker in the second geometric model may be at position −10, −15, 0, with a range of −15 to 15 for each x, y, z coordinate. In the first geometric model, the front-left speaker may be positioned at 45 degrees to the left of the listener and in the second geometric model, the front-left speaker may be positioned at 25 degrees. A warping may occur because the label speaker positions are aligned such that a point rendered at a same position of each speaker in the first geometric model will translate to the same position of that speaker in the second geometric model. As such, the point −1, 1, 0 in the first geometric model translates to point −10, −15, 0, in the example described, and the relative difference in position with respect to the listener defines one dimension of the warping, which is determined for all corresponding points between the models.

At708, method700may include determining an inverse of the geometric transform that compensates for the geometric transform. For example, spatial audio API20may calculate an inverse of geometric transform34that compensates for a geometric warping that may occur between the first spatial geometric model and the second spatial geometric model. In an implementation, geometric transform35may be a prediction of the second spatial geometric model. The prediction may be based on, for example, user input or requirements from an encoder. Spatial audio API20may place points into the prediction of the second spatial geometric model and transform the placed points into the first spatial geometric model to calculate the inverse geometric transform34.

At710, method700may include defining a first location in the first spatial geometric model. The first location may relate to a dynamic audio object32(FIG. 1) in the expected spatial geometry30. Spatial audio API20may identify the dynamic audio object32, for example, by room geometry input received by application10. In an implementation, a user of application10may place the dynamic audio object32in the expected spatial geometry30.

At712, method700may include applying the inverse of the geometric transform to the first location in the first spatial geometric model by mapping the first location to a second location in the second geometric model to correct for the geometric warping. For example, spatial audio API20may map the position of the dynamic audio objects32of the first spatial geometric model directly to positions of static audio objects28in the second spatial geometric model when the dynamic audio objects32x, y, z positions align with the documented x, y, z positions of the static audio objects28.

Referring now toFIGS. 8, 9A, and 9B, a method800(FIG. 8) for mapping a dynamic audio object between a first mesh (FIG. 9A) and a second mesh (FIG. 9B) associated with different spatial geometries may be executed by an operating system610(FIG. 1) on computer device602(FIG. 1). The method800is one example of a specific implementation of the present disclosure to compensate for warping between different spatial geometries in spatial audio.

At802, method800may include computing a first mesh of a first spatial geometric model. For example, spatial audio API20(FIG. 1) may compute a first mesh in the form of a surrounding mesh of the expected spatial geometry30(FIG. 1) for the static audio objects33in the expected spatial geometry30.

In an implementation, prior to computing the surrounding mesh, spatial audio API20may determine a count of the static audio objects28in the encoder spatial geometry26and a count of the static audio objects33in the expected spatial geometry30and take the greater of the two. For any missing static audio objects in the encoder spatial geometry26and/or the expected spatial geometry30, spatial audio API20may generate positions which are symmetric to existing static audio objects in the corresponding geometry. For example, if one geometry has a center back static audio object, but the other geometry does not have a center back static audio object, spatial audio API20may generate a static audio object for the center back directly midpoint between back right and back left static objects for the geometry that does not have a center back static audio object. Another example may include if one geometry has lower speakers and the other geometry does not have lower speakers, spatial audio API20may generate a mirror of the top speakers by reflecting the positions of the top speakers in the inverse plane. When spatial audio API20is generating any additional static audio objects, symmetric angels and distances may be maintained.

Spatial audio API20may generate a mesh for the expected spatial geometry30using all the static audio objects33in a listener plane to form a list of line segments. One implementation may include starting at any static object in listener plane, connect to next object. An example set of surround speakers may include, FrontLeft→FrontCenter→FrontRight→SideRight→BackRight→BackCenter→BackLeft→SideLeft→FrontLeft.

For each segment defined, spatial audio API20may connect to a corresponding upper speaker nearest the segment. For example, segment FrontLeft→FrontCenter may connect to TopFrontLeft, which defines a single face of the mesh. If lower speakers are present, each segment may connect again to lower speakers, so Front→Left→FrontCenter→BottomFrontLeft. Top speakers may be connected to each other, or to a central generated center point to maintain polar symmetry, and down to surrounding speakers. By connecting the top and bottom speakers as described above, a solid mesh made up of just triangles may be created which completely surrounds the listener. An example mesh910of the expected spatial geometry30is illustrated inFIG. 9A.

At804, method800may include computing a second mesh of a second spatial geometric model. Spatial audio API20may compute a second mesh in the form of a surrounding mesh of the encoder spatial geometry26(FIG. 1) for the static audio objects28in the encoder spatial geometry26.

In one implementation, spatial audio API20may use all the static audio objects28in a listener plane to form a list of line segments. For example, spatial audio API20may start at any static audio object28in listener plane, connect to next object. An example set of surround speakers may include, FrontLeft→FrontCenter→FrontRight→SideRight→BackRight→BackCenter→BackLeft→SideLeft→FrontLeft.

For each segment defined, spatial audio API20may connect to a corresponding upper speaker nearest the segment. For example, segment FrontLeft→FrontCenter may connect to TopFrontLeft, which defines a single face of the mesh. If lower speakers are present, each segment may connect again to lower speakers, so Front→Left→FrontCenter→BottomFrontLeft. Top speakers may be connected to each other, or to a central generated center point to maintain polar symmetry and down to surrounding speakers. By connecting the top and bottom speakers as described above, a solid mesh made up of just triangles may be created which completely surrounds the listener. An example mesh920of the encoder spatial geometry is illustrated inFIG. 9B.

As such, the first mesh and the second mesh each have a same number of faces, which directly correspond to a same ordered mesh face on the other mesh. Thus, if face1on mesh1is defined by FrontLeft, FrontCenter and TopFrontLeft, then face1on mesh2is defined the same by FrontLeft, FrontCenter and TopFrontLeft.

At806, method800may include mapping a dynamic audio object relative to the first mesh to a new translated point in the second mesh. For example, spatial audio API20may map a dynamic audio object32from the expected spatial geometry30to a new translated point in the encoder spatial geometry26.

One implementation for mapping a dynamic audio object32(D1) from the first mesh to a new translated point (D2) in the second mesh may include spatial audio API20defining a first line (L1) in the first mesh from origin (0, 0, 0) to D1and computing an intersection face (face F1) of the first mesh by using the first line L1and an intersection point P1. Spatial audio API20may also identify a second face (F2) in the second mesh that directly corresponds to face F1in the first mesh. Spatial audio API20may map Point P1in face F1into a new mapped intersection point P2in face F2while maintaining relative position face F1and face F2. Spatial audio API20may compute scaleD1=(magnitude of D1)/(magnitude of P1). Spatial audio API20may also compute D2=P2+(magnitude of P2)*scaleD1. Magnitude may include the length of the vector from an origin location (0, 0, 0) to the point (x, y, z).

One implementation for the mapping point P1to P2may include spatial audio API20defining a triangle for first face in the first mesh as A1, B1, C1, where point P1is the intersection point contained within the triangle for the first face, as illustrated inFIG. 10A. Intersection point P1may be mapped to an intersection point P2and scaled to get point D2, as illustrated inFIG. 11B. Spatial audio API20may also define a triangle for second face in the second mesh as A2, B2, C2, as illustrated inFIG. 10B. Spatial audio API20may also define line A as point A1to point P1and line B as Point B1to point C1and may find the intersection of line A and line B at point M1, as illustrated inFIG. 11A. Line M1will line on the line segment B1→C1. Spatial audio API20may compute scaleB1as (magnitude(B1→M1)/(magnitude(B1−C1). Spatial audio API20may compute scaleA1as (magnitude(A1→P1)/(magnitude(A1→M1). Spatial audio API20may compute M2=B2+(magnitude(B2→C2)*scaleB1). Spatial audio API20may compute P2=A2+(magnitude(A2→M2)*scaleA1). Spatial audio API20may also compute D2=P2+(magnitude of P2)*scaleD1.

Referring now toFIG. 12, illustrated is an example computer device602in accordance with an implementation, including additional component details as compared toFIG. 1. In one example, computer device602may include processor40for carrying out processing functions associated with one or more of components and functions described herein. Processor40can include a single or multiple set of processors or multi-core processors. Moreover, processor40can be implemented as an integrated processing system and/or a distributed processing system.

Computer device602may further include memory42, such as for storing local versions of applications being executed by processor40. Memory42can include a type of memory usable by a computer, such as random access memory (RAM), read only memory (ROM), tapes, magnetic discs, optical discs, volatile memory, non-volatile memory, and any combination thereof. Additionally, processor40and memory42may include and execute operating system610(FIG. 1).

Further, computer device602may include a communications component46that provides for establishing and maintaining communications with one or more parties utilizing hardware, software, and services as described herein. Communications component46may carry communications between components on computer device602, as well as between computer device602and external devices, such as devices located across a communications network and/or devices serially or locally connected to computer device602. For example, communications component46may include one or more buses, and may further include transmit chain components and receive chain components associated with a transmitter and receiver, respectively, operable for interfacing with external devices.

Additionally, computer device602may include a data store48, which can be any suitable combination of hardware and/or software, that provides for mass storage of information, databases, and programs employed in connection with implementations described herein. For example, data store48may be a data repository for applications10(FIG. 1), spatial audio API20(FIG. 1), spatial audio encoder38(FIG. 1) and/or spatial audio rendering component16(FIG. 1).

In an implementation, user interface component50may transmit and/or receive messages corresponding to the operation of applications10, spatial audio API20, spatial audio encoder38and/or spatial audio rendering component16. In addition, processor40executes applications10, spatial audio API20, spatial audio encoder38, and/or spatial audio rendering component16, and memory42or data store48may store them.

Furthermore, various implementations are described herein in connection with a device (e.g., computer device602), which can be a wired device or a wireless device. A wireless device may be a cellular telephone, a satellite phone, a cordless telephone, a Session Initiation Protocol (SIP) phone, a wireless local loop (WLL) station, a personal digital assistant (PDA), a handheld device having wireless connection capability, a computer device, a mixed reality or virtual reality device, or other processing devices connected to a wireless modem.

The various illustrative logics, logical blocks, and actions of methods described in connection with the embodiments disclosed herein may be implemented or performed with a specially-programmed one of a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general-purpose processor may be a microprocessor, but, in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computer devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. Additionally, at least one processor may comprise one or more components operable to perform one or more of the steps and/or actions described above.

While implementations of the present disclosure have been described in connection with examples thereof, it will be understood by those skilled in the art that variations and modifications of the implementations described above may be made without departing from the scope hereof. Other implementations will be apparent to those skilled in the art from a consideration of the specification or from a practice in accordance with examples disclosed herein.