Phase aligned foveated rendering

A display device, such as a head mounted device (HMD), displays a virtual scene. The display device includes a motion tracker for detecting rotation of the display device. The display device also includes a processor that is configured to selectively maintain or modify a position of an array of rendered pixels relative to the virtual scene in response to the detected motion. The processor is also configured to upsample the rendered pixels to generate values of display pixels for presentation by the display device. The processor is further configured to translate the values of the display pixels in a rendering plane of the display device based on the detected motion. The translated values of the display pixels can then be presented on a display of the display device.

BACKGROUND

Foveated rendering exploits the falloff in acuity of the human eye at the visual periphery to conserve power and computing resources used to generate digital images for display to users, e.g., to display augmented reality (AR) or virtual reality (VR) using a head mounted device (HMD). In foveated rendering, a user's central gaze direction is determined, either as the center of a current field-of-view or using eye tracking to determine where the user is looking. The user's field-of-view is then subdivided into a high-acuity region that surrounds the central gaze direction and one or more lower-acuity regions in the visual periphery. The high-acuity region includes a portion of the field-of-view that is within a first angular distance of the central gaze direction. The angular distance from the central gaze direction is referred to as the eccentricity. The lower-acuity regions include portions of the field-of-view that are at larger eccentricities. For example, the high-acuity region can include a portion of the field-of-view that is within an eccentricity of 5-10°, which corresponds to a portion of the field-of-view that projects to a retinal region in the human eye called the fovea. Pixels are rendered at high resolution within the high-acuity region, e.g., by rendering the pixels at a resolution corresponding to the native resolution supported by the display. Pixels in the low-acuity regions at eccentricities larger than 5-10° are rendered at lower resolutions, thereby reducing the power and computing resources needed to render the pixels. The rendered pixels in the low-acuity regions can subsequently be upsampled to generate display pixels at the native resolution of the display, e.g., using well-known interpolation techniques such as bilinear interpolation.

DETAILED DESCRIPTION

Rendering pixels at low resolution in low-acuity regions of a user's field-of-view and subsequently upsampling the rendered pixels to generate higher resolution display pixels can generate visual artifacts such as aliasing artifacts. A change in the user's field-of-view, e.g., due to movement of the user's head while wearing an HMD, changes the mapping of the rendered pixels to the virtual scene that the user is observing because the pattern of the rendered pixels is fixed to the display. Thus, the value of each rendered pixel changes as it sweeps across the virtual scene. The time-dependent value of the rendered pixels introduces a time dependence in values of the upsampled display pixels, which can generate a corresponding time dependence in the aliasing artifacts. Static aliasing artifacts in the low-acuity regions are not noticeable because of the falloff in acuity with eccentricity. However, the human eye remains very sensitive to motion in the visual periphery. Consequently, time-dependent aliasing artifacts in the low-acuity regions are very noticeable and can disrupt the user's sense of immersion in the virtual scene.

The time dependence of aliasing artifacts produced by motion of a display device (such as an HMD) can be reduced by maintaining a position of an array of rendered pixels relative to a virtual scene during motion of the display device, upsampling the rendered pixels to generate values of corresponding display pixels, and then translating coordinates of the display pixels based on the motion of the display device. Initially, the array of rendered pixels are positioned at a fixed, default orientation relative to the virtual scene and the display device. The relative position of the array and the virtual scene can then be selectively maintained or modified in response to motion of the display device by either maintaining or modifying the position of the array relative to the display device. For example, a relative rotation of the array and the virtual scene are maintained by shifting the array relative to the display device to compensate for motion of the display device as long as the display device has moved an angular distance that is less than an angular resolution of a single pixel. If the angular motion of the display device relative to the virtual scene is greater than one pixel's angular resolution, then the array's shifted position for each frame relative to the display device only uses the fractional residual angular resolution, ignoring rotations that are integer multiples of rendered pixel angular resolution. This process is referred to as “snapping” the array to the nearest rendered pixel. The translation applied to the display pixels is set to zero in response to snapping the array to the nearest rendered pixel. After snapping the array, the translation is determined based on the magnitude and direction of motion of the display device relative to the new position of the array relative to the virtual scene. The translation is applied to the display pixels along a two-dimensional (2-D) rendering plane. In some embodiments, the array of rendered pixels includes a set of buffered pixels that have values determined by portions of the virtual scene that border, but are outside of, a current field-of-view of the display device. The values of the buffered pixels can be upsampled to generate values of display pixels that are translated into the current field-of-view of display device based on the motion of the display system. In this case, the size of the border is one rendered pixel on each side to ensure total coverage of the display by rendered content.

FIG. 1is a diagram of an image acquisition and display system100that supports immersive virtual reality (VR) or augmented reality (AR) functionality according to some embodiments. Immersive VR and AR systems typically utilize an electronic device105that presents stereoscopic imagery to a user so as to give a sense of presence in a three-dimensional (3D) scene. For example, the electronic device105is designed to produce a stereoscopic image over a field-of-view that approaches or is equal to the field-of-view of a human eye, which is approximately 180°. The illustrated embodiment of the electronic device105can include a portable user device, such as head mounted display (HMD), a tablet computer, computing-enabled cellular phone (e.g., a “smartphone”), a notebook computer, a personal digital assistant (PDA), a gaming console system, and the like. In other embodiments, the electronic device105can include a fixture device, such as medical imaging equipment, a security imaging sensor system, an industrial robot control system, a drone control system, and the like. For ease of illustration, the electronic device105is generally described herein in the example context of an HMD system; however, the electronic device105is not limited to these example implementations.

The image acquisition and display system100includes an image acquisition device110that is used to acquire two-dimensional (2-D) images of a scene for presentation to a user via the electronic device105. The image acquisition device110can include either or both of a physical image acquisition device, such as a camera, that acquires images of an actual scene, or a virtual image acquisition device110that generates images of a virtual scene such as a scene represented by a three-dimensional (3-D) model. For example, if the electronic device105is supporting a VR application, the image acquisition device110can be a virtual image acquisition device110that generates images of the virtual scene for presentation to the user. For another example, if the electronic device105is supporting an AR application, the image acquisition device110can include both a physical image acquisition device to acquire an image of an actual scene as viewed from the perspective of the user and a virtual image acquisition device to generate a virtual image of a virtual scene. The image acquisition device110can then combine the virtual image and the actual image to create a composite AR image for presentation to the user.

The image acquisition and display system100includes one or more memories115that are used to store digital information representative of images acquired by the image acquisition device110. The memory115can be implemented as dynamic random access memory (DRAM), nonvolatile random access memory (NVRAM), flash memory, and the like. Some embodiments of the memory115also implement one or more caches for storing recently accessed information. The image acquisition and display system100also includes one or more processing units120that are configured to access information from the memory115and execute instructions such as instructions stored in the memory115. The one or more processing units120can also store the results of the executed instructions in the memory115. The one or more processing units120can include a central processing unit (CPU), a graphics processing unit (GPU), and the like. As discussed herein, the electronic device105can also include one or more processing units and memories. The operations described herein can therefore be executed by the electronic device105, by the processing units120, or the workload can be shared between the electronic device105and the processing units120.

FIG. 2illustrates a display system200that includes an electronic device205configured to provide immersive VR or AR functionality according to some embodiments. The electronic device205is used to implement some embodiments of the electronic device105shown inFIG. 1. A back plan view of an example implementation of the electronic device205in an HMD form factor in accordance with at least one embodiment of the present disclosure is shown inFIG. 2. The electronic device205can be implemented in other form factors, such as a smart phone form factor, tablet form factor, a medical imaging device form factor, a standalone computer, a system-on-a-chip (SOC), and the like, which implement configurations analogous to those illustrated. As illustrated by the back plan view, the electronic device105can include a face gasket210mounted on a surface215for securing the electronic device205to the face of the user (along with the use of straps or a harness).

The electronic device205includes a display220that is used to generate images such as VR images or AR images that are provided to the user. The display220is divided into two substantially identical portions, a right portion to provide images to the right eye of the user and a left portion to provide images to the left eye of the user. In other embodiments, the display220is implemented as two different displays, one dedicated to each eye. The electronic device205implements foveated rendering to present images to the user. The display220is therefore subdivided into different regions based on a distance from the user's center of gaze, e.g., the eccentricity. For example, the field-of-view for the user's left eye can be subdivided into a high-acuity region225that surrounds a central gaze direction230. The field-of-view for the user's left eye is further subdivided into lower-acuity regions235,240in the visual periphery. Similarly, the field-of-view for the user's right eye can be subdivided into a high acuity region245that surrounds a central gaze direction250and lower acuity regions255,260in the visual periphery. The central gaze directions230,250can be set equal to the center of a current field-of-view or they can be determined on the basis of eye tracking measurements that detect the central gaze direction of the user's eyes. In some embodiments, more or fewer lower acuity regions can be defined for the display220.

Pixels are rendered at high resolution within the high-acuity regions225,245, e.g., by rendering the pixels at a resolution that is equal to the native resolution supported by the display. Pixels in the low-acuity regions235,240,255,260are rendered at lower resolutions, thereby reducing the power and computing resources needed to render the pixels. The rendered pixels in the low-acuity regions235,240,255,260are subsequently upsampled to generate display pixels at the native resolution of the display, e.g., using well-known interpolation techniques such as bilinear interpolation.

FIG. 3illustrates a display system300that includes an electronic device305configured to provide AR or VR functionality to a user wearing the electronic device305via a display according to some embodiments. The electronic device305is used to implement some embodiments of the electronic device105shown inFIG. 1and the electronic device205shown inFIG. 2. The electronic device305is shown inFIG. 3as being mounted on a head310of a user. As illustrated, the electronic device305includes a housing315that includes a display320that generates an image for presentation to the user. The display320is implemented using some embodiments of the display220shown inFIG. 2. In the illustrated embodiment, the display320is formed of a left display321and a right display322that are used to display stereoscopic images to corresponding left eye and right eye. However, in other embodiments, the display320is a single monolithic display320that generates separate stereoscopic images for display to the left and right eyes. The electronic device305also includes eyepiece lenses325and330disposed in corresponding apertures or other openings in a user-facing surface332of the housing315. The display320is disposed distal to the eyepiece lenses325and330within the housing315. The eyepiece lens325is aligned with the left eye display321and the eyepiece lens330is aligned with the right eye display322.

In a stereoscopic display mode, imagery is displayed by the left eye display321and viewed by the user's left eye via the eyepiece lens325. Imagery is concurrently displayed by the right eye display322and viewed by the user's right eye via the eyepiece lens325. The imagery viewed by the left and right eyes is configured to create a stereoscopic view for the user. Some embodiments of the displays320,321,322are fabricated to include a bezel (not shown inFIG. 3) that encompasses outer edges of the displays320,321,322. In that case, the lenses325,330or other optical devices are used to combine the images produced by the displays320,321,322so that bezels around the displays320,321,322are not seen by the user. Instead, lenses325,330merge the images to appear continuous across boundaries between the displays320,321,322.

In some embodiments, some or all of the electronic components that control and support the operation of the display320and other components of the electronic device305are implemented within the housing315. For example, the electronic device305can include a processing unit such as a GPU335and a memory340. In some embodiments the workload associated with acquiring actual or virtual images and rendering these images for display on the display320can be shared with external processing units such as the processing unit120shown inFIG. 1. Some embodiments of the electronic device305include an eye tracker345to track movement of the user's eyes and determine a center of gaze for each eye in real-time. The electronic device305also includes one or more motion sensors350. Examples of motion sensors350include accelerometers, gyroscopic orientation detectors, or other devices capable of detecting motion of the electronic device305.

In the illustrated embodiment, the GPU335is configured to render pixels at different resolutions depending on an eccentricity from a center of gaze for the user. For example, the displays321,322can be subdivided into high acuity regions and low acuity regions. The GPU335renders pixels in the high acuity regions at a higher resolution (e.g., at the native resolution of the display321,322) and renders pixels in the low acuity regions at lower resolutions. The GPU335then upsamples the rendered pixels to generate values of display pixels at the native resolution for presentation to the user by the displays321,322. As discussed herein, a change in the user's field-of-view, e.g., due to movement of the user's head310while wearing the electronic device305, changes the mapping of rendered pixels to the scene that the user is observing if the pattern of the rendered pixels is fixed relative to the display320in the electronic device305. Thus, the value of each rendered pixel changes as it sweeps across the virtual scene, which can generate a corresponding time dependence in the aliasing artifacts. Time-dependent aliasing artifacts in the low-acuity regions are very noticeable and can disrupt the user's sense of immersion in the virtual scene.

The noticeability of time-dependent aliasing artifacts in the low acuity regions is reduced by selectively maintaining or modifying a position of an array of rendered pixels relative to a virtual scene in response to detecting motion of the electronic device305. The array of rendered pixels is used to define lower resolution pixels in one or more of the low acuity regions. The GPU335maintains the position of the array of rendered pixels relative to the virtual scene as long as the electronic device305has moved an angular distance that is less than an angular resolution of a rendered pixel. Maintaining the relative orientation of the array of rendered pixels and the virtual scene reduces time variability of the values of the rendered pixels, which can reduce the number or visibility of time-dependent aliasing artifacts in the low acuity regions. However, maintaining the relative orientation of the array and the virtual scene changes the relative orientation of the array and the display320. Thus, after rendering the lower resolution pixels in the low acuity regions and upsampling the rendered pixels to generate values of the display pixels, the GPU335translates the values of the display pixels in a rendering plane of the display320based on the detected motion to maintain the correct relative orientation of the display pixels and the display320.

The GPU335modifies the position of the array of rendered pixels relative to the virtual scene in response to the electronic device305moving an angular distance that is greater than or equal to the angular resolution of a rendered pixel. For example, the GPU335can set the position of the array of rendered pixels to correspond to a first orientation relative to the virtual scene and a second orientation relative to the display320. Prior to any subsequent motion of the electronic device305, the first and second orientations are aligned with each other. If the orientation of the array of rendered pixels is modified to maintain the first (fixed) orientation of the array relative to the virtual scene as the electronic device305moves through an angular distance, the first orientation becomes displaced from the second orientation by the angular distance. The GPU335can shift the position of the array of rendered pixels by integer increments of the pixel angular resolution, realigning the array with the second orientation once the angular distance becomes greater than or equal to the angular resolution, thereby establishing a new value of the first orientation that is again fixed relative to the virtual scene for subsequent motion. This process is referred to as “snapping” the position of the array of rendered pixels to the nearest rendered pixel.

FIG. 4is a block diagram illustrating a virtual scene400that is displayed to a user via a head mounted device (HMD)405according to some embodiments. The virtual scene400corresponds to virtual scenes displayed by the electronic device105shown inFIG. 1, the electronic device205shown inFIG. 2, and the electronic device305shown inFIG. 3. In the illustrated embodiment, the virtual scene400includes a ball410, a small box415, and a large box420. Initially, the HMD405is in a first orientation relative to the virtual scene400, as indicated by the arrow425. The HMD405in the first orientation renders an image430of the virtual scene400for presentation to the user. The HMD405subsequently moves, e.g., due to motion of the head of the user wearing the HMD, to a second orientation relative to the virtual scene400, as indicated by the arrow435. The HMD405in the second orientation renders an image440of the virtual scene400for presentation to the user. The positions of the ball410, the small box415, and the large box420are shifted toward the left in the image440, relative to their positions in the image430.

FIG. 5illustrates rendered pixels and display pixels that are generated by an HMD that maintains a fixed orientation of an array500of rendered pixels relative to a display of the HMD. The array500is indicated by the dashed lines. The array500defines the rendered pixels for a first image505of a virtual scene (which corresponds to the image430shown inFIG. 4) and the rendered pixels for a second image510of the virtual scene (which corresponds to the image440shown inFIG. 4). Since the orientation of the array500is fixed relative to the display, values of the rendered pixels defined by the array change in response to motion of the HMD. For example, a value of the rendered pixel515(which corresponds to the rendered pixel that is second from the top and second from the left in the array500) differs from a value of the rendered pixel520, which corresponds to the rendered pixel at the same location in the array500as the rendered pixel515.

Upsampling of the values of the rendered pixels515,520creates different patterns of aliasing artifacts. For example, upsampling the value of the rendered pixel515to generate values of a set525of display pixels generates a first pattern of values that includes aliasing artifacts caused by attempting to depict a smooth curve using discrete values of display pixels. Upsampling the value of the rendered pixel520to generate values of a set530of display pixels generates a second pattern of values that includes different aliasing artifacts. Thus, fixing the orientation of the array500relative to the display introduces time-dependent aliasing artifacts in the values of the upsampled display pixels in the sets525,530.

FIG. 6illustrates rendered pixels and display pixels that are generated by an HMD that maintains a fixed orientation of an array600of rendered pixels relative to a virtual scene according to some embodiments. The array600is indicated by the dashed lines. The array600defines the rendered pixels for a first image605of a virtual scene, which corresponds to the image430shown inFIG. 4. In the illustrated embodiment, the array600also includes additional pixels that border, but are outside of, the first image605. In the interest of clarity, only the additional pixels that border, but are outside of, the right-hand vertical border and the lower border of the first image605are shown inFIG. 6. However, in some embodiments, the array600includes additional pixels that border, but are outside of, all the boundaries of the first image605.

The second image610represents the virtual scene subsequent to motion of the HMD. For example, the second image610correspond to the image440shown inFIG. 4. The array615maintains a fixed orientation relative to the virtual scene and, consequently, the orientation of the array615shifts relative to the orientation of the array600, the display, and the boundaries of the second image610. Since the orientation of the array615is fixed relative to the virtual scene, changes in the values of the rendered pixels in the array610are reduced or eliminated relative to the values of the corresponding rendered pixels in the array600. For example, a value of the rendered pixel620(which corresponds to the rendered pixel that is second from the top and second from the left in the array600) is substantially the same as a value of the rendered pixel625, which corresponds to the rendered pixel at the same location in the array615as the rendered pixel620in the array600.

The rendered pixels620,625can then be upsampled to generate corresponding sets630,635of display pixels. Fixing the orientation of the arrays600,615relative to the virtual scene causes the values of the rendered pixels620,625to remain substantially the same. Consequently, upsampling the rendered pixels620,625to generate values of the sets630,635of display pixels creates substantially the same pattern of aliasing artifacts, which reduces or eliminates the time dependence of aliasing artifacts associated with rendered pixels in the arrays600,615. The sets630,635of display pixels can then be translated (or otherwise transformed) from a coordinate system that is fixed relative to the virtual scene to a coordinate system that is fixed relative to the display. The amount of translation is determined by an offset (e.g., an angular distance) between the shifted array615and the initial position of the array600that is produced by motion of the HMD.

FIG. 7illustrates translation of display pixels from a coordinate system700that is fixed relative to the virtual scene to a coordinate system705that is fixed relative to a display of an HMD according to some embodiments. The vertical axes of the coordinate systems700,705represent inclination and the horizontal axes of the coordinate systems700,705represent azimuth. However, other embodiments of the coordinate systems700,705may use different definitions of the horizontal and vertical axes.

A first image710is rendered at a first time prior to motion of the HMD relative to an initial or default orientation that determines a first orientation of an array of rendered pixels relative to the virtual scene and a second orientation of the array of rendered pixels relative to the display of the HMD. Since the HMD has not yet moved relative to the initial or default orientation, the first and second orientations are the same. Consequently, the coordinate systems700,705overlap in the first image710. A first set715of display pixels are generated by upsampling a corresponding rendered pixel that represents a portion of the first image710. Since the coordinate systems700,705overlap (and the first and second orientations are the same), there is no need to translate or transform the display pixels from the coordinate system700to the coordinate system705.

A second image720is rendered at a second time subsequent to motion of the HMD by an angular distance relative to the initial or default orientation. The coordinate system700that is fixed relative to the virtual scene is therefore shifted by an amount determined by the angular distance relative to the coordinate system705, which is fixed relative to the display in the HMD. A second set725of display pixels are generated by upsampling a corresponding rendered pixel that represents a portion of the second image720. The pixel is rendered in the coordinate system700and so the second set725of display pixels must be translated or transformed from the coordinate system700to the coordinate system705that is fixed relative to the display of the HMD. For example, the position of the second set725of display pixels in the coordinate system700is indicated by the dashed box730. The second set725is translated by an offset735that is equal to the angular distance moved by the HMD. The translated second set725is then at the correct location in the coordinate system705of the display of the HMD.

FIG. 8illustrates a sequence800of images of a virtual scene that are generated using an array of rendered pixels having an orientation relative to the virtual scene that is selectively maintained or modified according to some embodiments. The sequence800can be acquired or displayed by an electronic device such as the electronic device105shown inFIG. 1, the electronic device205shown inFIG. 2, the electronic device305shown inFIG. 3, and the HMD405shown inFIG. 4. The sequence800includes images801,802,803,804that are collectively referred to herein as “the images801-804.” Each of the images801-804is generated at a different orientation of the electronic device relative to the virtual scene. Thus, from the perspective of the user of the electronic device, the objects in the images801-804appear to shift to the left in response to motion of the electronic device towards the right.

The image801is generated based on an array805of rendered pixels that are positioned at an initial or default orientation relative to the display of the electronic device. Thus, a first orientation of the array805relative to the virtual scene is the same as a second orientation of the array805relative to the display. Values of the display pixels are then generated based on the rendered pixels, as discussed herein. The first and second orientations of the array805are the same, and so no translation is necessary between the coordinate systems that are fixed relative to the virtual scene and the display.

The image802is generated after the electronic device has moved by an angular distance that is less than an angular resolution of the rendered pixels. The orientation of the array805is therefore shifted to maintain the first (fixed) orientation of the array805relative to the virtual scene. The first orientation of the array805is offset relative to the second orientation that is fixed relative to the display by an amount that is equal to the angular distance. Values of the display pixels are generated based on the rendered pixels and the values of the display pixels are translated from the coordinate system that is fixed relative to the virtual scene to the coordinate system that is fixed relative to the display, as discussed herein.

The image803is generated after the electronic device has moved by a larger angular distance, which is still less than the angular resolution of the rendered pixels. The orientation of the array805is therefore shifted to maintain the first (fixed) orientation of the array805relative to the virtual scene. The first orientation of the array805is offset relative to the second orientation that is fixed relative to the display by an amount that is equal to the larger angular distance. Values of the display pixels are generated based on the rendered pixels and the values of the display pixels are translated from the coordinate system that is fixed relative to the virtual scene to the coordinate system that is fixed relative to the display, as discussed herein.

The image804is generated after the electronic device has moved by an even larger angular distance that is greater than or equal to the angular resolution of the rendered pixels. The orientation of the array805is therefore modified, or snapped, back into alignment with the initial or default orientation. Snapping the orientation of the array805into alignment with the initial default orientation includes shifting the array805by an amount810that is equal to the angular distance between the first orientation of the array805and the initial or default orientation of the array805. The new orientation of the array805is therefore the same as the initial or default orientation of the array805relative to the display.

FIG. 9is a flow diagram of a method900for selectively maintaining or modifying an orientation of an array of rendering pixels relative to a virtual scene according to some embodiments. The method900is implemented in some embodiments of an electronic device such as the electronic device105shown inFIG. 1, the electronic device205shown inFIG. 2, the electronic device305shown inFIG. 3, and the HMD405shown inFIG. 4.

At block905, a processing unit implemented in the electronic device sets a position of an array of rendering pixels to a default value relative to the virtual scene. As discussed herein, setting the position of the array of rendering pixels to the default value relative to the virtual scene also corresponds to setting the position of the array of rendering pixels to a default value relative to a display implemented in the electronic device.

At block910, the processing unit sets a value of an angular distance between an orientation of the array of rendering pixels and the default orientation equal to zero.

At block915, a motion tracker implemented in the electronic device detects motion of the electronic device, e.g., due to movement of the head of the user wearing the electronic device.

At block920, the processing unit modifies the angular distance based on the detected motion. For example, the processing unit can modify the angular distance to be equal to an angular distance between the current orientation of the electronic device relative to the virtual scene and the default orientation.

At decision block925, the processing unit determines whether the angular distance is less than a threshold value that is determined based on an angular resolution of rendered pixels in the array of rendered pixels. The threshold value can be set equal to the angular resolution of a rendered pixel. If the angular distance is less than the threshold, the method900flows to block930. If the angular distance is larger than the threshold, the method flows to block935.

At block930, the processing unit maintains the orientation of the rendering array relative to the virtual scene, e.g., by shifting the orientation of the rendering array relative to an orientation of the display of the electronic device. The method900then flows to block915and the motion tracker continues to monitor motion of the electronic device.

At block935, the processing unit snaps (or reorients) the rendering array to a new or updated position relative to the virtual scene based on the angular distance. For example, the processing unit can modify the orientation of the rendering array to align with the default orientation. The method900then flows to block910and the processing unit such as the angular distance back to zero in response to snapping the rendering array back into alignment with the default orientation.

FIG. 10is a flow diagram of a method1000for rendering, upsampling, and translating display pixels representative of a virtual scene according to some embodiments. The method1000is implemented in some embodiments of an electronic device such as the electronic device105shown inFIG. 1, the electronic device205shown inFIG. 2, the electronic device305shown inFIG. 3, and the HMD405shown inFIG. 4.

At block1005, a processing unit in the electronic device renders values of rendered pixels based on a rendering array. The rendered pixels have a resolution that is lower than a native resolution of a display in the electronic device. For example, the rendered pixels can represent portions of the field-of-view that are in relatively low acuity regions such as the low acuity regions235,240,255,260shown inFIG. 2. As discussed herein, an orientation of the rendering array can be fixed relative to the virtual scene. The orientation of the rendering array can therefore be offset relative to an orientation that is fixed relative to the display in the processing unit.

At block1010, the processing unit upsampling the rendered pixels to generate values of display pixels for presentation by the display in the processing unit. For example, the rendered pixels can be upsampled to the native resolution of the display. As discussed herein, the rendered pixels are upsampled in a coordinate system that is fixed relative to the virtual scene. The display pixels should therefore be translated to compensate for any offset between the coordinate system that is fixed relative to the virtual scene and a coordinate system of the display.

At block1015, the processing unit translates the corners of the display pixels based on an angular distance that represents an offset between the coordinate system that is fixed relative to the virtual scene and the coordinate system of the display. The values of the display pixels can then be presented to the user via the display in the electronic device.