Patent Publication Number: US-2023136064-A1

Title: Priority-based graphics rendering for multi-part systems

Description:
TECHNICAL FIELD 
     This disclosure generally relates to generating graphics for an artificial reality scene. 
     BACKGROUND 
     An augmented reality (AR) system may generally include a real-world environment that includes AR content overlaying one or more features of the real-world environment. In typical AR systems, image data may be rendered on, for example, a robust head-mounted display (HMD) that may be coupled through a physical wired or wireless connection to a base graphics generation device responsible for generating the image data. However, in some instances, in which the HMD includes, for example, lightweight AR glasses and/or other wearable electronic devices as opposed to more robust headset devices, the AR glasses or other lightweight wearable electronic devices may, in comparison, include reduced battery capacity, processing power, low-resolution cameras, and/or relatively simple tracking optics. Thus, graphics rendering may be performed by the base graphics generation device, and then the image frames may be provided to the AR glasses. However, because all of the graphics rendering is typically performed by the base graphics generation device prior to transmitting image frames to the AR glasses via a wireless link, potential end-to-end latency between the graphics rendering on the base graphics generation device and the reprojection of image frames on the AR glasses may be increased (e.g., leading to the possibility of latency-based image artifacts being displayed to the user). It may be thus useful to provide techniques to improve artificial reality systems. 
     SUMMARY OF PARTICULAR EMBODIMENTS 
     The present embodiments include providing a priority based graphics rendering technique in which the graphics rendering frame rate is adjusted to a multiple of the heartbeat interval of the wireless link and the graphics rendering data is prioritized per rendering cycle. For example, in certain instances, the graphics rendering data may be segmented and prioritized (e.g., scored) based on how sensitive each segment of the graphics rendering data is to latency. For example, the higher latency sensitive graphics rendering data may be rendered during a first GPU rendering thread, while the lower latency sensitive graphics rendering data may be rendered during a subsequent GPU rendering thread. In this way, the latency of the higher latency sensitive graphics rendering data may be reduced to a single heartbeat interval of the wireless link. In certain instances, prioritizing (e.g. scoring) the graphics rendering data may be determined based on, but not limited to, image content (e.g., 2D objects vs 3D objects); object features (e.g., object edges and contours vs other features); application requirement (e.g., video game); developer preference; user head pose, user hand pose; user eye gaze; user activity; graphics data rendering deadline; whether objects are world-locked or head-locked; location of objects relative to a camera viewpoint; and so forth. In other instances, the present techniques may further include dynamically adjust the performance (e.g., processing speed) of the base graphics generation device based on the processing workload and the desired rendering deadline. For example, if an application or other process require that a frame be rendered within the 10 milliseconds (ms), the base graphics generation device may dynamically adjust its performance based on the processing workload, power capacity, and deadline, so as to complete the task within the deadline, for example, with only minimal impact to resource efficiency. 
     The embodiments disclosed herein are only examples, and the scope of this disclosure is not limited to them. Particular embodiments may include all, some, or none of the components, elements, features, functions, operations, or steps of the embodiments disclosed above. Embodiments according to the invention are in particular disclosed in the attached claims directed to a method, a storage medium, a system and a computer program product, wherein any feature mentioned in one claim category, e.g. method, can be claimed in another claim category, e.g. system, as well. The dependencies or references back in the attached claims are chosen for formal reasons only. However, any subject matter resulting from a deliberate reference back to any previous claims (in particular multiple dependencies) can be claimed as well, so that any combination of claims and the features thereof are disclosed and can be claimed regardless of the dependencies chosen in the attached claims. The subject-matter which can be claimed comprises not only the combinations of features as set out in the attached claims but also any other combination of features in the claims, wherein each feature mentioned in the claims can be combined with any other feature or combination of other features in the claims. Furthermore, any of the embodiments and features described or depicted herein can be claimed in a separate claim and/or in any combination with any embodiment or feature described or depicted herein or with any of the features of the attached claims. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG.  1    illustrates an example embodiment of a priority-based graphics rendering technique. 
         FIG.  2    illustrates an example base graphics generation system. 
         FIGS.  3 A- 3 B  illustrate an example scene graph and corresponding scenes. 
         FIG.  3 B  illustrates an example augmented reality system. 
         FIG.  4    illustrates another example embodiment of a priority-based graphics rendering technique. 
         FIGS.  5 A- 5 B  illustrate examples of artificial reality display systems. 
         FIG.  6    is a flow diagram of a method of an example priority-based graphics rendering technique. 
         FIG.  7    illustrates an example computer network. 
         FIG.  8    illustrates an example computer system. 
     
    
    
     DESCRIPTION OF EXAMPLE EMBODIMENTS 
     Because artificial reality devices involve creating digital scenes or superposing computer-generated imagery onto a view of the real world, they provide a platform for designers and engineers to provide new forms of information, entertainment, or methods of collaboration. For example, artificial reality devices may allow users to communicate, seemingly in person, over long distances, or assist users by informing them of the environment around them in an unobtrusive manner. Because artificial reality experiences can often be customized, the user&#39;s experience with artificial reality may be deeply personal and highly engaging if presented with sufficient clarity and convenience. 
     One way that artificial reality experiences can augment human ability is with computer-generated images and/or text added to the real world, as in an augmented or mixed reality. From this simple principle, a variety of compelling use cases can be considered. Labels (e.g., texts, glyphs, etc.) or images describing a real-world object may be fixed in the world space (e.g., location-aware labels acting as street signs or providing a live map of a bike path), or images fixed to a real-world object as it moves through the space (e.g., a label added to a bus as it going on its route that provides detailed information about its route or capacity). Labels could also be used to help a user navigate through an unfamiliar city (e.g., creating a waypoint for the nearest restroom), or help find a friend in a crowd (e.g., a socially-aware waypoint fixed to another user). Other experiences worth considering may be based on interactions with real-world objects. For example, a user could “project” video onto a wall or screen that allows for the video to be played and visible to only herself or to others with access to a shared augmented space. As another example, a user could fix computer-generated text to a physical object to act as an augmented-reality book or magazine. Content could be displayed relative to the object (allowing a user to physical asset aside an augmented-reality) or could be displayed in a fixed relation to the user&#39;s (e.g., a tutorial video constantly playing in a corner of the view). Presented media could be customized to the user, so that the same content display space could content relevant to each person viewing the same physical space. As another example, a user could interact with computer-generated graphics by “touching” an icon, or “manipulating” the computer-generated images manually. These graphics could be shown to multiple users working on a project, enabling opportunities for team collaboration (e.g., multiple architects working on a three-dimensional digital prototype in a building together in real-time). 
     To facilitate use, the display that outputs the computer-generated graphics should be intuitive, constantly accessible, and unobtrusive. One approach for displaying high definition artificial reality graphics to a user is based on a head-mounted display. The user wears an apparatus, such as a visor, headset, or glasses, capable of displaying computer graphics display. In augmented or mixed reality experiences, the computer graphics can be seen alongside, or on top of, the physical world. However, rendering these computer graphics is computationally intensive. Therefore, in most cases rendering is performed by powerful computers communicatively attached (e.g., via a cable or wireless communication protocol, such as Bluetooth) to a head-mounted display. In such a configuration, the head-mounted display is limited by bulky cords, bandwidth and power limitations, heat restrictions, and other related constraints. Yet, the limits of these constraints are being pushed. Head-mounted displays that are comfortable and efficient enough for day-long wearing, yet powerful enough to display sophisticated graphics are currently being developed. 
     One technique used to reduce actual display size without impacting apparent display size is known as a scanning display. In a scanning display, multiple smaller images are combined to form a larger composite image. The scanning display uses source light, one or more scanning elements comprising reflectors, and an optics system to generate and output image light. The output image light may be output to the eye of the user. The source light may be provided by emitters, such as light-emitting diodes (LEDs). For example, the light source may be an array of 2560×1440 LEDs. The reflectors may be any suitable reflective surface attached to the scanning element. In particular embodiments, the scanning element may be a scanning mirror driven using one or more microelectromechanical systems (MEMS) components. The optics system may comprise lenses used to focus, redirect, and otherwise augment the light. The scanning element may cause the source light, treated by light guiding components, to be output to the eye of the user in a specific pattern corresponding to a generation pattern used by the emitters to optimize display draw rate. Because, for example, all emitters need not be active at once, and in addition to a variety of other factors, scanning displays may require less power to run, and may generate less heat, than traditional display comprised of the same emitters. They may have less weight as well, owing in part to the quality of the materials used in the scanning element and optics system. One consequence of using a scanning display is that in exchange for, e.g., power, weight, and heat efficiency, a scanning displays may not perfectly display images as presented to them, e.g., images intended for traditional displays. There may be non-uniform distortions such as geometric warping of images and distortion of colors and specifically brightness. As is explained further herein, these distortions can be corrected by post-processing graphics to-be displayed to counteract the distortion before they are passed to the display, creating the effect that there is no distortion. Although this disclosure describes displays in a particular manner, this disclosure contemplates any suitable displays. 
     Since its existence, artificial reality (e.g., AR, VR, MR) technology has been plagued with the problem of latency in rendering AR/VR/MR objects in response to sudden changes in a user&#39;s perspective of an AR/VR/MR scene. To create an immersive environment, users may need to be able to move their heads around when viewing a scene and the environment may need to respond immediately by adjusting the view presented to the user. Each head movement may slightly change the user&#39;s perspective of the scene. These head movements may be small but sporadic and difficult, if not impossible, to predict. A problem to be solved is that the head movements may occur quickly, requiring that the view of the scene be modified rapidly to account for changes in perspective that occur with the head movements. If this is not done rapidly enough, the resulting latency may cause a user to experience a sensory dissonance that can lead to virtual reality sickness or discomfort, or at the very least, a disruption to the immersive nature of the experience. 
     Particular embodiments described herein are directed to a base graphics generation system that is wirelessly coupled to a headset display device (HMD) and how the base graphics generation system (or otherwise referred herein as a “Stage”) renders artificial reality content and communicates the rendered content to the HMD.  FIG.  1    illustrates an example of a priority-based graphics rendering technique (bottom half of  FIG.  1   ) and a traditional technique (top half). The wireless heartbeats (WHB) illustrated in  FIG.  1    represent time intervals at which a Stage communicates with an HMD. The interval of the WHBs may be a fixed period, e.g., 45 Hz or 60 Hz, or approximately every 22.22 ms or 16.67 ms. The Vsync illustrated in  FIG.  1    represents the graphics processing unit (GPU) framerate, e.g., time intervals during which the GPU renders graphic content for a particular frame. Ideally, GPU vsync and wireless heartbeat are aligned in a way that allows the GPU to finish rendering the frame right before wireless interval starts. However, due to the latency resulting from the rendering process, the GPU typically does not have enough time to render and transmit an entire frame within one wireless heartbeat from when the camera pose is received. The delayed transmission of the rendered content may result in latency-based image artifacts being displayed to a user. The top half of FIG. 1  illustrates this problem, where it shows that the rendering process for Frame A, including the post-processing, takes longer than one wireless heartbeat. For example, referring to the top half of  FIG.  1   , the rendering process may begin at the Pre Processing  112  box. At this step, the Stage may receive positional and/or sensor data from the HMD (e.g., camera pose). The Stage then determines which assets should be render into a frame, and, at the Render Thread Frame A  114  box, the Stage instructs the GPU to render Frame A. Subsequently, at the Post Processing  116  box, the rendered content is processed (e.g., packaged and compressed) for transmission to the HMD. As shown in  FIG.  1   , the time needed to render and process Frame A exceed a single wireless heartbeat (e.g., two). That is, by the time the Stage sends the rendered frame to the HMD, two wireless heartbeats have passed, and the pose used to render Frame A may be outdated when the rendered Frame A is displayed on the HMD. 
     The embodiments disclosed herein provide a solution to the problem discussed above by prioritizing assets within a particular frame, increasing the GPU framerate (Vsync), separating assets into one or more groups based on the prioritization, and rendering the groups into separate sub-frames, or “superframes.” For example, high-priority assets may be grouped together and rendered into a first superframe using an increased GPU framerate, such that the first superframe can be sent to the HMD within one wireless heartbeat from the time the camera pose is received. The low-priority assets may be placed in another group (or multiple groups), which are then rendered and processed into additional superframes and sent to the HMD in subsequent wireless heartbeats. This technique allows high-priority assets to be rendered and sent to the HMD within a single wireless heartbeat (e.g., 45 Hz or 60 Hz) while low-priority assets are rendered and sent to the HMD at subsequent wireless heartbeats. The bottom half of  FIG.  1    illustrates this technique. The example embodiment  200  in  FIG.  1    shows that the GPU framerate, or Vsync, is approximately double the rate of the Vsync shown in the traditional systems  100 , meaning the rendering time, per each GPU frame, is approximately reduced in half. In the example embodiment  200 , assets that are to be rendered for Frame A are separated into two groups: high-priority and low-priority. At the Render Superframe A—High Priority  214  step, the assets in the high-priority group are rendered and processed into a first superframe and transmitted to the HMD at the second wireless heartbeat (at the first wireless heartbeat, the Stage may receive pose and positional data from the HMD). At the Render Superframe A—Low Priority  224  step, assets in the low-priority group are rendered and processed into a second superframe to be transmitted at the third wireless heartbeat. The assets that are to be rendered for Frame B are similarly prioritized and rendered into two superframes. The high-priority assets of superframe B may be rendered and sent to the HMD at the same wireless heartbeat as the low-priority assets of superframe A. In particular embodiments, each of the superframes A (both the high- and low-priority assets) may be rendered using the same camera pose, for example, the camera pose received at the PreP  212 . Similarly, each of the Superframes B (both the high- and low-priority assets) may be rendered using the same camera pose, for example, the camera pose received at the PreP  232 . 
     In particular embodiments, when two or more frames or superframes are sent to the HMD at the same wireless heartbeat, frames or superframes with higher-priority assets may be sent to the HMD first. For example, in the example embodiment  200  illustrated in  FIG.  1   , it can be seen that superframe A with low priority assets  224  and superframe B with high priority assets  234  are ready to be sent to the HMD at the same wireless heartbeat (e.g., the third wireless heartbeat). In such a scenario, the superframe B with the high priority assets may be placed in the wireless transmission queue before the superframe A with the low priority assets to allow the HMD to receive the high priority assets first. 
     In particular embodiments, the GPU framerate may be dynamically adjusted (e.g., at a multiple of the wireless heartbeat), such that a particular number of superframes may be rendered and processed for transmission within a particular number of wireless heartbeats. For example, the embodiment  200  of  FIG.  1    shows that the GPU framerate has been increased to allow the third rendering task (Superframe B corresponding to high priority assets) to be completed prior to the third wireless heartbeat. 
       FIG.  2    illustrates an example artificial reality graphics rendering system  300 , e.g., components of a Stage  300 . As depicted, components of a Stage  300  may include applications  310 , scene manager  305 , and GPU  345 . In particular embodiments, the Stage  300  may host and service applications  310 , which may include artificial reality experiences displayed on an HMD. For example, the applications  310  may include video gaming applications (e.g., single-player games, multi-player games, first-person point of view (POV) games), mapping applications, music playback applications, video-sharing platform applications, video-streaming applications, e-commerce applications, social media applications, user interface (UI) applications, or other AR/VR applications. In particular embodiments, the applications  310  or other AR/VR content may be analyzed and managed by a scene manager  305 . In particular embodiments, the scene manager  305  may analyze and manage 3D content obtained from applications  310  (e.g., geometry, texture, etc.) and keep track of the available hardware and/or software components for hosting and servicing the applications  310  or other AR/VR content. In particular embodiments, the scene manager  305  may maintain and keep track of pose information (e.g., head pose data, object pose data) received from the HMD. In particular embodiments, the scene manager  305  may utilize the main UI thread  330  to read a scene graph and perform transformation updates to the assets. In particular embodiments, the scene manager  305  may, via the main UI thread  330 , issue worker UI Threads  340  to generate and update assets&#39; geometry and/or texture. 
     In particular embodiments, in order to determine the priority of assets that are to be rendered, the scene manager  305  may determine a priority score for each of the assets. The priority scores are determined based on various factors/determinants, including the type of the asset (e.g., 2D object vs. 3D object), lock of the asset (e.g., world-locked, body-locked, hand-locked, head-locked), distance between the asset and a user, size of the asset, visibility of the asset, focus of a user&#39;s gaze, and application-specific factors (e.g., assets for gaming application, assets for text application). For example, world-locked objects may be assigned a higher priority score than head-locked objects; objects appearing within the FOV of a user may be assigned a higher priority score than objects appearing outside of the FOV (e.g., relative to the screen or, if eye-tracking is available, relative to the user&#39;s gaze); objects with complex geometry (e.g., more triangles, irregular shape) may be assigned a higher priority score than object with less-complex geometry; objects with complex shading (e.g., color) may be assigned a higher priority score than object with less-complex shading; large objects may be assigned a higher priority score than smaller objects; objects that are closer to a user may be may be assigned a higher priority score than objects that are further away; 3D objects may be assigned a higher priority score than 2D objects; videos may be assigned higher objects associated with certain applications (e.g., gaming application) may be assigned a higher priority score than objects associated with other applications (e.g., weather widget); and assets or applications that are currently in focus based on user interaction may be assigned a higher priority score than assets or applications that are not in focus. 
     In particular embodiments, a scene manager  305  may generate and maintain a scene graph  320  to organize and keep track of the factors/determinants used to determine the priority scores. A scene graph consists of nodes, arranged in a tree structure, and can be used to organizes and controls the rendering of its constituent objects.  FIG.  3 A  shows an example scene graph with various SceneNodes and their constituent objects, and  FIG.  3 B  illustrates an example of how these objects may look when rendered. Referring to  FIG.  3 A , the scene manager  305  may determine that SceneNode  3  should be assigned a high priority score because its asset, 3D shoes, is visible to the user, distanced close to the user, is a 3D object, and is medium in size. The scene manager  305  may determine that SceneNode  4  should be assigned a medium priority score rather than a high priority score because its asset, a text message, while having similar characteristics as the 3D shoes, is a 2D object. The scene manager  305  may similarly determine that SceneNode  8  should be assigned a low priority score based on the characteristics of its asset, a news widget (e.g., small in size, a 2D object, and partially invisible). Based on these scores, the scene manager  305  may determine that the 3D shoes should be rendered first, followed by the text message, then the news widget. 
     In particular embodiments, the scene manager  305  may determine how to segment or separate out the assets into one or more groups in a dynamic fashion, for example, by first instructing the GPU to render the highest-priority assets, then checking to see whether there is sufficient time to render the next group of assets before the next wireless heartbeat (with sufficient time to allow post-processing of the rendered content). For example, referring to the scene graph shown in  FIG.  3 A , the scene manager  305  may instruct the GPU to first render the highest-priority scene (3D shoe) then determine whether there is sufficient time remaining to render the next scene before the next wireless heartbeat. If yes, the scene manager  305  may instruct the GPU to render the next priority scene (text message) in the same superframe as the 3D shoe. Then, if the scene manager  305  determines that there is not enough time to render the next priority scene (news widget), the scene manager  305  may place the news widget into a second group to be rendered into a second superframe. 
     In particular embodiments, the scene manager  305  may determine how to segment or separate out the assets or SceneNodes in one or more groups by predicting the amount of rendering time each asset or SceneNode will require. For example, referring to the scene graph shown in  FIG.  3 A , the scene manager  305  may analyze the SceneNode  3  and, based on the characteristics of the asset (3D shoe), determine that SceneNode  3  is the only scene that can be rendered before the next rendering deadline. In such a scenario, the scene manager  305  may assign SceneNode  3  in the first group (e.g., for the first superframe). The scene manager  305  may then similarly determine that SceneNodes  4  and  8  can both be rendered before the second wireless heartbeat and instruct the GPU to render both scenes in to a second superframe prior to the second wireless heartbeat. In particular embodiments, the scene manager  305  may predict the amount of rendering time each asset or SceneNode will require based on historical data indicating the amount of time a particular type of asset takes for rendering. In other embodiments, the prediction may be made based on the amount of memory read or write that a particular asset will require for rendering, for example, based on the number of triangles or vertices in the geometry, size of the source texture, color depth of the source texture, distance of the object to the camera, etc. In some embodiments, the prediction may be made based on the same or similar determinants used to determine the priority scores. In yet other embodiments, the amount of rendering time each asset or SceneNode will require may be provided by the applications that provide the assets. 
     In particular embodiments, the applications  310  may perform the rendering tasks on their own and send the rendered content to the scene manager  305 . In particular embodiments, the scene manager  305  may instruct the applications on how to prioritize the assets based on similar methods described above. The scene manager  305  may then utilize a compositor assemble the various images of the assets. 
     In particular embodiments, the GPU frequency may be dynamically adjusted based on the workload and the desired rendering deadline. While GPUs typically run at a frequency needed to support a target frame rate (e.g., 2 GHz), GPUs do not adjust their performance level to meet the rendering deadline of individual assets within a scene. This is not power-efficient since the relationship between power consumption and processing frequency is not linear (e.g., if processing at 1 GHz for a certain time period takes n Watts, processing at 2 GHz for the same amount of time might take 4n Watts). In particular embodiments, the GPU&#39;s frequency is adaptively adjusted based on the workload and the desired rendering deadline. For example, if a system wants a certain number of assets to be rendered within 25 ms, it could assess how much workload there is to see how fast the GPU needs to run in order to complete the task on time. If there isn&#39;t that much work to be done, the GPU, rather than doing a sprint and then stop during that 25 ms, could instead walk for 25 ms to complete the same workload. By adaptively slowing down, the GPU would expend a lot less power. For another example, referring to  FIG.  3 A , if the scene manager  305  determines that SceneNode  8  should be rendered into a second superframe but the GPU is expected to finish the rendering task early (determined based on the methods of predicting the amount of rendering time discussed above), the scene manager  305  may instruct the GPU to lower it&#39;s frequency so that the rendering task completes slower, or with just enough time to post-process and transmit the rendered content to the HMD before a particular wireless heartbeat. In particular embodiments, the GPU frequency may be adjusted during runtime, for example, halfway through the GPU frame. 
     In particular embodiments, when there is sufficient amount of GPU idle time in between the rendering tasks, the rendering tasks for a particular frame may be separated into multiple GPU framerates to utilize the GPU more efficiently. For example,  FIG.  4    illustrates an embodiment  400  where Frame A is rendered and processed in two wireless heartbeats and Frame B is subsequently rendered and processed in the next two wireless heartbeats. Given that the GPU would be idle in between the rendering tasks for Frame A and Frame B, the scene manager  305  may separate the rendering tasks for Frame A into two groups such that high-priority assets are rendered first at the Render Frame A  414  box and low-priority assets are rendered at the Render Frame Z  424  box. In other words, the scene manager  305  may separate out the rendering tasks for Frame A into two frames such that the GPU renders the high priority assets in Frame A and the low priority assets in Frame Z, i.e., during the period when the GPU would have been idle. The significance of these embodiments is not necessarily for reducing the rendering latency because, in contrast to other embodiments disclosed herein, the GPU framerate is not increased (e.g., rendering all of the assets for Frame A still takes two wireless heartbeats), but rather these embodiments provide the GPU additional time to render assets. With this additional time, the GPU frequency could be adjusted, as discussed above, to utilize the GPU more efficiently. In particular embodiments, Frame Z illustrated in  FIG.  4    may be rendered based on the pose received at the PreP  412  box. In other embodiments, Frame Z may be rendered based on a newly received pose at PreP  422  Box. 
       FIG.  5 A  illustrates an example artificial reality system  500 A. In particular embodiments, the artificial reality system  500 A may comprise a headset  504 , a controller  506 , and a computing system  508 . A user  502  may wear the headset  504  that may display visual artificial reality content to the user  502 . The headset  504  may include an audio device that may provide audio artificial reality content to the user  502 . The headset  504  may include one or more cameras which can capture images and videos of environments. The headset  504  may include an eye tracking system to determine the vergence distance of the user  502 . The headset  504  may be referred as a head-mounted display (HMD). The controller  506  may comprise a trackpad and one or more buttons. The controller  506  may receive inputs from the user  502  and relay the inputs to the computing system  508 . The controller  206  may also provide haptic feedback to the user  502 . The computing system  508  may be connected to the headset  504  and the controller  506  through cables or wireless connections. The computing system  508  may control the headset  504  and the controller  506  to provide the artificial reality content to and receive inputs from the user  502 . The computing system  508  may be a standalone host computer system, an on-board computer system integrated with the headset  504 , a mobile device, or any other hardware platform capable of providing artificial reality content to and receiving inputs from the user  502 . 
       FIG.  5 B  illustrates an example augmented reality system  500 B. The augmented reality system  500 B may include a head-mounted display (HMD)  510  (e.g., glasses) comprising a frame  512 , one or more displays  514 , and a computing system  520 . The displays  514  may be transparent or translucent allowing a user wearing the HMD  510  to look through the displays  514  to see the real world and displaying visual artificial reality content to the user at the same time. The HMD  510  may include an audio device that may provide audio artificial reality content to users. The HMD  510  may include one or more cameras which can capture images and videos of environments. The HMD  510  may include an eye tracking system to track the vergence movement of the user wearing the HMD  510 . The augmented reality system  500 B may further include a controller comprising a trackpad and one or more buttons. The controller may receive inputs from users and relay the inputs to the computing system  520 . The controller may also provide haptic feedback to users. The computing system  520  may be connected to the HMD  510  and the controller through cables or wireless connections. The computing system  520  may control the HMD  510  and the controller to provide the augmented reality content to and receive inputs from users. The computing system  520  may be a standalone host computer system, an on-board computer system integrated with the HMD  510 , a mobile device, or any other hardware platform capable of providing artificial reality content to and receiving inputs from users. 
       FIG.  6    illustrates an example method  600  for a priority based graphics rendering technique. The method may begin at step  604  by receiving a viewpoint from a headset. At step  602 , the method may continue by receiving, from one or more applications, a plurality of assets to be rendered using the viewpoint. At step  603 , the method may continue by determining, for each of the plurality of assets, a priority score. At step  604 , the method may continue by identifying, based on the determined priority scores, a first subset and a second subset of the plurality of assets. At step  605 , the method may continue by instructing a GPU to render, within a first GPU frame, a first superframe using the first subset. At step  606 , the method may continue by enqueuing the first superframe for transmission to the headset with a first priority and at a first pre-scheduled time. At step  607 , the method may continue by instructing the GPU to render, within a second GPU frame, a second superframe using the second subset. At step  608 , the method may continue by enqueuing the second superframe for transmission to the headset with a second priority lower than the first priority and at a second pre-scheduled time. At step  609 , the method specifies that the GPU has a GPU framerate faster than a pre-scheduled timing interval for transmitting data from the computing system to the headset. Particular embodiments may repeat one or more steps of the method of  FIG.  6   , where appropriate. Although this disclosure describes and illustrates particular steps of the method of  FIG.  6    as occurring in a particular order, this disclosure contemplates any suitable steps of the method of  FIG.  6    occurring in any suitable order. Moreover, although this disclosure describes and illustrates an example method for a priority based graphics rendering technique, this disclosure contemplates any suitable method for rendering graphics based on prioritization, which may include all, some, or none of the steps of the method of  FIG.  6   , where appropriate. Furthermore, although this disclosure describes and illustrates particular components, devices, or systems carrying out particular steps of the method of  FIG.  6   , this disclosure contemplates any suitable combination of any suitable components, devices, or systems carrying out any suitable steps of the method of  FIG.  6   . 
       FIG.  7    illustrates an example network environment  700  associated with a social-networking system. Network environment  700  includes a user  701 , a client system  730 , a social-networking system  760 , and a third-party system  770  connected to each other by a network  710 . Although  FIG.  7    illustrates a particular arrangement of user  701 , client system  730 , social-networking system  760 , third-party system  770 , and network  710 , this disclosure contemplates any suitable arrangement of user  701 , client system  730 , social-networking system  760 , third-party system  770 , and network  710 . As an example and not by way of limitation, two or more of client system  730 , social-networking system  760 , and third-party system  770  may be connected to each other directly, bypassing network  710 . As another example, two or more of client system  730 , social-networking system  760 , and third-party system  770  may be physically or logically co-located with each other in whole or in part. Moreover, although  FIG.  7    illustrates a particular number of users  701 , client systems  730 , social-networking systems  760 , third-party systems  770 , and networks  710 , this disclosure contemplates any suitable number of users  701 , client systems  730 , social-networking systems  760 , third-party systems  770 , and networks  710 . As an example and not by way of limitation, network environment  700  may include multiple users  701 , client system  730 , social-networking systems  760 , third-party systems  770 , and networks  710 . 
     In particular embodiments, user  701  may be an individual (human user), an entity (e.g., an enterprise, business, or third-party application), or a group (e.g., of individuals or entities) that interacts or communicates with or over social-networking system  760 . In particular embodiments, social-networking system  760  may be a network-addressable computing system hosting an online social network. Social-networking system  760  may generate, store, receive, and send social-networking data, such as, for example, user-profile data, concept-profile data, social-graph information, or other suitable data related to the online social network. Social-networking system  760  may be accessed by the other components of network environment  700  either directly or via network  710 . In particular embodiments, social-networking system  760  may include an authorization server (or other suitable component(s)) that allows users  701  to opt in to or opt out of having their actions logged by social-networking system  760  or shared with other systems (e.g., third-party systems  770 ), for example, by setting appropriate privacy settings. A privacy setting of a user may determine what information associated with the user may be logged, how information associated with the user may be logged, when information associated with the user may be logged, who may log information associated with the user, whom information associated with the user may be shared with, and for what purposes information associated with the user may be logged or shared. Authorization servers may be used to enforce one or more privacy settings of the users of social-networking system  30  through blocking, data hashing, anonymization, or other suitable techniques as appropriate. In particular embodiments, third-party system  770  may be a network-addressable computing system that can host social media information or AR, VR, or MR content. Third-party system  770  may be accessed by the other components of network environment  700  either directly or via network  710 . In particular embodiments, one or more users  701  may use one or more client systems  730  to access, send data to, and receive data from social-networking system  760  or third-party system  770 . Client system  730  may access social-networking system  760  or third-party system  770  directly, via network  710 , or via a third-party system. As an example and not by way of limitation, client system  730  may access third-party system  770  via social-networking system  760 . Client system  730  may be any suitable computing device, such as, for example, a personal computer, a laptop computer, a cellular telephone, a smartphone, a tablet computer, or an augmented/virtual reality device. 
     This disclosure contemplates any suitable network  710 . As an example and not by way of limitation, one or more portions of network  710  may include an ad hoc network, an intranet, an extranet, a virtual private network (VPN), a local area network (LAN), a wireless LAN (WLAN), a wide area network (WAN), a wireless WAN (WWAN), a metropolitan area network (MAN), a portion of the Internet, a portion of the Public Switched Telephone Network (PSTN), a cellular telephone network, or a combination of two or more of these. Network  710  may include one or more networks  710 . 
     Links  750  may connect client system  730 , social-networking system  760 , and third-party system  770  to communication network  710  or to each other. This disclosure contemplates any suitable links  750 . In particular embodiments, one or more links  750  include one or more wireline (such as for example Digital Subscriber Line (DSL) or Data Over Cable Service Interface Specification (DOCSIS)), wireless (such as for example Wi-Fi or Worldwide Interoperability for Microwave Access (WiMAX)), or optical (such as for example Synchronous Optical Network (SONET) or Synchronous Digital Hierarchy (SDH)) links. In particular embodiments, one or more links  750  each include an ad hoc network, an intranet, an extranet, a VPN, a LAN, a WLAN, a WAN, a WWAN, a MAN, a portion of the Internet, a portion of the PSTN, a cellular technology-based network, a satellite communications technology-based network, another link  750 , or a combination of two or more such links  750 . Links  750  need not necessarily be the same throughout network environment  700 . One or more first links  750  may differ in one or more respects from one or more second links  750 . 
       FIG.  8    illustrates an example computer system  800  that may be useful in performing one or more of the foregoing techniques as presently disclosed herein. In particular embodiments, one or more computer systems  800  perform one or more steps of one or more methods described or illustrated herein. In particular embodiments, one or more computer systems  800  provide functionality described or illustrated herein. In particular embodiments, software running on one or more computer systems  800  performs one or more steps of one or more methods described or illustrated herein or provides functionality described or illustrated herein. Particular embodiments include one or more portions of one or more computer systems  800 . Herein, reference to a computer system may encompass a computing device, and vice versa, where appropriate. Moreover, reference to a computer system may encompass one or more computer systems, where appropriate. 
     This disclosure contemplates any suitable number of computer systems  800 . This disclosure contemplates computer system  800  taking any suitable physical form. As example and not by way of limitation, computer system  800  may be an embedded computer system, a system-on-chip (SOC), a single-board computer system (SBC) (such as, for example, a computer-on-module (COM) or system-on-module (SOM)), a desktop computer system, a laptop or notebook computer system, an interactive kiosk, a mainframe, a mesh of computer systems, a mobile telephone, a personal digital assistant (PDA), a server, a tablet computer system, an augmented/virtual reality device, or a combination of two or more of these. Where appropriate, computer system  800  may include one or more computer systems  800 ; be unitary or distributed; span multiple locations; span multiple machines; span multiple data centers; or reside in a cloud, which may include one or more cloud components in one or more networks. Where appropriate, one or more computer systems  800  may perform without substantial spatial or temporal limitation one or more steps of one or more methods described or illustrated herein. 
     As an example, and not by way of limitation, one or more computer systems  800  may perform in real time or in batch mode one or more steps of one or more methods described or illustrated herein. One or more computer systems  800  may perform at different times or at different locations one or more steps of one or more methods described or illustrated herein, where appropriate. In certain embodiments, computer system  800  includes a processor  802 , memory  804 , storage  806 , an input/output (I/O) interface  808 , a communication interface  810 , and a bus  812 . Although this disclosure describes and illustrates a particular computer system having a particular number of particular components in a particular arrangement, this disclosure contemplates any suitable computer system having any suitable number of any suitable components in any suitable arrangement. 
     In certain embodiments, processor  802  includes hardware for executing instructions, such as those making up a computer program. As an example, and not by way of limitation, to execute instructions, processor  802  may retrieve (or fetch) the instructions from an internal register, an internal cache, memory  804 , or storage  806 ; decode and execute them; and then write one or more results to an internal register, an internal cache, memory  804 , or storage  806 . In particular embodiments, processor  802  may include one or more internal caches for data, instructions, or addresses. This disclosure contemplates processor  802  including any suitable number of any suitable internal caches, where appropriate. As an example, and not by way of limitation, processor  802  may include one or more instruction caches, one or more data caches, and one or more translation lookaside buffers (TLBs). Instructions in the instruction caches may be copies of instructions in memory  804  or storage  806 , and the instruction caches may speed up retrieval of those instructions by processor  802 . 
     Data in the data caches may be copies of data in memory  804  or storage  806  for instructions executing at processor  802  to operate on; the results of previous instructions executed at processor  802  for access by subsequent instructions executing at processor  802  or for writing to memory  804  or storage  806 ; or other suitable data. The data caches may speed up read or write operations by processor  802 . The TLBs may speed up virtual-address translation for processor  802 . In particular embodiments, processor  802  may include one or more internal registers for data, instructions, or addresses. This disclosure contemplates processor  802  including any suitable number of any suitable internal registers, where appropriate. Where appropriate, processor  802  may include one or more arithmetic logic units (ALUs); be a multi-core processor; or include one or more processors  602 . Although this disclosure describes and illustrates a particular processor, this disclosure contemplates any suitable processor. 
     In certain embodiments, memory  804  includes main memory for storing instructions for processor  802  to execute or data for processor  802  to operate on. As an example, and not by way of limitation, computer system  800  may load instructions from storage  806  or another source (such as, for example, another computer system  800 ) to memory  804 . Processor  802  may then load the instructions from memory  804  to an internal register or internal cache. To execute the instructions, processor  802  may retrieve the instructions from the internal register or internal cache and decode them. During or after execution of the instructions, processor  802  may write one or more results (which may be intermediate or final results) to the internal register or internal cache. Processor  802  may then write one or more of those results to memory  804 . In particular embodiments, processor  802  executes only instructions in one or more internal registers or internal caches or in memory  804  (as opposed to storage  806  or elsewhere) and operates only on data in one or more internal registers or internal caches or in memory  804  (as opposed to storage  806  or elsewhere). 
     One or more memory buses (which may each include an address bus and a data bus) may couple processor  802  to memory  804 . Bus  812  may include one or more memory buses, as described below. In particular embodiments, one or more memory management units (MMUs) reside between processor  802  and memory  804  and facilitate accesses to memory  804  requested by processor  802 . In particular embodiments, memory  804  includes random access memory (RAM). This RAM may be volatile memory, where appropriate. Where appropriate, this RAM may be dynamic RAM (DRAM) or static RAM (SRAM). Moreover, where appropriate, this RAM may be single-ported or multi-ported RAM. This disclosure contemplates any suitable RAM. Memory  804  may include one or more memories  804 , where appropriate. Although this disclosure describes and illustrates particular memory, this disclosure contemplates any suitable memory. 
     In particular embodiments, storage  806  includes mass storage for data or instructions. As an example, and not by way of limitation, storage  806  may include a hard disk drive (HDD), a floppy disk drive, flash memory, an optical disc, a magneto-optical disc, magnetic tape, or a Universal Serial Bus (USB) drive or a combination of two or more of these. Storage  806  may include removable or non-removable (or fixed) media, where appropriate. Storage  806  may be internal or external to computer system  800 , where appropriate. In particular embodiments, storage  806  is non-volatile, solid-state memory. In certain embodiments, storage  806  includes read-only memory (ROM). Where appropriate, this ROM may be mask-programmed ROM, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), electrically alterable ROM (EAROM), or flash memory or a combination of two or more of these. This disclosure contemplates mass storage  806  taking any suitable physical form. Storage  806  may include one or more storage control units facilitating communication between processor  802  and storage  806 , where appropriate. Where appropriate, storage  806  may include one or more storages  806 . Although this disclosure describes and illustrates particular storage, this disclosure contemplates any suitable storage. 
     In certain embodiments, I/O interface  808  includes hardware, software, or both, providing one or more interfaces for communication between computer system  800  and one or more I/O devices. Computer system  800  may include one or more of these I/O devices, where appropriate. One or more of these I/O devices may enable communication between a person and computer system  800 . As an example, and not by way of limitation, an I/O device may include a keyboard, keypad, microphone, monitor, mouse, printer, scanner, speaker, still camera, stylus, tablet, touch screen, trackball, video camera, another suitable I/O device or a combination of two or more of these. An I/O device may include one or more sensors. This disclosure contemplates any suitable I/O devices and any suitable I/O interfaces  808  for them. Where appropriate, I/O interface  808  may include one or more device or software drivers enabling processor  802  to drive one or more of these I/O devices. I/O interface  808  may include one or more I/O interfaces  808 , where appropriate. Although this disclosure describes and illustrates a particular I/O interface, this disclosure contemplates any suitable I/O interface. 
     In certain embodiments, communication interface  810  includes hardware, software, or both providing one or more interfaces for communication (such as, for example, packet-based communication) between computer system  800  and one or more other computer systems  800  or one or more networks. As an example, and not by way of limitation, communication interface  810  may include a network interface controller (NIC) or network adapter for communicating with an Ethernet or other wire-based network or a wireless NIC (WNIC) or wireless adapter for communicating with a wireless network, such as a WI-FI network. This disclosure contemplates any suitable network and any suitable communication interface  810  for it. 
     As an example, and not by way of limitation, computer system  800  may communicate with an ad hoc network, a personal area network (PAN), a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), or one or more portions of the Internet or a combination of two or more of these. One or more portions of one or more of these networks may be wired or wireless. As an example, computer system  800  may communicate with a wireless PAN (WPAN) (such as, for example, a BLUETOOTH WPAN), a WI-FI network, a WI-MAX network, a cellular telephone network (such as, for example, a Global System for Mobile Communications (GSM) network), or other suitable wireless network or a combination of two or more of these. Computer system  800  may include any suitable communication interface  810  for any of these networks, where appropriate. Communication interface  810  may include one or more communication interfaces  810 , where appropriate. Although this disclosure describes and illustrates a particular communication interface, this disclosure contemplates any suitable communication interface. 
     In certain embodiments, bus  812  includes hardware, software, or both coupling components of computer system  800  to each other. As an example and not by way of limitation, bus  812  may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a front-side bus (FSB), a HYPERTRANSPORT (HT) interconnect, an Industry Standard Architecture (ISA) bus, an INFINIBAND interconnect, a low-pin-count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCIe) bus, a serial advanced technology attachment (SATA) bus, a Video Electronics Standards Association local (VLB) bus, or another suitable bus or a combination of two or more of these. Bus  812  may include one or more buses  812 , where appropriate. Although this disclosure describes and illustrates a particular bus, this disclosure contemplates any suitable bus or interconnect. 
     Herein, a computer-readable non-transitory storage medium or media may include one or more semiconductor-based or other integrated circuits (ICs) (such, as for example, field-programmable gate arrays (FPGAs) or application-specific ICs (ASICs)), hard disk drives (HDDs), hybrid hard drives (HHDs), optical discs, optical disc drives (ODDs), magneto-optical discs, magneto-optical drives, floppy diskettes, floppy disk drives (FDDs), magnetic tapes, solid-state drives (SSDs), RAM-drives, SECURE DIGITAL cards or drives, any other suitable computer-readable non-transitory storage media, or any suitable combination of two or more of these, where appropriate. A computer-readable non-transitory storage medium may be volatile, non-volatile, or a combination of volatile and non-volatile, where appropriate. 
     Herein, “or” is inclusive and not exclusive, unless expressly indicated otherwise or indicated otherwise by context. Therefore, herein, “A or B” means “A, B, or both,” unless expressly indicated otherwise or indicated otherwise by context. Moreover, “and” is both joint and several, unless expressly indicated otherwise or indicated otherwise by context. Therefore, herein, “A and B” means “A and B, jointly or severally,” unless expressly indicated otherwise or indicated otherwise by context. 
     The scope of this disclosure encompasses all changes, substitutions, variations, alterations, and modifications to the example embodiments described or illustrated herein that a person having ordinary skill in the art would comprehend. The scope of this disclosure is not limited to the example embodiments described or illustrated herein. Moreover, although this disclosure describes and illustrates respective embodiments herein as including particular components, elements, feature, functions, operations, or steps, any of these embodiments may include any combination or permutation of any of the components, elements, features, functions, operations, or steps described or illustrated anywhere herein that a person having ordinary skill in the art would comprehend. Furthermore, reference in the appended claims to an apparatus or system or a component of an apparatus or system being adapted to, arranged to, capable of, configured to, enabled to, operable to, or operative to perform a particular function encompasses that apparatus, system, component, whether or not it or that particular function is activated, turned on, or unlocked, as long as that apparatus, system, or component is so adapted, arranged, capable, configured, enabled, operable, or operative. Additionally, although this disclosure describes or illustrates particular embodiments as providing particular advantages, particular embodiments may provide none, some, or all of these advantages.