Patent ID: 12249092

DESCRIPTION OF EXAMPLE EMBODIMENTS

Artificial reality systems may have a limited power budget given the form factor of the artificial reality systems. Since artificial reality systems, such as an augmented reality system, may be used to interact with a real-world environment, the artificial reality systems may need to be embodied as a portable computing system for a user to interact with different environments. As an example and not by way of limitation, the artificial reality system may be a head-mounted display (HMD) that a user may wear. In order for an artificial reality system to interact with a real-world environment, the artificial reality system may need to capture images (with one or more cameras coupled to or included in the artificial reality system) of the real-world environment to build a three-dimensional (3D) model of a scene, determine a pose of a headset, and the like. However, the process of capturing images, and more specifically sensor captures and reads may be power intensive. This may especially be the case for always-on devices. While some processes may need a full image to function properly, the process of determining a pose of the artificial reality system (e.g., a headset) may need only a small percentage of an image captured to localize the headset. As an example and not by way of limitation, 10% of the pixels of an image may be needed for localizing a headset.

In order to reduce the power used for a process of determining the pose of an artificial reality system (e.g., headset), the artificial reality system may use a visual inertial odometry (VIO). The VIO may determine the pose of an artificial reality system by tracking environmental features observed by the artificial reality system. As an example and not by way of limitation, the VIO may determine the pose of an augmented reality headset by tracking a table within an environment. For a given series of frames (comprising images) that are captured, an artificial reality system may predict the pose of the headset in a next frame in the series of frames. The predicted pose of the artificial reality system may be used to determine where features of interests would likely be within a field of view of the camera. The artificial reality system may instruct the camera sensor to only capture pixels near the predicted feature locations to generate a sparse image capture. The sparse image capture may then be used by VIO to determine the pose of the artificial reality system.

In particular embodiments, a computing system may comprise a VIO system. As an example and not by way of limitation, an artificial reality system may comprise a VIO system. In particular embodiments, the computing system may be embodied as an artificial reality system. In particular embodiments, the artificial reality system may be embodied as an augmented reality system or a virtual reality system. The artificial reality system may track a set of feature points within a series of frames through the VIO system using one or more cameras. The set of feature points may be with respect to a 3D environment. As an example and not by way of limitation, if a 3D scene comprises a living room with a desk, then a set of feature points may correspond to the corners of the desk. Each of the feature points may be associated with a 3D location within the 3D environment. The feature points may be indicative of a landmark in the environment. In particular embodiments, the artificial reality system may capture a plurality of frames containing a plurality of images using a camera. The artificial reality system may process a set of the plurality of images to identify the feature points in the environment. In particular embodiments, the VIO system may receive data from one or more sensors. The VIO system may use the data from the one or more sensors to localize the artificial reality system. As an example and not by way of limitation, the VIO system may use the data from sensors to determine a pose of the artificial reality system within a 3D scene. For instance, if a user is wearing the artificial reality system within the living room, the artificial reality system may determine where the user is with respect to objects in the living room using one or more of cameras, inertial measurement unit, accelerometer, motion sensors, and the like. In particular embodiments, the artificial reality system may identify features of interest that would be within a field of view of the camera. The features of interest may comprise edges of objects, corners of objects, and the like. As an example and not by way of limitation, the artificial reality system may identify a corner of a table as a feature of interest and track the corner of the table from one frame to a subsequent frame. The artificial reality system may store a set of 3D locations associated with the feature points in the 3D environment. The feature points in the 3D environment may be captured by one or more cameras of the artificial reality system at a camera pose. The camera pose may be a previous camera pose of the cameras. The artificial reality system may access the set of 3D locations stored on the artificial reality system. Although this disclosure describes tracking a set of feature points in a particular manner, this disclosure contemplates tracking a set of feature points in any suitable manner.

In particular embodiments, the artificial reality system may predict a location of where feature points are expected to appear in an image sensor. The artificial reality system may use the VIO system to track a set of feature points and predict the locations of where the set of feature points will appear in an image sensor. The feature points may be from previously observed feature points in previous frames. In particular embodiments, the artificial reality system may determine a predicted camera pose using the previous camera pose and motion measurements generated using a motion sensor associated with the camera of the artificial reality system. In particular embodiments, the motion sensor may include an inertial measurement unit (IMU) and the motion measurements may be data generated by the IMU. The predicted camera pose may be an IMU-based estimated pose. Although this disclosure describes predicting a location of where points are expected in a particular manner, this disclosure contemplates predicting a location of where points are expected in any suitable manner.

In particular embodiments, the artificial reality system may generate an occupancy grid corresponding to pixels of an image or image sensor. The occupancy grid may indicate which pixel or groups of pixels of an image would likely be occupied by projected feature points if the camera is to capture an image of the environment from its predicted pose. In particular embodiments, the artificial reality system may use the occupancy grid to organize where feature points are expected to appear in an image sensor. For instance, the artificial reality system may track a set of feature points (e.g., feature point with known 3D locations based on computations made in the previous frame) and project the set of feature points to the corresponding locations in the occupancy grid. In particular embodiments, the artificial reality system may project the set of 3D locations toward the predicted camera pose and onto a 2D image plane associated with the camera of the artificial reality system. In particular embodiments, the camera of the artificial reality system may comprise a camera model. The camera model may include one or more intrinsic camera parameters of the camera and one or more camera lens distortion characteristics. The artificial reality system may access the one or more intrinsic camera parameters and one or more camera lens distortion characteristics to project the 3D locations toward the predicted camera pose and onto the 2D image plane. In particular embodiments, the artificial reality system may generate the occupancy grid based on the projection of the set of 3D locations toward the predicted camera pose and onto the 2D image plane. Although this disclosure describes generating an occupancy grid in a particular manner, this disclosure contemplates generating an occupancy grid in any suitable manner.

In particular embodiments, the occupancy grid may be divided into grids. The grids of the occupancy grid may be occupied by the estimated projection points. The estimated projection points may correspond to where a set of feature points are estimated to appear in an image if the image is captured from a predicted camera pose in the next frame. In particular embodiments, the resolution or grid size of the occupancy grid may depend on a desired number of estimated projection points within each grid (e.g., the resolution or grid size may be set so that most grids have no more than n projection points). In particular embodiments, the division of the occupancy grid into grids may be based on the estimated projection points so that only a threshold number of estimated projection points are located in each grid. Some grids may be empty of estimated projection points.

In particular embodiments, the artificial reality system may generate a pixel activation map for activating particular pixel sensors of a camera and/or indicating which pixels captured by the camera are to be read out from the image buffer. In particular embodiments, the artificial reality system may generate the activation map based on the projected set of 3D locations on the 2D image plane. The artificial reality system may use the occupancy grid to generate the pixel activation map. The pixel activation map may indicate which pixels of a camera to activate and/or which pixel values are to be read. The artificial reality system may use the VIO system to generate the pixel activation map for the camera. In particular embodiments, the artificial reality system may determine which pixels of an image sensor are of interest. The pixels of interest may be the pixels of the camera that the artificial reality system would activate and/or read out. Although this disclosure contemplates generating a pixel activation map in a particular manner, this disclosure contemplates generating a pixel activation map in any suitable manner.

In particular embodiments, the pixel activation map may include pixels that need to be searched to find the previously observed feature points. In particular embodiments, the artificial reality system may compute epipolar lines based on the location of where a feature point of interest appeared in the last frame, the pose of the camera in the last frame, and the predicted pose of the camera in the current frame. The artificial reality system may generate the epipolar lines based on the projection of the set of 3D locations toward the predicted camera pose and onto the 2D image plane. The epipolar lines may reduce the search space for the points of interest. The computed epipolar lines may be expanded to define an epipolar search space. In particular embodiments, the epipolar search space may be represented by small dots (if the line is very short) and line segments. In particular embodiments, the artificial reality system may instruct the camera to activate a subset of pixel sensors to capture a new image of the environment by using the activation map. A subset of pixel sensors may be a fraction of the available pixel sensors of the camera. As an example and not by way of limitation, the subset of pixel sensors may be 50% of the pixel sensors of the camera. As such, the artificial reality system may instruct the camera to activate 50% of the pixel sensors of the camera and not activate the other 50% of the pixel sensors. The artificial reality system may read pixel values of the new image corresponding to the subset of pixel sensors activated by the camera. The artificial reality system may track the feature points in the environment based on the pixel values. Although this disclosure describes a pixel activation map in a particular manner, this disclosure contemplates a pixel activation map in any suitable manner.

The pixel activation map may additionally or alternatively include regions that should be searched to find new features of interest. In particular embodiments, the artificial reality system may use the occupancy grid to determine areas that are empty. The empty grids in the occupancy grid indicate that the corresponding regions in the to-be captured image lack features. As an example and not by way of limitation, the artificial reality system may identify areas in the occupancy grid greater than a threshold area that fails to contain a point of interest. For instance, after the artificial reality system projects feature points onto the occupancy grid, the artificial reality system may determine which grids lack any estimated projection points. In particular embodiments, the artificial reality system may analyze one or more unoccupied cells in the occupancy grid to determine whether to include the one or more unoccupied cells in the activation map. In particular embodiments, the artificial reality system may decide to search the pixels of the to-be-captured image corresponding to those empty grids or unoccupied cells in the occupancy grid. The determination or decision may be based on a predetermined framerate. As an example and not by way of limitation, if the camera framerate is 90 frames per second, then the artificial reality system may determine to include unoccupied cells in a tenth of the captured frames. The frequency at which unoccupied cells may be included may be lower than the camera framerate. The system may do so by including the pixels corresponding to those empty grids in the pixel activation map.

In particular embodiments, the artificial reality system may aggregate epipolar line search and an empty grid search to generate the pixel activation map. In particular embodiments, the pixel activation map may be sent to the camera of the artificial reality system. In particular embodiments, the camera may capture the corresponding pixels indicated by the pixel activation map, which may then be read out according to the pixel activation map. Pixels not activated by the pixel activation map will not be activated by the camera and will not be read out, thereby providing significant power savings without sacrificing the tracking quality of the VIO system. In particular embodiments, the artificial reality system may store the resulting read pixels to be processed. In particular embodiments, the resulting read pixels may be used to find known points (e.g., features points) and find new points for tracking and/or localization. Although this disclosure describes generating a pixel activation map for a camera in a particular manner, this disclosure contemplates generating a pixel activation map for a camera in any suitable manner.

Referring toFIG.1, a diagram of an artificial reality system localization process100is shown. The process100may illustrate the embodiments described herein. A computing system may perform the process100. As an example and not by way of limitation, an artificial reality system may perform the process100. The process100may begin with the system (e.g., an artificial reality system) receiving an input indicative of the points to track102. For instance, the points to track102may be the feature points as described herein. The system may use the VIO system to generate an IMU-based estimated pose104of the system. The IMU-based estimated pose104may be used with the points to track102as an input into a point tracker106. The point tracker106may include point projection108and line search110. The points projection process108may process the points to track102and the IMU-based estimated pose104to generate an occupancy grid112. The points projection process108may use approximate point location to update the occupancy grid112. The line search110may use both the IMU-based estimated pose104and the output of the points projection process108to compute the epipolar lines. The output of the points projection process108and the output of the line search110may be used to generate the occupancy grid112. The occupancy grid112may include occupied cells114and unoccupied cells116, where the occupied cells114include projected points118. The occupancy grid112may be used as an input to a point selector process120to generate the pixel activation map as described herein. The point selector process120may process the unoccupied cells116to determine whether to select one or more points. In particular embodiments, one or more processes of the localization process100may be performed in parallel. As an example and not by way of limitation, the line search110may be performed in parallel to the point selector120. The system may determine to perform one or more processes of the process100at a predetermined interval. The predetermined interval may be less than a framerate of a camera of the system that is continuously capturing images. As an example and not by way of limitation, if the system has a camera capturing 90 frames per second, the system may perform the point selector process120at 30 frames per second. The point selector120may identify areas of the occupancy grid112to explore. It may be beneficial to have a wide distribution of key points across the image space. As such, the system may identify areas that are too empty or lack points of interests or feature points. The system may periodically add areas to explore. For instance, for every 10 frames a region may be added to the activation map to explore.

Referring toFIG.2, an example process200of generating an activation map is shown. A computing system may perform the process200. As an example and not by way of limitation, an artificial reality system may perform the process200. The process200may begin with the system receiving an input indicative of points to track202. For instance, the points to track202may be the feature points as described herein. The feature points may be identified by processing a plurality of images as described herein. The system may use the VIO system to generate an IMU-based estimated pose204of the system. The IMU-based estimated pose204may be used for points projection206, generating a warp matrix210, and computing epipolar line segments212. The points projection206may process points to track202and IMU-based estimated pose204to update the occupancy grid208. The projected points from the points projection206may be sent to generate a warp matrix210. The warp matrix may be used to account for the lens distortion and the movement of the system. The warp matrix may be used to change the view from a captured image to another view. The warp matrix may be used to warp patches of previously-identified points (e.g., features of interest, feature points, and the like) in order to orient the patch to be included in an activation map that would select pixel sensors corresponding to the previously-identified points in the current frame. The output of the warp matrix may be used to compute epipolar line segments212. The epipolar lines may be added to the activation map as described herein. The processes202,206,210, and212may be done before pixels of the camera are exposed. The generation of the warp matrix210and the computation of the epipolar line segments212may be used for line search metadata generation. The processes214,218, and220may be done after pixels of the camera are exposed. For instance, after an image is captured using an activation map generated from the occupancy map and the epipolar line segments. The processes214,218, and220may also be used for line search computation. The process200may proceed with applying the warp matrix in the warp patch214after capturing the image using the activation map. After warping the patch214using the warp matrix, the process200may search the epipolar segments218using pixels in the current frame216. The search of the epipolar segments218may be for warped patches in the currently captured pixels. The subpixel refine220may be used after identifying a warped patch to search the warped patch at a finer level to identify feature points. The process200may also mathematically compute the most likely location of feature points by using the confidence scores as weights.

Referring toFIG.3, an artificial reality system localization process300is shown. In particular embodiments, a system may perform the localization process300as described herein. As an example and not by way of limitation, an artificial reality system may perform the localization process300. For example, an augmented reality system may perform the localization process300. The process300may begin with the system receiving a plurality of frames in step302. As an example and not by way of limitation, the system may receive a plurality of images corresponding to a scene. For instance an augmented reality system may capture images of a scene using one or more cameras. In step304, the system may identify a set of features of interest as described herein. As an example and not by way of limitation, the system may identify corners of tables captured within the images. In step306, the system may use a VIO system to track the set of feature points that are identified from the set of features of interest as described herein. In step308, the system may generate an occupancy grid as described herein. The system may use the VIO system to estimate where the feature points will likely be located in a current frame by projecting the 3D feature points to the 2D image space of a camera located at a predicted pose. The projected points may be used to generate an occupancy grid. In step310, the system may generate a pixel activation map as described herein. As an example and not by way of limitation, the system may use the occupancy grid and epipolar lines to generate a pixel activation map. In step312, the system may send the pixel activation map to the camera. If the system has additional cameras, additional pixel activation maps may be generated using the process described herein. In step314, the system may activate pixels of the one or more cameras based on the pixel activation map. In step316, the system may read out the activated pixels of the cameras. The system may store the data gathered from step316and use the tracked features to perform localization and/or object tracking.

FIG.4illustrates an example method400for tracking features in an environment. The method400may begin at step410, where a computing system (e.g., artificial reality system) may access a set of 3D locations associated with features in an environment previously captured by a camera from a previous camera pose. At step420, the computing system may determine a predicted camera pose using the previous camera pose and motion measurements generated using a motion sensor associated with the camera. At step430, the computing system may project the set of 3D locations toward the predicted camera pose and onto a 2D image plane associated with the camera. At step440, the computing system may generate, based on the projected set of 3D locations on the 2D image plane, an activation map specifying a subset of the pixel sensors of the camera that are to be activated. At step450, the computing system may instruct, using the activation map, the camera to activate the subset of pixel sensors to capture a new image of the environment. At step460, the computing system may read pixel values of the new image corresponding to the subset of pixel sensors activated by the camera. At step470, the computing system may track the features in the environment based on the pixel values. Particular embodiments may repeat one or more steps of the method ofFIG.4, where appropriate. Although this disclosure describes and illustrates particular steps of the method ofFIG.4as occurring in a particular order, this disclosure contemplates any suitable steps of the method ofFIG.4occurring in any suitable order. Moreover, although this disclosure describes and illustrates an example method for tracking features in an environment, including the particular steps of the method ofFIG.4, this disclosure contemplates any suitable method of tracking features in an environment, including any suitable steps, which may include all, some, or none of the steps of the method ofFIG.4, where appropriate. Furthermore, although this disclosure describes and illustrates particular components, devices, or systems carrying out particular steps of the method ofFIG.4, this disclosure contemplates any suitable combination of any suitable components, devices, or systems carrying out any suitable steps of the method ofFIG.4.

FIG.5illustrates an example artificial reality system500. In particular embodiments, the artificial reality system500may comprise a headset504, a controller506, and a computing system508. A user502may wear the headset504that may display visual artificial reality content to the user502. The headset504may include an audio device that may provide audio artificial reality content to the user502. As an example and not by way of limitation, the headset504may display visual artificial content and audio artificial reality content corresponding to a virtual meeting. The headset504may include one or more cameras which can capture images and videos of environments. The headset504may include a plurality of sensors to determine a head pose of the user502. As an example and not by way of limitation, the sensors may include one or more of an accelerometer, inertial measurement unit (IMU), and the like. The headset504may include a microphone to receive audio input from the user502. The headset504may be referred as a head-mounted display (HMD). The controller506may comprise a trackpad and one or more buttons. The controller506may receive inputs from the user502and relay the inputs to the computing system508. The controller506may also provide haptic feedback to the user502. The computing system508may be connected to the headset504and the controller506through cables or wireless connections. The computing system508may control the headset504and the controller506to provide the artificial reality content to and receive inputs from the user502. The computing system508may be a standalone host computer system, an on-board computer system integrated with the headset504, a mobile device, or any other hardware platform capable of providing artificial reality content to and receiving inputs from the user502.

FIG.6illustrates an example computer system600. In particular embodiments, one or more computer systems600perform one or more steps of one or more methods described or illustrated herein. In particular embodiments, one or more computer systems600provide functionality described or illustrated herein. In particular embodiments, software running on one or more computer systems600performs one or more steps of one or more methods described or illustrated herein or provides functionality described or illustrated herein. Particular embodiments include one or more portions of one or more computer systems600. Herein, reference to a computer system may encompass a computing device, and vice versa, where appropriate. Moreover, reference to a computer system may encompass one or more computer systems, where appropriate.

This disclosure contemplates any suitable number of computer systems600. This disclosure contemplates computer system600taking any suitable physical form. As example and not by way of limitation, computer system600may be an embedded computer system, a system-on-chip (SOC), a single-board computer system (SBC) (such as, for example, a computer-on-module (COM) or system-on-module (SOM)), a desktop computer system, a laptop or notebook computer system, an interactive kiosk, a mainframe, a mesh of computer systems, a mobile telephone, a personal digital assistant (PDA), a server, a tablet computer system, or a combination of two or more of these. Where appropriate, computer system600may include one or more computer systems600; be unitary or distributed; span multiple locations; span multiple machines; span multiple data centers; or reside in a cloud, which may include one or more cloud components in one or more networks. Where appropriate, one or more computer systems600may perform without substantial spatial or temporal limitation one or more steps of one or more methods described or illustrated herein. As an example and not by way of limitation, one or more computer systems600may perform in real time or in batch mode one or more steps of one or more methods described or illustrated herein. One or more computer systems600may perform at different times or at different locations one or more steps of one or more methods described or illustrated herein, where appropriate.

In particular embodiments, computer system600includes a processor602, memory604, storage606, an input/output (I/O) interface608, a communication interface610, and a bus612. Although this disclosure describes and illustrates a particular computer system having a particular number of particular components in a particular arrangement, this disclosure contemplates any suitable computer system having any suitable number of any suitable components in any suitable arrangement.

In particular embodiments, processor602includes hardware for executing instructions, such as those making up a computer program. As an example and not by way of limitation, to execute instructions, processor602may retrieve (or fetch) the instructions from an internal register, an internal cache, memory604, or storage606; decode and execute them; and then write one or more results to an internal register, an internal cache, memory604, or storage606. In particular embodiments, processor602may include one or more internal caches for data, instructions, or addresses. This disclosure contemplates processor602including any suitable number of any suitable internal caches, where appropriate. As an example and not by way of limitation, processor602may include one or more instruction caches, one or more data caches, and one or more translation lookaside buffers (TLBs). Instructions in the instruction caches may be copies of instructions in memory604or storage606, and the instruction caches may speed up retrieval of those instructions by processor602. Data in the data caches may be copies of data in memory604or storage606for instructions executing at processor602to operate on; the results of previous instructions executed at processor602for access by subsequent instructions executing at processor602or for writing to memory604or storage606; or other suitable data. The data caches may speed up read or write operations by processor602. The TLBs may speed up virtual-address translation for processor602. In particular embodiments, processor602may include one or more internal registers for data, instructions, or addresses. This disclosure contemplates processor602including any suitable number of any suitable internal registers, where appropriate. Where appropriate, processor602may include one or more arithmetic logic units (ALUs); be a multi-core processor; or include one or more processors602. Although this disclosure describes and illustrates a particular processor, this disclosure contemplates any suitable processor.

In particular embodiments, memory604includes main memory for storing instructions for processor602to execute or data for processor602to operate on. As an example and not by way of limitation, computer system600may load instructions from storage606or another source (such as, for example, another computer system600) to memory604. Processor602may then load the instructions from memory604to an internal register or internal cache. To execute the instructions, processor602may retrieve the instructions from the internal register or internal cache and decode them. During or after execution of the instructions, processor602may write one or more results (which may be intermediate or final results) to the internal register or internal cache. Processor602may then write one or more of those results to memory604. In particular embodiments, processor602executes only instructions in one or more internal registers or internal caches or in memory604(as opposed to storage606or elsewhere) and operates only on data in one or more internal registers or internal caches or in memory604(as opposed to storage606or elsewhere). One or more memory buses (which may each include an address bus and a data bus) may couple processor602to memory604. Bus612may include one or more memory buses, as described below. In particular embodiments, one or more memory management units (MMUs) reside between processor602and memory604and facilitate accesses to memory604requested by processor602. In particular embodiments, memory604includes random access memory (RAM). This RAM may be volatile memory, where appropriate. Where appropriate, this RAM may be dynamic RAM (DRAM) or static RAM (SRAM). Moreover, where appropriate, this RAM may be single-ported or multi-ported RAM. This disclosure contemplates any suitable RAM. Memory604may include one or more memories604, where appropriate. Although this disclosure describes and illustrates particular memory, this disclosure contemplates any suitable memory.

In particular embodiments, storage606includes mass storage for data or instructions. As an example and not by way of limitation, storage606may include a hard disk drive (HDD), a floppy disk drive, flash memory, an optical disc, a magneto-optical disc, magnetic tape, or a Universal Serial Bus (USB) drive or a combination of two or more of these. Storage606may include removable or non-removable (or fixed) media, where appropriate. Storage606may be internal or external to computer system600, where appropriate. In particular embodiments, storage606is non-volatile, solid-state memory. In particular embodiments, storage606includes read-only memory (ROM). Where appropriate, this ROM may be mask-programmed ROM, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), electrically alterable ROM (EAROM), or flash memory or a combination of two or more of these. This disclosure contemplates mass storage606taking any suitable physical form. Storage606may include one or more storage control units facilitating communication between processor602and storage606, where appropriate. Where appropriate, storage606may include one or more storages606. Although this disclosure describes and illustrates particular storage, this disclosure contemplates any suitable storage.

In particular embodiments, I/O interface608includes hardware, software, or both, providing one or more interfaces for communication between computer system600and one or more I/O devices. Computer system600may include one or more of these I/O devices, where appropriate. One or more of these I/O devices may enable communication between a person and computer system600. As an example and not by way of limitation, an I/O device may include a keyboard, keypad, microphone, monitor, mouse, printer, scanner, speaker, still camera, stylus, tablet, touch screen, trackball, video camera, another suitable I/O device or a combination of two or more of these. An I/O device may include one or more sensors. This disclosure contemplates any suitable I/O devices and any suitable I/O interfaces608for them. Where appropriate, I/O interface608may include one or more device or software drivers enabling processor602to drive one or more of these I/O devices. I/O interface608may include one or more I/O interfaces608, where appropriate. Although this disclosure describes and illustrates a particular I/O interface, this disclosure contemplates any suitable I/O interface.

In particular embodiments, communication interface610includes hardware, software, or both providing one or more interfaces for communication (such as, for example, packet-based communication) between computer system600and one or more other computer systems600or one or more networks. As an example and not by way of limitation, communication interface610may include a network interface controller (NIC) or network adapter for communicating with an Ethernet or other wire-based network or a wireless NIC (WNIC) or wireless adapter for communicating with a wireless network, such as a WI-FI network. This disclosure contemplates any suitable network and any suitable communication interface610for it. As an example and not by way of limitation, computer system600may communicate with an ad hoc network, a personal area network (PAN), a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), or one or more portions of the Internet or a combination of two or more of these. One or more portions of one or more of these networks may be wired or wireless. As an example, computer system600may communicate with a wireless PAN (WPAN) (such as, for example, a BLUETOOTH WPAN), a WI-FI network, a WI-MAX network, a cellular telephone network (such as, for example, a Global System for Mobile Communications (GSM) network), or other suitable wireless network or a combination of two or more of these. Computer system600may include any suitable communication interface610for any of these networks, where appropriate. Communication interface610may include one or more communication interfaces610, where appropriate. Although this disclosure describes and illustrates a particular communication interface, this disclosure contemplates any suitable communication interface.

In particular embodiments, bus612includes hardware, software, or both coupling components of computer system600to each other. As an example and not by way of limitation, bus612may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a front-side bus (FSB), a HYPERTRANSPORT (HT) interconnect, an Industry Standard Architecture (ISA) bus, an INFINIBAND interconnect, a low-pin-count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCIe) bus, a serial advanced technology attachment (SATA) bus, a Video Electronics Standards Association local (VLB) bus, or another suitable bus or a combination of two or more of these. Bus612may include one or more buses612, where appropriate. Although this disclosure describes and illustrates a particular bus, this disclosure contemplates any suitable bus or interconnect.

Herein, a computer-readable non-transitory storage medium or media may include one or more semiconductor-based or other integrated circuits (ICs) (such, as for example, field-programmable gate arrays (FPGAs) or application-specific ICs (ASICs)), hard disk drives (HDDs), hybrid hard drives (HHDs), optical discs, optical disc drives (ODDs), magneto-optical discs, magneto-optical drives, floppy diskettes, floppy disk drives (FDDs), magnetic tapes, solid-state drives (SSDs), RAM-drives, SECURE DIGITAL cards or drives, any other suitable computer-readable non-transitory storage media, or any suitable combination of two or more of these, where appropriate. A computer-readable non-transitory storage medium may be volatile, non-volatile, or a combination of volatile and non-volatile, where appropriate.

Herein, “or” is inclusive and not exclusive, unless expressly indicated otherwise or indicated otherwise by context. Therefore, herein, “A or B” means “A, B, or both,” unless expressly indicated otherwise or indicated otherwise by context. Moreover, “and” is both joint and several, unless expressly indicated otherwise or indicated otherwise by context. Therefore, herein, “A and B” means “A and B, jointly or severally,” unless expressly indicated otherwise or indicated otherwise by context.

The scope of this disclosure encompasses all changes, substitutions, variations, alterations, and modifications to the example embodiments described or illustrated herein that a person having ordinary skill in the art would comprehend. The scope of this disclosure is not limited to the example embodiments described or illustrated herein. Moreover, although this disclosure describes and illustrates respective embodiments herein as including particular components, elements, feature, functions, operations, or steps, any of these embodiments may include any combination or permutation of any of the components, elements, features, functions, operations, or steps described or illustrated anywhere herein that a person having ordinary skill in the art would comprehend. Furthermore, reference in the appended claims to an apparatus or system or a component of an apparatus or system being adapted to, arranged to, capable of, configured to, enabled to, operable to, or operative to perform a particular function encompasses that apparatus, system, component, whether or not it or that particular function is activated, turned on, or unlocked, as long as that apparatus, system, or component is so adapted, arranged, capable, configured, enabled, operable, or operative. Additionally, although this disclosure describes or illustrates particular embodiments as providing particular advantages, particular embodiments may provide none, some, or all of these advantages.