Driving display device with voltage compensation based on load estimation

Embodiments relate to estimating power consumption for displaying an image at a display device and sending a load signal indicating expected power consumption for displaying the image to the display device to enable the display device to adjust input voltage at its display integrated circuit (IC). The load signal may be received at a compensation circuit that generates and sends a control signal to a power IC in the display device so that the power IC adjusts its output voltage according to the control signal. In this way, the input voltage at the display IC is maintained relatively constant even when the power consumption changes to display different images.

BACKGROUND

Field of Technology

This disclosure relates generally to a head-mounted display (HMD), and more particularly, to compensating for drop of input voltage to the display device based on load for displaying an image on a display device of the HMD.

Discussion of the Related Art

A display device generally experiences different current load based on the images it displays. When high load images (e.g., bright images or static images) are displayed, the display panel and a display integrated circuit (IC) of the display device uses higher current compared to when low load images (e.g., darker images or dynamic images) are displayed. Depending on the current load at the display device, a voltage drop between the display IC and a power IC that provides power to the display IC also changes. The power IC of the display device is generally responsible for sensing the load at the display IC and controlling input current or voltage to the display IC. However, the power IC may not properly sense and adjust its current or voltage output appropriately, leading to flickering of images or displaying of degraded images on the display panel as the load changes.

SUMMARY

Embodiments relate to controlling an input voltage or an input current to a display integrated circuit (IC) based on expected load determined by a graphics processing unit (GPU). The GPU includes an image processing circuit that processes images for display, and a load estimation circuit that receives the processed image and estimates power consumption for displaying the processed image. The load estimation circuit generates and sends a load signal representing power estimated for displaying the processed images. The display device includes a display integrated circuit (IC) that receives the processed image from the GPU and generates signals for driving a display panel, a power IC that controls input voltage to the display IC, and a compensation circuit that receives the load signal from the load estimation circuit and sends a control signal to adjust the input voltage to the display IC to account for a voltage drop between the power IC and the display IC based on the load signal.

The figures depict various embodiments for purposes of illustration only. Alternative embodiments of the structures and methods illustrated herein may be employed without departing from the principles described herein.

DETAILED DESCRIPTION

Embodiments relate to estimating power consumption for displaying an image at a display device and sending a load signal indicating expected power consumption for displaying the image to the display device to enable the display device to adjust input voltage at its display integrated circuit (IC). The load signal may be received at a compensation circuit that generates and sends a control signal to a power IC in the display device so that the power IC adjusts its output voltage according to the control signal. In this way, the input voltage at the display IC is maintained relatively constant even when the power consumption changes to display different images.

FIG. 1is a diagram of a HMD100, in accordance with an embodiment. The HMD100may be a part of an artificial reality system. The HMD100includes a front rigid body105having a front side120A, top side120B, bottom side120C, right side120D, and left side120E, and a band110. In some embodiments, portions of a front side120A of the HMD100are at least partially transparent in the visible band (˜380 nm to 750 nm), and portions of the HMD100that are between the front side120A of the HMD100and an eye of the user are at least partially transparent (e.g., a partially transparent electronic display).

The front rigid body105includes one or more electronic displays (not shown inFIG. 1), an inertial measurement unit (IMU)130, one or more position sensors125, and one or more locators135. In the embodiment shown byFIG. 1, the position sensors125are located within the IMU130, and neither the IMU130nor the position sensors125are visible to the user.

The locators135may be located in fixed positions on the front rigid body105relative to one another and relative to a reference point115. In the example ofFIG. 1, the reference point115is located at the center of the IMU130. Each of the locators135may emit light that is detectable by an imaging device (e.g., an imaging device210illustrated inFIG. 2, described in greater detail below). In some embodiments, the locators135may comprise passive elements (e.g., a retroreflector) that reflect light from a light source that may be detectable by an imaging device. Locators135, or portions of locators135, are located on the front side120A, the top side120B, the bottom side120C, the right side120D, and/or the left side120E of the front rigid body105in the example ofFIG. 1. The imaging device may determine a position (includes orientation) of the HMD100based upon the detected locations of the locators135, which may be used to determine the content to be displayed to the user. For example, where the HMD100is part of a HMD system, the position of the HMD100may be used to determine which virtual objects positioned in different locations are visible to the user of the HMD100.

FIG. 2is a HMD system200in accordance with an embodiment. The system200may be for use as an artificial reality system. In this example, the system200includes a HMD205, an imaging device210, and an I/O interface215, which are each coupled to a console225. WhileFIG. 2shows a single HMD205, a single imaging device210, and a single I/O interface215, in other embodiments, any number of these components may be included in the system. For example, there may be multiple HMDs200each having an associated I/O interface215and being monitored by one or more imaging devices210, with each HMD205, I/O interface215, and imaging devices210communicating with the console225. In alternative configurations, different and/or additional components may also be included in the system200.

The HMD205may act as an artificial reality HMD. In some embodiments, an artificial reality HMD augments views of a physical, real-world environment with computer-generated elements (e.g., images, video, sound, etc.). The HMD205presents content to a user. In some embodiments, the HMD100is an embodiment of the HMD205. Example content includes images, video, audio, or some combination thereof. Audio content may be presented via a separate device (e.g., speakers and/or headphones) external to the HMD205that receives audio information from the HMD205, the console225, or both. The HMD205includes an electronic display230, an optics block232, one or more locators235, the position sensors125, the internal measurement unit (IMU)130, the eye tracking system238, and an optional varifocal module240. The HMD205further includes a graphics processing unit (GPU)234and a display device (not shown inFIG. 2). Operation of the GPU234and the display device is described below with reference toFIG. 3in detail.

The electronic display230displays 2D or 3D images to the user in accordance with data received from the console225. In various embodiments, the electronic display230comprises a single electronic display element or multiple electronic displays (e.g., a display for each eye of a user). Examples of the electronic display element include: a liquid crystal display (LCD), an organic light emitting diode (OLED) display, an inorganic light emitting diode (ILED) display, an active-matrix organic light-emitting diode (AMOLED) display, a transparent organic light emitting diode (TOLED) display, a waveguide display, some other display, or some combination thereof. In some embodiments, the electronic display230is driven by a display integrated circuit (IC). The display IC is described below with reference toFIG. 3in detail.

The optics block232magnifies image light received from the electronic display230, corrects optical errors associated with the image light, and presents the corrected image light to a user of the HMD205. The optics block232includes a plurality of optical elements. Example optical elements included in the optics block232include: an aperture, a Fresnel lens, a convex lens, a concave lens, a filter, a reflecting surface, a feature waveguide, or any other suitable optical element that affects image light. Moreover, the optics block232may include combinations of different optical elements. In some embodiments, one or more of the optical elements in the optics block232may have one or more coatings, such as partially reflective or anti-reflective coatings.

The locators235are objects located in specific positions on the HMD205relative to one another and relative to a specific reference point on the HMD205. The locators135are an embodiment of the locators235. A locator235may be a light emitting diode (LED), a corner cube reflector, a reflective marker, a type of light source that contrasts with an environment in which the HMD205operates, or some combination thereof. Active locators235(i.e., an LED or other type of light emitting device) may emit light in the visible band (˜380 nm to 750 nm), in the infrared (IR) band (˜440 nm to 1700 nm), in the ultraviolet band (10 nm to 380 nm), some other portion of the electromagnetic spectrum, or some combination thereof.

The locators235can be located beneath an outer surface of the HMD205, which is transparent to the wavelengths of light emitted or reflected by the locators235or is thin enough not to substantially attenuate the wavelengths of light emitted or reflected by the locators235. Further, the outer surface or other portions of the HMD205can be opaque in the visible band of wavelengths of light. Thus, the locators235may emit light in the IR band while under an outer surface of the HMD205that is transparent in the IR band but opaque in the visible band.

As described above with reference toFIG. 1, the IMU130is an electronic device that generates IMU data based on measurement signals received from one or more of the position sensors125, which generate one or more measurement signals in response to motion of HMD705. Examples of the position sensors125include accelerometers, gyroscopes, magnetometers, other sensors suitable for detecting motion, correcting error associated with the IMU130, or some combination thereof.

Based on the measurement signals from the position sensors125, the IMU130generates IMU data indicating an estimated position of the HMD205relative to an initial position of the HMD205. For example, the position sensors125include multiple accelerometers to measure translational motion (forward/back, up/down, left/right) and multiple gyroscopes to measure rotational motion (e.g., pitch, yaw, and roll). The IMU130can, for example, rapidly sample the measurement signals and calculate the estimated position of the HMD205from the sampled data. For example, the IMU130integrates measurement signals received from the accelerometers over time to estimate a velocity vector and integrates the velocity vector over time to determine an estimated position of a reference point on the HMD205. The reference point is a point that may be used to describe the position of the HMD205. While the reference point may generally be defined as a point in space, in various embodiments, a reference point is defined as a point within the HMD205(e.g., a center of the IMU130). Alternatively, the IMU130provides the sampled measurement signals to the console225, which determines the IMU data.

The IMU130can additionally receive one or more calibration parameters from the console225. As further discussed below, the one or more calibration parameters are used to maintain tracking of the HMD205. Based on a received calibration parameter, the IMU130may adjust one or more of the IMU parameters (e.g., sample rate). In some embodiments, certain calibration parameters cause the IMU130to update an initial position of the reference point to correspond to a next calibrated position of the reference point. Updating the initial position of the reference point as the next calibrated position of the reference point helps reduce accumulated error associated with determining the estimated position. The accumulated error, also referred to as drift error, causes the estimated position of the reference point to “drift” away from the actual position of the reference point over time.

The eye tracking system238determines eye tracking information associated with one or both eyes of a user wearing the HMD205. The eye tracking information determined by the eye tracking system238may comprise information about an orientation of the user's eye, i.e., information about an angle of an eye-gaze. The eye tracking system238includes a source assembly that illuminates one or both eyes of the user with a light pattern. A camera assembly captures images of the light pattern reflected by a portion of the eye(s) being tracked. At least one of the captured images includes a subset of the plurality of glints that are reflected by the boundary region. The eye tracking system238determines a position of the eye(s). The eye tracking system238then determines eye tracking information using the determined position(s). For example, given a position of an eye the eye tracking system238can determine a gaze angle.

In some embodiments, the varifocal module240is further integrated into the HMD205. The varifocal module240may be coupled to the eye tracking system238to obtain eye tracking information determined by the eye tracking system238. The varifocal module240may adjust focus of one or more images displayed on the electronic display230, based on the determined eye tracking information obtained from the eye tracking system238. In this way, the varifocal module240can mitigate vergence-accommodation conflict in relation to image light. The varifocal module240can be interfaced (e.g., either mechanically or electrically) with at least one of the electronic display230and at least one optical element of the optics block232. Then, the varifocal module240may adjust focus of the one or more images displayed on the electronic display230by adjusting position of at least one of the electronic display230and the at least one optical element of the optics block232, based on the determined eye tracking information obtained from the eye tracking system238. By adjusting the position, the varifocal module240varies focus of image light output from the electronic display230towards the user's eye. The varifocal module240may also adjust resolution of the images displayed on the electronic display230by performing foveated rendering of the displayed images, based at least in part on the determined eye tracking information obtained from the eye tracking system238. In this case, the varifocal module240provides appropriate image signals to the electronic display230. The varifocal module240provides image signals with a maximum pixel density for the electronic display230only in a foveal region of the user's eye-gaze, while providing image signals with lower pixel densities in other regions of the electronic display230.

The imaging device210generates image data in accordance with calibration parameters received from the console225. Image data includes one or more images showing observed positions of the locators235that are detectable by imaging device210. The imaging device210may include one or more cameras, one or more video cameras, other devices capable of capturing images including one or more locators235, or some combination thereof. Additionally, the imaging device210may include one or more filters (e.g., for increasing signal to noise ratio). The imaging device210detects light emitted or reflected from the locators235in a field of view of the imaging device210. In embodiments where the locators235include passive elements (e.g., a retroreflector), the imaging device210may include a light source that illuminates some or all of the locators235, which retro-reflect the light towards the light source in the imaging device210. Image data is communicated from the imaging device210to the console225, and the imaging device210receives one or more calibration parameters from the console225to adjust one or more imaging parameters (e.g., focal length, focus, frame rate, ISO, sensor temperature, shutter speed, aperture, etc.).

The I/O interface215is a device that allows a user to send action requests to the console225. An action request is a request to perform a particular action. For example, an action request may be to start or end an application or to perform a particular action within the application. The I/O interface215may include one or more input devices. Example input devices include a keyboard, a mouse, a game controller, or any other suitable device for receiving action requests and communicating the received action requests to the console225. An action request received by the I/O interface215is communicated to the console225, which performs an action corresponding to the action request. In some embodiments, the I/O interface215may provide haptic feedback to the user in accordance with instructions received from the console225. For example, haptic feedback is provided by the I/O interface215when an action request is received, or the console225communicates instructions to the I/O interface215causing the I/O interface215to generate haptic feedback when the console225performs an action.

The console225provides content to the HMD205for presentation to the user in accordance with information received from the imaging device210, the HMD205, or the I/O interface215. In the example shown inFIG. 2, the console225includes an application store245, a tracking module250, and an engine260. Some embodiments of the console225have different or additional modules than those described in conjunction withFIG. 2. Similarly, the functions further described below may be distributed among components of the console225in a different manner than is described here.

The application store245stores one or more applications for execution by the console225. An application is a group of instructions, that when executed by a processor, generates content for presentation to the user. Content generated by an application may be in response to inputs received from the user via movement of the HMD205or the I/O interface215. Examples of applications include gaming applications, conferencing applications, video playback application, or other suitable applications.

The tracking module250calibrates the system200using one or more calibration parameters and may adjust one or more calibration parameters to reduce error in determining position of the HMD205. For example, the tracking module250adjusts the focus of the imaging device210to obtain a more accurate position for observed locators235on the HMD205. Moreover, calibration performed by the tracking module250also accounts for information received from the IMU130. Additionally, if tracking of the HMD205is lost (e.g., imaging device210loses line of sight of at least a threshold number of locators235), the tracking module250re-calibrates some or all of the system200components.

Additionally, the tracking module250tracks the movement of the HMD205using image information from the imaging device210and determines positions of a reference point on the HMD205using observed locators from the image information and a model of the HMD205. The tracking module250also determines positions of the reference point on the HMD205using position information from the IMU information from the IMU215on the HMD205. Additionally, the tracking module250may use portions of the IMU information, the image information, or some combination thereof, to predict a future location of the HMD205, which is provided to the engine260.

The engine260executes applications within the system200and receives position information, acceleration information, velocity information, predicted future positions, or some combination thereof for the HMD205from the tracking module250. Based on the received information, the engine260determines content to provide to the HMD205for presentation to the user, such as a virtual scene, one or more virtual objects to overlay onto a real world scene, etc. Additionally, the engine260performs an action within an application executing on the console225in response to an action request received from the I/O interface215and provides feedback to the user that the action was performed. The provided feedback may be visual or audible feedback via the HMD205or haptic feedback via VR I/O interface215.

In some embodiments, based on the eye tracking information (e.g., orientation of the user's eye) received from the eye tracking system238, the engine260determines resolution of the content provided to the HMD205for presentation to the user on the electronic display230. The engine260provides the content to the HMD205having a maximum pixel resolution on the electronic display230in a foveal region of the user's gaze, whereas the engine260provides a lower pixel resolution in other regions of the electronic display230, thus achieving less power consumption at the HMD205and saving computing cycles of the console225without compromising a visual experience of the user. In some embodiments, the engine260can further use the eye tracking information to adjust where objects are displayed on the electronic display230to prevent vergence-accommodation conflict.

FIG. 3is a diagram illustrating a graphics processing unit (GPU)234and display device325of the HMD100, in accordance with an embodiment. The graphics processing unit (GPU)234and a display device325operably coupled to the GPU234may be part of the HMD100, as described above with reference toFIGS. 1 and 2.

The GPU234is a circuit that performs operation to efficiently generate images for output to the display device325. The GPU234also generates and provides a load signal314indicating expected power consumption for displaying the images output to the display device325. For this purpose, the GPU234may include, among other components, an image processing circuit315, a frame buffer370and a load estimation circuit320. The image processing circuit315includes circuit components (e.g., transistors) for performing at least one of asynchronous time warp (ATW) and asynchronous space warp (ASW) frame-rate smoothing techniques on image data received from CPU or system memory of HMD100. The images processed by the image processing circuit315is stored in a buffer frame312.

The load estimation circuit320is a circuit that generates a load signal314representing power estimated for displaying the processed images. The load estimation circuit320is coupled313to the frame buffer370to access the processed image stored in the frame buffer370. The load estimation circuit320performs computation to estimate the power consumption for displaying the processed image by analyzing overall brightness of the pixels in the processed image and/or the dynamic change of pixel values in a current image relative to pixel values in a previous image.

The load signal314may be set at several levels decided in part by a requirement of a driver integrated circuit (IC)350. In one embodiment, the load signal314indicates one of three values (e.g., high, middle, low) representing different levels of power estimates. That is, a load signal314set at a high level indicates heavy loading and a load signal set at a low level indicates light loading.

The display device325displays images311received from the GPU234. For this purpose, the display device325may include, among other components, a power IC340, a compensation power circuit330, display IC350, and a display panel360. The display IC is 350 coupled311to the GPU234(e.g., image processing circuit315of the GPU234) to receive the processed image. The display IC350generates signals for driving a display panel360. The signals for driving the display panel360include, for example, gate driving signals for turning on or off thin film transistors (TFT) in pixels of the display panel360and data line signals for controlling brightness of the pixels.

The compensation power circuit330generates a control signal326to adjust the input voltage to the display IC350to account for a voltage drop between the power IC340and the display IC350based on the load signal314from the load estimation circuit320of the GPU234. The compensation power circuit330increases its output voltage so that the input voltage to the display IC350is increased when current between the power IC340and display IC350is increased (i.e., the load signal314indicates a high value), and decreases its output voltage so that the input voltage to the display IC350is increased when the current between the power IC340and display IC350is decreased (i.e., the load signal314indicates a low value). In this way, the input voltage at the display IC350is maintained relatively constant even when the current load of the display IC350fluctuates.

The power IC340is coupled to the display IC350to control an input voltage to the display IC350. The power IC340may include a voltage regulator to provide a desired output voltage at its output.

For example, in a conventional HMD system, the power IC340provides a static voltage input of 1.8V to the display IC350. Suppose the wire327coupling the power IC340and the display IC350has an electrical resistance of 0.5 Ohms. As images displayed on the display device325changes from light current load to heavy current load, the current between the power IC340and the display IC350for displaying the images increases from 100 mA to 800 mA. Consequently, the voltage input to the display IC350decreases from 1.75V to 1.4V. Assuming that the display IC350requires input voltage of 1.8V±0.3V to properly operate, the drop in the voltage input to the display IC350is insufficient to properly drive the display panel360.

In contrast, the HMD system of the present disclosure calculates the current load for display images on the display device325in the load estimation circuit320of the GPU234and generates the load signal314indicating one of three values (e.g., high, medium, low) representing different power estimates to the compensation power circuit330of power IC340. The compensation power circuit330then sends the control signal326to cause the power IC340to dynamically change the input voltage to the display IC350. For example, as the images for displaying on the display device325changes from light current load to heavy current load, the load estimation circuit320changes the output voltage of the power IC340from 1.85V to 2.2V, so the input value into the display IC350remains at 1.8V (assuming the same resistance of the wire327and the current consumption as in the example of the conventional HMD system).

In some embodiments, the power compensation circuit330is a part of the power IC340. In alternative embodiments, the power compensation circuit330is separate from the power IC340. Moreover, one or more of the power compensation circuit330and the power IC340may be provided at the GPU234instead of the display device325.

The display panel360may be one of a light-emitting diode display (LED), a plasma display (PDP), a liquid crystal display (LCD), and an organic light-emitting diode display (OLED). The display panel360is an embodiment of the electronic display230ofFIG. 2.

Example Method of Computing Load for Displaying an Image

FIG. 4is a flowchart illustrating a method of computing load for displaying an image on the display device325of a HMD, in accordance with an embodiment. An image processing circuit processes400an image for display. Processing an image for display includes utilizing at least one of asynchronous time warp (ATW) and asynchronous space warp (ASW) frame-rate smoothing techniques.

A load estimation circuit receives410the processed image and estimates power consumption for displaying the processed image.

The load estimation circuit generates420a load signal representing power estimated for displaying the processed image. The load signal may indicate one of three values representing different power estimates.

A compensation power circuit in a display device receives430the load signal generated by the load estimation circuit.

The compensation power circuit sends440a control signal to a power integrated circuit (IC) to adjust an input voltage to a display IC to account for a voltage drop between the power IC and the display IC based on the load signal.

The power IC controls450the input voltage to the display IC according to the control signal. The power IC increases or decreases its output voltage so that the input voltage to the display IC remains relatively constant even if current between the power IC and the display IC is increased or decreased.

The display IC generates460signals for driving a display panel responsive to receiving the processed image from a graphics processing unit (GPU) and the input voltage from the power IC.

The foregoing description of the embodiments has been presented for the purpose of illustration; it is not intended to be exhaustive or to limit the patent rights to the precise forms disclosed. Many modifications and variations are possible in light of the above disclosure.