PATENT DOCUMENT

Publication Number: US-10885883-B2
Application Number: US-201816477155-A
Country: US
Kind Code: B2

Title: Electronic device with foveated display system

Abstract:
An electronic device may have a display. A gaze detection system may gather information on a user&#39;s point of gaze on the display. Based on the point-of-gaze information, control circuitry in the electronic device may produce image data for an image with areas of different resolutions. A full-resolution portion of the image may overlap the point of gaze. Lower resolution portions of the image may surround the full-resolution portion. The display may have a pixel array. The pixel array may include rows and columns of pixels. Data lines may be used to supply data to the columns of pixels in accordance with row selection signals supplied to the rows of pixels. Display driver circuitry may be used to display the image using the pixel array. The display driver circuitry may have row selection circuitry and column expander circuitry that are responsive to a resolution mode selection signal.

Claims:
What is claimed is: 
     
       1. An electronic device, comprising:
 a gaze tracking system configured to produce point of gaze information; 
 control circuitry configured to produce image data associated with regions of an image of different resolutions based on the point of gaze information; 
 a pixel array on which the image is displayed, wherein the image includes a full-resolution region having a first resolution overlapping a point of gaze identified in the point of gaze information, a first lower-resolution region having a second resolution that is lower than the first resolution, and a second lower-resolution region having a third resolution that is lower than the second resolution; and 
 display driver circuitry configured to receive the image data from the control circuitry and configured to use the pixel array to display the image, wherein the display driver circuitry comprises:
 row selection circuitry configured to supply row selection signals to rows of pixels in the pixel array; and 
 column expander circuitry coupled to data lines in the pixel array, wherein the column expander circuitry comprises a first buffer and a second buffer, wherein the first buffer receives the image data on a first number of first buffer input lines and provides output signal to a second number of second buffer input lines, wherein the second number is larger than the first number, wherein the second buffer receives the output signal on the second number of second buffer input lines and provides data to a third number of pixel array data lines, wherein the third number is larger than the second number, wherein the first buffer comprises multiplexer circuitry that is configured to receive a resolution mode selection signal, wherein the first buffer is a bus-line buffer comprising a bus-line register that receives multiplexer output signals from the multiplexer circuitry, wherein the bus-line buffer comprises routing paths that distribute signals from the first buffer input lines to the multiplexer circuitry, wherein the bus-line register has outputs that are coupled to the second buffer input lines, and wherein the third number is at least 10 times larger than the second number. 
 
 
     
     
       2. The electronic device defined in  claim 1  wherein the bus-line register comprises:
 multiple sets of registers; and 
 multiplexers having outputs coupled to inputs of at least some of the registers. 
 
     
     
       3. The electronic device defined in  claim 2  wherein the multiplexers of the bus-line register have at least some inputs that receive the multiplexer output signals from the multiplexer circuitry. 
     
     
       4. The electronic device defined in  claim 3  wherein the multiplexers of the bus-line register have at least some inputs that receive outputs from at least some of the multiplexers in the bus-line register. 
     
     
       5. The electronic device defined in  claim 4  wherein the display driver circuitry comprises display controller circuitry configured to supply the resolution mode selection signal to the bus-line buffer. 
     
     
       6. The electronic device defined in  claim 5  wherein the multiplexer circuitry is configured to receive the resolution mode selection signal. 
     
     
       7. The electronic device defined in  claim 6  wherein the bus-line register is configured to receive the resolution mode selection signal. 
     
     
       8. The electronic device defined in  claim 7  wherein the multiplexers of the bus-line register are configured to receive the resolution mode selection signal and wherein the row selection circuitry is configured to receive the resolution mode selection signal. 
     
     
       9. The electronic device defined in  claim 1 , wherein the first number is equal to 8 and the second number is equal to 64. 
     
     
       10. An electronic device, comprising:
 a gaze tracking system configured to produce point of gaze information; 
 control circuitry configured to produce image data associated with regions of an image of different resolutions based on the point of gaze information; 
 a pixel array on which the image is displayed, wherein the image includes a full-resolution region overlapping a point of gaze identified in the point of gaze information and includes multiple lower-resolution regions; and 
 display driver circuitry configured to receive the image data from the control circuitry and configured to use the pixel array to display the image, wherein the display driver circuitry comprises:
 row selection circuitry configured to supply row selection signals to rows of pixels in the pixel array; and 
 column expander circuitry coupled to data lines in the pixel array, wherein the column expander circuitry comprises a first buffer and a second buffer, wherein the first buffer receives the image data on a first number of first buffer input lines and provides output signal to a second number of second buffer input lines, wherein the second number is larger than the first number, wherein the second buffer receives the output signal on the second number of second buffer input lines and provides data to a third number of pixel array data lines, wherein the third number is larger than the second number, and wherein the second buffer comprises a plurality of sets of registers, each set of registers containing the second number of registers, and each set of registers being configured to receive data from the second number of second buffer input lines when an associated enable signal for that set of registers is asserted. 
 
 
     
     
       11. A display, comprising:
 an array of pixels configured to display an image with regions of different resolutions; and 
 display driver circuitry that includes:
 first circuitry configured to supply row selection signals to rows of pixels in the pixel array based at least partly on a resolution mode selection signal; and 
 second circuitry configured to receive image data for the image, wherein the second circuitry is coupled to data lines in the pixel array and distributes the received image data to the data lines based at least partly on the resolution mode selection signal, wherein the second circuitry has a first buffer and a second buffer, wherein the first buffer is configured to expand a first number of data input lines that receive the image data into a second number of first buffer output lines, wherein the second buffer receives the image data from the second number of first buffer output lines and distributes the image data to a third number of the data lines in the pixel array, wherein the third number is greater than the second number, and wherein the first buffer includes multiplexer circuitry that routes different data from the data input lines to the first buffer output lines based on the resolution mode selection signal. 
 
 
     
     
       12. The display defined in  claim 11  further comprising display controller circuitry configured to provide an enable signal to the second buffer, wherein the enable signal controls which of the data lines in the pixel receives the image data from the first buffer output lines. 
     
     
       13. The display defined in  claim 12  wherein the second buffer comprises multiplexers configured to receive the image data from the first buffer output lines. 
     
     
       14. The display defined in  claim 13  wherein the second buffer comprises registers that receive output from the multiplexers.

Description:
This patent application claims priority to provisional patent application No. 62/450,223, filed on Jan. 25, 2017, which is hereby incorporated by reference herein in its entirety. 
    
    
     BACKGROUND 
     This relates generally to devices with displays, and, more particularly, to devices with foveated displays. 
     Displays may be incorporated into devices such as head-mounted devices and other equipment. It may be desirable to provide users with immersive content using the display. For example, it may be desirable for a user&#39;s entire field-of-view to be filled with content on a display. 
     If care is not taken, however, images viewed by a user will not be satisfactory. High-resolution images may require excessive bandwidth and may therefore be difficult or impossible to display effectively at satisfactory frame rates. The bandwidth used in conveying image data to a display may be reduced by lowering image resolution, but excessively reduced image resolution may degrade image quality more than desired. 
     SUMMARY 
     An electronic device such as a head-mounted display or other display system may have a display. A gaze detection system may gather information on a user&#39;s point of gaze on the display. Based on the point-of-gaze information, control circuitry in the electronic device may produce image data for an image with multiple resolutions. A full-resolution area of the image overlaps the point of gaze. Lower resolution image areas are located in peripheral regions of the image. 
     The display has a pixel array. Display driver circuitry may be used to display the image using the pixel array. The pixel array may include rows and columns of pixels. Data lines may be used to supply data to columns of pixels in accordance with row selection signals supplied to rows of pixels. 
     The display driver circuitry may have row selection circuitry for supplying the row selection signals to the pixel array and may have column expander circuitry for routing data to the data lines of the pixel array. The row selection circuitry and column expander circuitry may be responsive to a resolution mode selection signal. 
     The column expander circuitry may have a bus-line buffer that receives the image data from the control circuitry on a first number of data input lines. The bus-line buffer may provide the received image data to an expanded number of bus-line buffer output lines. The column expander circuitry may also have a line buffer that receives the image data from the bus-line buffer output lines. The line buffer may supply image data to a third number of data lines in the pixel array. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG. 1  is a schematic diagram of an illustrative electronic device such as a head-mounted display in accordance with an embodiment. 
         FIG. 2  is a diagram of an illustrative foveated display in accordance with an embodiment. 
         FIG. 3  is a flow chart of illustrative operations involved in operating an electronic device with a foveated display in accordance with an embodiment. 
         FIG. 4  is a diagram of an illustrative area of a display being loaded with image data in a full-resolution mode in accordance with an embodiment. 
         FIG. 5  is a diagram of an illustrative area of a display being loaded with reduced-resolution image data in accordance with an embodiment. 
         FIG. 6  is a diagram of a pixel array and associated display driver circuitry in accordance with an embodiment. 
         FIG. 7  is a diagram of illustrative column expander circuitry for a display driver circuit in accordance with an embodiment. 
         FIG. 8  is a diagram showing how data may be loaded depending on resolution operating mode in accordance with an embodiment. 
         FIG. 9  is a table showing how the inputs and outputs of the multiplexer circuitry of the bus-line buffer in the column expander circuitry of  FIG. 7  may be configured in accordance with an embodiment. 
         FIG. 10  is a circuit diagram of an illustrative bus-line register for the column expander circuitry of  FIG. 7  in accordance with an embodiment. 
         FIGS. 11 and 12  are diagrams showing how data may be loaded into the illustrative bus-line register of  FIG. 10  during operation in a half-resolution mode in accordance with an embodiment. 
         FIG. 13  is a circuit diagram of illustrative line buffer circuitry for the column expander circuitry of  FIG. 7  in accordance with an embodiment. 
         FIG. 14  is a diagram of illustrative row selection circuitry for display driver circuitry in accordance with an embodiment. 
     
    
    
     DETAILED DESCRIPTION 
     Head-mounted displays and other devices may be used for virtual reality and augmented reality systems. These devices may include portable electronic devices such as cellular telephones, tablet computers, and other portable equipment, head-up displays in cockpits, vehicles, and other environments, projectors, and other equipment. Device configurations in which virtual reality and/or augmented reality content is provided to a user with a head-mounted display may sometimes be described herein as an example. This is, however, merely illustrative. 
     A head-mounted display such as a pair of augmented reality glasses that is worn on the head of a user may be used to provide a user with computer-generated content that is overlaid on top of real-world content. The real-world content may be viewed directly by a user (e.g., by observing real-world objects through an optical coupler in a display system that merges light from real-world objects with light from a display). Configurations in which images of real-world objects are captured by a forward-facing camera and displayed for a user on a display may also be used. 
     In electronic devices such as head-mounted display devices, it may be desirable to display images for users over a wide angle of view. Displays that cover wide angles of view at high resolutions may consume relatively large amounts of image data and may therefore impose bandwidth burdens on electronic devices such as head-mounted displays. These bandwidth burdens may be reduced by using a display scheme in which high resolution images are displayed in alignment with the user&#39;s current point of gaze and in which low resolution images are displayed in the user&#39;s peripheral vision. Display schemes such as these may sometimes be referred to as foveated display schemes. 
     A schematic diagram of an illustrative head-mounted display or other electronic device of the type that may be provided with a foveated display arrangement is shown in  FIG. 1 . As shown in  FIG. 1 , electronic device  10  (e.g., a head-mounted display or other electronic device) may have control circuitry  50 . Control circuitry  50  may include storage and processing circuitry for controlling the operation of electronic device  10 . Circuitry  50  may include storage such as hard disk drive storage, nonvolatile memory (e.g., electrically-programmable-read-only memory configured to form a solid state drive), volatile memory (e.g., static or dynamic random-access-memory), etc. Processing circuitry in control circuitry  50  may be based on one or more microprocessors, microcontrollers, digital signal processors, baseband processors, power management units, audio chips, graphics processing units (e.g., graphics processing unit  22 ), application specific integrated circuits, and other integrated circuits. Software code may be stored on storage in circuitry  50  and run on processing circuitry in circuitry  50  to implement control operations for electronic device  10  (e.g., data gathering operations, operations involving the adjustment of components using control signals, etc.). 
     Electronic device  10  may include input-output circuitry  52 . Input-output circuitry  52  may be used to allow data to be received by electronic device  10  from external equipment (e.g., a tethered computer, a portable device such as a handheld device or laptop computer, or other electrical equipment) and to allow a user to provide electronic device  10  with user input. Input-output circuitry  52  may also be used to gather information on the environment in which electronic device  10  is operating. Output components in circuitry  52  may allow electronic device  10  to provide a user with output and may be used to communicate with external electrical equipment. 
     As shown in  FIG. 1 , input-output circuitry  52  may include a display such as display  26 . Display  26  may be used to display images for a user of a head-mounted display or other electronic device. Display  26  may, if desired, be incorporated into an optical coupler system that allows a user to observe real-world objects while computer-generated content is overlaid on top of the real-world objects by presenting computer-generated images on the display. A system of this type may be formed form a transparent pixel array (e.g., a transparent organic light-emitting diode display panel) or may be a formed by a display that provides images to a user through a beam splitter, holographic coupler, or other optical coupler (e.g., a display device such as a liquid crystal on silicon display). 
     Display  26  may have a pixel array such as pixel array  30  with pixels  32 . Display driver circuitry  28  may receive image data from control circuitry  50  (e.g., images that have been rendered using graphics processing unit  22 ) and may use pixel array  30  to display the images. Display driver circuitry  28  may, for example, supply image data to columns of pixels  32  in pixel array  30  over respective data lines (e.g., data lines that run vertically through array  30  so that each data line supplies image data to a corresponding column of pixels  32 ) and may supply gate line signals (sometimes referred to as horizontal control signals, row selection signals, or control signals) to rows of pixels  32 . When a given row selection signal is asserted, data may be loaded into the pixels  32  of that row from the data lines coupled to the pixels of that row. 
     In full-resolution image regions, each pixel  32  may be loaded with an individual bit of image data. In reduced-resolution image regions, image data bits may be expanded to cover multiple pixels. Data expansion may take place horizontally using column expander circuitry and vertically using row selection circuitry to simultaneously address multiple rows. 
     In general, display  26  may be any suitable type of display (e.g., a liquid crystal-on-silicon display, a light-emitting diode display in which pixels  32  are formed from crystalline semiconductor laser dies or organic light-emitting diodes, a liquid crystal display, a plasma display, a microelectromechanical systems display, or other suitable display). With one illustrative configuration, pixel array  30  is a liquid-crystal-on-silicon pixel array formed on a silicon substrate. Display driver circuitry  28  may be formed on the same silicon substrate or may be formed using one or more other silicon substrates. 
     Input-output circuitry  52  may include components such as input-output devices  60  for gathering data and user input and for supplying a user with output. Devices  60  may include cameras and other components that form part of gaze tracking system  62 . The camera(s) or other components of system  62  may face a user&#39;s eyes and may track the user&#39;s gaze (e.g., images and other information captured by system  62  may be analyzed by control circuitry  50 , circuitry  52 , and/or circuitry in system  62  to determine the direction in which the user&#39;s eyes are oriented). This gaze information obtained by system  62  may be used to determine the location on pixel array  30  where the user&#39;s eyes are directed (sometimes referred to as the point of gaze of the user). If desired, system  62  may also gather information on the focus of the user&#39;s eyes and other information such as eye movement information. System  62  may sometimes be referred to as a gaze detection system, eye tracking system, gaze tracking system, or eye monitoring system. If desired, image sensors other than cameras (e.g., infrared and/or visible light-emitting diodes and light detectors, etc.) may be used in monitoring a user&#39;s gaze in system  62 . 
     By determining the user&#39;s point of gaze, graphics processing unit  22  can expend processing effort on rendering the portion of the display where the point of gaze is located at full resolution, while rendering peripheral portions at one or more progressively lower resolutions. The portions of display  26  that are in a user&#39;s peripheral vision may be rendered with the lowest resolution and portions of display  26  that lie between the peripheral regions and the portion of display  26  that overlaps the user&#39;s point of gaze may be rendered with one or more intermediate levels of resolution. 
     During operation, control circuitry  50  and graphics processing unit  22  may obtain information on the location of the user&#39;s current point of gaze from gaze tracking system  62  and can render different portions of each image to be displayed accordingly. Images to be displayed on display  26  may, for example, be computer-generated content (e.g., augmented reality or virtual reality content from a game, navigation application, etc.). Before transmitting data for a given image to be displayed on display  26  from graphics processing unit  22  to display driver circuitry  28 , graphics processing unit  22  can obtain the current point of gaze of the user from system  62  and can, based on this gaze information, render portions of the image that are nearest to the point of gaze with a higher resolution than portions of the image that are farther from the point of gaze (e.g., graphics processing unit  22  may produce foveated image data for display  26  based on point-of-gaze information received from gaze tracking system  62 ). This reduces the amount of bandwidth required to transmit data for the image from graphics processing unit  22  to display driver circuitry  28  of display  26 . Once display driver circuitry  28  receives the foveated image data, display driver circuitry  28  can take appropriate action to display full resolution data in an appropriate full-resolution portion of pixel array  30  and to display one or more sets of lower resolution data in one or more respective lower-resolutions portions of pixel array  30 . 
     An illustrative example of a foveated image being displayed on pixel array  30  of display  26  is shown in  FIG. 2 . In the example of  FIG. 2 , display  26  has a rectangular outline. The size of display  26  may be selected to be sufficiently large to cover most or all of a user&#39;s field of view. Based on gaze tracking information from gaze tracking system  62 , graphics processing unit  22  may determine that a user&#39;s current point of gaze is located at point PG (e.g., in the upper right corner of display  26  in the example of  FIG. 2 ). Based on this location, graphics processing unit  22  may render image data in full resolution for full-resolution area x 1  of  FIG. 2  (an area that overlaps PG). Peripheral image data (e.g., image data for region x 8  of  FIG. 2 ) may be rendered at a reduced resolution (e.g., ⅛ of the full resolution). Intermediate areas that lie between full resolution area x 1  and reduced resolution area x 8  may be rendered at ¼ resolution (see, e.g., the x 4  area of display  26 ) and at ½ resolution (see, e.g., the x 2  area of display  26 ). In general, any suitable number of different resolutions may be used in rendering image data for display  26  in device  10 . The use of four different areas with four respective different resolutions in the example of  FIG. 2  is illustrative. 
     As the user follows visible content on display  26 , point-of-gaze location PG will shift to different regions on display  26 . Graphics processing unit  22  may use this information to adjust the locations of the high resolution and lower resolution areas for which image data is being rendered with different resolutions. To ensure that display driver circuitry  28  is informed of which resolution applies in each portion of display  26  for a given image, graphics processing unit  22  and/or control circuitry  50  may supply display driver circuitry  28  with information on the boundaries of regions x 8 , x 4 , x 2 , and x 1  (e.g., gaze tracking system information such as point of gaze PG or more processed information such as information on the boundary locations for regions x 8 , x 4 , x 2 , and x 1  that is derived from point of gaze PG). 
     Illustrative operations involved operating device  10  are shown in  FIG. 3 . 
     During the operations of block  70 , control circuitry  50  may use gaze tracking system  62  to gather information on the user&#39;s point-of-gaze. The point-of-gaze (see, e.g., point of gaze PG of  FIG. 2 ) may be located at a particular coordinate (X,Y) on display  26 . As the user&#39;s gaze moves (e.g., to track on-screen objects on display  26 ), the location of point of gaze PG can be updated dynamically. 
     A source of content (e.g., a computer program such as a game application or other application running on control circuitry  50 ) may produce content for viewing on display  26 . To avoid overburdening the circuitry of device  10 , graphics processing unit  22  may generate foveated display image data during the operations of block  72 . The foveated data includes high resolution (full resolution) data for a full-resolution region of display  26  (e.g., the x 1  region of  FIG. 2 ) and image data of progressively lower resolutions for lower resolution regions (e.g., regions x 2 , x 4 , and x 8  of  FIG. 2 ). By rendering images with different resolutions in different areas, the amount of image data that is associated with the content to be displayed on display  26  can be significantly reduced (e.g., in comparison to rendering the entire image on display  26  at full resolution). This lowers the rendering burden on graphics processing unit  22  and lowers the amount of data bus bandwidth consumed in transferring the rendered image data from graphics processing unit  22  to display driver circuitry  28  over one or more data busses between graphics processing unit  22  and display driver circuitry  28 . 
     During the operations of block  74 , display driver circuitry  28  can expand compressed data and can provide pixels  32  of pixel array  30  with corresponding image data values and control signals to recreate a desired image. Column expansion circuitry may be used to load image data to the data lines of pixel array  30  (e.g., column expansion circuitry may route data from a relative small number of data input lines to a larger number of data lines). Row selection circuitry may be used to assert row selection signals for rows of pixels in pixel array  30  and thereby cause data on the data lines to be loaded into the pixels of selected row(s). 
     The image displayed on pixel array  30  (display  26 ) by display driver circuitry  28  contains regions of different resolutions (e.g., regions x 1 , x 2 , x 4 , and x 8  of  FIG. 2 ). Because high resolution content is centered about point of gaze PG, where the user&#39;s visual acuity is highest, the image will appear to be sharp (of high resolution) to the user, regardless of where the current point of gaze of the user is located. Peripheral portions of the user&#39;s vision will contain lower resolution content, but because the user&#39;s vision is less sensitive in those regions, there will be little or no impact on the user&#39;s perception of the resolution and quality of the displayed content. 
       FIG. 4  is a diagram showing how display driver circuitry  28  may map incoming data to the pixels of an illustrative full-resolution portion  261 R of display  26  (e.g., region x 1  of  FIG. 2 ). Data may be received in 8-bit digital words (as an example). Incoming data words (e.g., a first word d 1 ′ . . . d 8 ′, a second word d 1  . . . d 8 , etc.) may be loaded into the pixels of each row of display region  26 FR in sequence as shown in  FIG. 4  so that each pixel receives a unique data bit in full-resolution region  26 FR. 
       FIG. 5  is a diagram showing how display driver circuitry  28  may map incoming data of lower resolution (e.g., data with ⅛ of full resolution) to the pixels of an illustrative reduced-resolution portion  26 RR of display  26  (e.g., a portion of region x 8  of  FIG. 2 ). As shown in  FIG. 5 , each incoming data word d 1 , . . . d 8  may be expanded, so that, for example, the first eight columns and first eight rows of region  26 RR are provided with data bit value d 1 , the second eight columns and first eight rows of region  26 RR are provided with data bit value d 2 , etc. Column expander circuitry can control the distribution of data bits horizontally (e.g., the distribution of a single bit b 1  into 8 columns in the x 8   FIG. 5  example). Row selection circuitry can control the distribution of data bits vertically (e.g., the distribution of a single bit b 1  into 8 rows of a given column in a x 8  configuration). 
     Illustrative display driver circuitry  28  and an illustrative pixel array  30  for display  26  are shown in  FIG. 6 . Pixel array  30  has an array (rows and columns) of pixels  32 . Data lines (col) of  FIG. 6  extend vertically through array  30 . There may be a single data line associated with each column of pixels  32  (as an example). Row selection lines (e.g., H lines in  FIG. 6 ) are coupled to the outputs of row selection circuitry  86  and receive row selection signals from circuitry  86 . Each row selection line extends horizontally through a respective row of pixels  32  in array  30  and controls data loading from the data lines (col) to those pixels  32 . 
     One or more data buses (serial and/or parallel) may be used to convey image data from an image data source in control circuitry  50  (e.g., graphics processing unit  22 ) to the “video-in” input of display controller  80 . The “eye tracker” input to controller  80  may receive gaze tracking system information (e.g., the user&#39;s current point of gaze) or other information that allows display driver circuitry  28  to distinguish lower and higher resolution areas of an image from each other. The gaze tracking information (e.g., point of gaze) may be received from system  62  and/or control circuitry  50 . The image data that is received at the video-in input has been foveated as described in connection with  FIGS. 2, 3, 4, and 5 , which reduces the bandwidth associated with conveying the image data across the data bus. 
     Display driver circuitry  28  may be implemented on a silicon substrate or other semiconductor substrate. If desired, pixel array  30  may be implemented on the same silicon substrate (e.g., in a configuration in which display  26  is a liquid-crystal-on-silicon display). Display driver circuitry  28  may include control circuitry such as finite state machine  82  and column decoder circuitry  84 . (Circuits such as circuits  82  and  84  may, if desired, be formed as part of display controller  80 .) 
     Finite state machine  82  may be used in providing control signals such as resolution mode selection signal sel and column block enable signal en to column expander circuitry. The selection signal sel may be used to place multiplexer circuitry in bus-line buffer  90  in different configurations depending on which resolution operating mode is desired. The column block enable signal en may be used to control a line buffer in bus-line buffer  90  (e.g., to select which block of columns in the line buffer is being loaded with data). The value of sel (in the present example), can be 00 (for operation in full resolution mode), 01 for operation in ½ resolution mode, 10, for operation in ¼ resolution mode, or 11, for operation in ⅛ resolution mode. 
     Pixel array  30  may contain columns of pixels  32  arranged in a number of column blocks (e.g., 5 or more column blocks, at least 10 column blocks, at least 20 column blocks, at least 40 column blocks, etc.). As an example, pixel array  30  may contain 2560 columns arranged in 40 column blocks. The rows of array  30  may be asserted individually (for full resolution areas) or may be asserted in sets of two or more (e.g., sets of 8 rows may be asserted at the same time when loading data for ⅛ resolution areas). Column expander circuitry  88  may use bus-line buffer  90  to receive m lanes of input image data (data-in) and may provide correspondingly expanded set of columns of output (e.g., 64 columns=m×n, where m and n are equal to 8 in the present example) to line buffer  92 . During operation, column decoder  84  may supply a signal en to line buffer  92  that informs line buffer  92  of which of the 40 column blocks of pixel array  30  is to receive the 64 columns of output of bus-line buffer  90 . Row selection circuitry such as row selection block  86  may receive control signals from finite state machine  82  and may generate corresponding gate lines signals (horizontal control signals H) to load selected rows of pixels in pixel array  30  with data from line buffer  92 . 
     Illustrative column expander circuitry  88  is shown in  FIG. 7 . In the example of  FIG. 7 , column expander circuitry  88  is used to distribute image data that is received on 8 bus-line buffer input lines  98  to 2560 columns (data lines) in pixel array  30 . Differently sized image data inputs and pixel arrays may be used, if desired. 
     As shown in  FIG. 7 , bus-line buffer input lines  98  may be provided with image data signals (e.g., bits d 1  . . . d 8 ) from display controller  80 . Zero line  96  may be provided with a zero data value. Lines  98  and  96  in bus-line buffer  90  may be routed to the inputs of multiplexer circuitry such as bus-line-buffer multiplexers  100 . Multiplexers  100  may supply their outputs t 1  . . . t 64  to bus-line register  102 . An illustrative pattern that may be used for the input and output connections of bus-line buffer multiplexers  100  is given by the entries of the table of  FIG. 9 . The leading “1” of the first eight entries of the table of  FIG. 9  indicate that bit d 1  is routed to the first input of each of the first 8 multiplexers  100 , etc. The corresponding bus-line-buffer multiplexer outputs tm for each of the 64 multiplexers  100  in buffer  90  are also specified by the rows of the table of  FIG. 9 . 
     Each multiplexer  100  may receive a control signal (resolution mode selection signal sel) from display controller circuitry such as finite state machine  82 . Bus-line register  102  may also receive the resolution mode selection signal. The resolution mode selection signal sel directs multiplexers  100  to route selected inputs (0, d 1 , . . . d 8 ) to each bus-line-buffer multiplexer output (tm) in accordance with the current resolution mode (x 1 , x 2 , x 4 , or x 8  in the present example). In bus-line register  102 , the signal sel controls the distribution of image data from multiplexer outputs t 1 -t 64  to bus-line-register outputs out 1 - 64 , which serve as the outputs of bus-line buffer  90 . Illustrative bus-line register circuitry  102  is shown in  FIG. 10 . Bus-line register  102  receives the outputs tm from multiplexer circuitry  100  and supplies corresponding output signals out 1  . . . out 64  to line buffer  92 . Bus-line register  102  provides image data (outputs out 1  . . . out 64 ) to line buffer  92  in groups of 64 (sometimes referred to as column blocks). Bus-line register  102  loads and shifts data at each clock pulse CLK. As shown in  FIG. 8 , in native (full) resolution mode (sel=00), 8 data bits are loaded at an initial clock cycle CLK and serve as the leading 8 values of outputs t 1  . . . t 64 . The remaining 64 outputs of register  102  are don&#39;t care outputs (e.g., these outputs may be set to zero as shown in  FIG. 8 ). In the next increment of clock CLK (sometimes referred to as a pixel clock), the next 8 bits are loaded, etc. Eight clock cycles are therefore needed to fully load bus-line register  102  in full-resolution mode. Progressively smaller numbers of clock cycles are used in filling bus-line register  102  in lower resolution modes. It takes 8 clock cycles to fill bus-line register  102  in full-resolution mode (sel=00), it takes 4 clock cycles to fill bus line register  102  in ½ resolution mode (sel=10), it takes 2 clock cycles to fill bus-line register  102  in ¼ resolution mode (sel=10), and it takes 1 clock cycle to fill bus-line register  102  in ⅛ resolution mode (sel=11). 
     Illustrative bus-line register circuitry  102  is shown in  FIG. 10 . Register circuitry  102  may be operated in multiple modes corresponding to different resolutions (e.g., full resolution x 1  mode, ½ resolution x 2  mode, ¼ resolution x 4  mode, and ⅛ resolution x 8  mode in the present example). As shown in  FIG. 10 , circuitry  102  may include an array of registers  108  and multiplexers  110 . Registers  108  may be controlled by clock signals CLK. Each multiplexer  110  may receive a respective sel signal at its control input. Registers  108  of  FIG. 10  have been arranged in 8 sets (columns)  104 , each with eight rows  106 . The outputs tm from multiplexer circuitry  100  of  FIG. 7  are supplied to the inputs of registers  108  in the first column  104  of circuitry  102  and to the inputs of multiplexers  110 . Outputs Q of registers  108  are used to supply 64 columns of output data out, to line buffer  92  of  FIG. 7 . 
     The sel signal applied to circuitry  102  controls the distribution of image data to outputs out 1  . . . out 64 . Consider, as an example, operation of circuitry  102  in x 2  mode. The contents of registers  108  in this operating mode are illustrated in  FIGS. 11 and 12 .  FIG. 11  shows operation of circuitry  102  on a first clock cycle. As shown in  FIG. 11 , during this first clock cycle, data bit d 1  is loaded into the first two registers  108  in the first column  104 , data bit d 2  is loaded into the second two registers  108  in the first column  104 , etc. and data bit d 5  is loaded into the first two registers  108  of the second column  104 , etc. Each data bit on one of lines  98  is expanded into two respective data bits in registers  108  due to the operation of circuitry  98  and  100  in the x 2  mode. In the second clock cycle, the data loaded into registers  108  is shifted to the right by two columns and a new set of data (d 1 ′ . . . d 8 ′) is loaded into the first two columns of registers  108  as shown in  FIG. 12 . This loading and shifting process continues until circuitry  102  is fully loaded with ½ resolution data. Operations in x 1  mode, x 4 , and x 8  are the same, with correspondingly less or more data expansion by circuitry  98  and  100  prior to loading the data into registers  108 . 
     Continuing with the x 2  expansion example,  FIG. 10  shows how the first column  104  in x 2  mode receives d 1 , d 1 , d 2 , d 2 , d 3 , d 3 , d 4 , and d 4  from outputs t 1 , t 2 , . . . t 8  of multiplexer circuitry  100 . The second column in x 2  mode receives data from outputs t 9  . . . t 16 . During x 2  mode, data d 5  is routed to the first two multiplexers  110  coupled to the first two registers  108  in the second column by via outputs t 9  and t 10 , respectively. As a result, d 5  is loaded into the first two registers  108  in the second column. This pattern repeats over all of circuitry  102 , thereby allowing data to be expanded in accordance with the current resolution operating mode (signal sel). 
     As an example, in x 1  mode, the first clock cycle is used to load  8  unique bits d 1 , . . . d 8  into the eight registers  108  of the first column  104 . Eight clock cycles are therefore used to shift all of the data into circuitry  102  and thereby establish outputs out 1  . . . out 64 . 
     As another example, in x 8  mode, a single clock cycle is used to load all 64 of registers  108  and thereby establish outputs out 1  . . . out 64 . During this clock cycle, data bit d 1  is loaded into the eight registers  108  in the first of columns  104 , data bit d 2  is loaded into the eight registers  108  in the second of columns  104 , . . . and data bit d 8  is loaded into the eight registers  108  in the eighth of columns  104 , thereby establishing outputs out 1  . . . out 8  are equal to d 1 , . . . , and outputs out 57  . . . out 64  are equal to d 8 . 
       FIG. 13  is a circuit diagram of illustrative line buffer circuitry  92  being used to distribute image data received from bus-line buffer outputs out 1  . . . out 64  (64 bit column blocks) to the columns (data lines) of pixel array  30  (col 1  . . . col 2560 ). As shown in  FIG. 13 , line buffer circuitry  92  may have routing lines  112 . Routing lines  112  serve as line buffer inputs and receive the outputs from bus-line buffer  90 . Routing lines  112  are coupled to line buffer registers  114  in a pattern that allows each column block to receive each of the 64 outputs out 1  . . . out 64 . Registers  114  are controlled by enable signals en 1  . . . en 40 . In this example, there are 40 column blocks with 64 registers  114  each, so each column block can be loaded by asserting an appropriate one of the 40 enable signals en 1  . . . en 40 . If desired, line buffer circuitry  92  may be implemented using static random-access cells where bit-line data is received from out 1  . . . out  64  while word-line select signals (en 1  . . . en 40 ) are received from display controller  80  (e.g., finite state machine  82  and column decoder logic  84 ). Logic  84  may generate the enable signals based on a 6-bit input from finite state machine  82 . 
     The circuitry of  FIG. 13  allows arbitrary column blocks to be addressed. This helps avoid sending repeated data into line buffer  92 . For example, when in a x 1  row, ⅛ resolution data need only be updated in the line buffer once every 8 lines, ¼ resolution data need by updated in the line buffer only once every 4 lines, and ½ resolution data need only be updated once every 2 lines, while naturally native resolution data may be updated into the line buffer on every line. 
     Illustrative row selection circuitry  86  is shown in  FIG. 14 . Row selection circuitry  86  may be formed from a set of row scan blocks  86 B that supply row selection signals to the rows of pixels  32  in pixel array  30 . The circuitry of an illustrative row selection block (illustrative block  86 B′) is shown in detail in  FIG. 14 . Block  86 ′ may include a chain of row selection registers  116  that receive data on an input (input in), received from the previous block in the chain of blocks  86 B forming circuitry  86 ) and that provide data on a corresponding output (output out, provided to the next block in the chain of blocks  86 B forming circuitry  86 ). Enable signals en of registers  116  may be selectively asserted by finite state machine  82  when line buffer  92  is ready to load a given row. Resolution mode selection signal sel may be used to configure multiplexers  118  for appropriate operation in each of the different resolution modes supported by display driver circuitry  28 . In x 1  mode (traces A of  FIG. 14 ), each row signal is asserted in a separate clock cycle, so that full resolution data may be loaded into each row separately. In x 2  mode (traces B of  FIG. 14 ), each clock cycle is used to load data into a respective pair of rows. In x 4  and x 8  modes, four rows and eight rows may respectively be loaded with data from the data lines in pixel array  30  on each clock cycle. 
     In accordance with an embodiment, an electronic device is provided that includes a gaze tracking system configured to produce point of gaze information, control circuitry configured to produce image data associated with regions of an image of different resolutions based on the point of gaze information, a pixel array on which the image is displayed, the image includes a full-resolution region overlapping a point of gaze identified in the point of gaze information and includes multiple lower-resolution regions, and display driver circuitry configured to receive the image data from the control circuitry and configured to use the pixel array to display the image, the display driver circuitry includes, row selection circuitry configured to supply row selection signals to rows of pixels in the pixel array, and column expander circuitry coupled to data lines in the pixel array, the column expander circuitry includes a first buffer and a second buffer, the first buffer receives the image data on a first number of first buffer input lines and provides output signal to a second number of second buffer input lines, the second number is larger than the first number, the second buffer receives the output signal on the second number of second buffer input lines and provides data to a third number of pixel array data lines, and the third number is larger than the second number. 
     In accordance with another embodiment, the first buffer includes multiplexer circuitry that is configured to receive a resolution mode selection signal. 
     In accordance with another embodiment, the first buffer is a bus-line buffer includes a bus-line register that receives multiplexer output signals from the multiplexer circuitry. 
     In accordance with another embodiment, the bus-line buffer includes routing paths that distribute signals from the first buffer input lines to the multiplexer circuitry. 
     In accordance with another embodiment, the bus-line register has outputs that are coupled to the second buffer input lines. 
     In accordance with another embodiment, the bus-line register includes multiple sets of registers, and multiplexers having outputs coupled to inputs of at least some of the registers. 
     In accordance with another embodiment, the multiplexers of the bus-line register have at least some inputs that receive the multiplexer output signals from the multiplexer circuitry. 
     In accordance with another embodiment, the multiplexers of the bus-line register have at least some inputs that receive outputs from at least some of the multiplexers in the bus-line register. 
     In accordance with another embodiment, the display driver circuitry includes display controller circuitry configured to supply the resolution mode selection signal to the bus-line buffer. 
     In accordance with another embodiment, the multiplexer circuitry is configured to receive the resolution mode selection signal. 
     In accordance with another embodiment, the bus-line register is configured to receive the resolution mode selection signal. 
     In accordance with another embodiment, the multiplexers of the bus-line register are configured to receive the resolution mode selection signal. 
     In accordance with another embodiment, the row selection circuitry is configured to receive the resolution mode selection signal. 
     In accordance with another embodiment, the third number is at least 10 times larger than the second number. 
     In accordance with another embodiment, the second buffer includes a plurality of sets of registers, each set of registers containing the second number of registers, and each set of registers being configured to receive data from the second number of second buffer input lines when an associated enable signal for that set of registers is asserted. 
     In accordance with an embodiment, a display is provided that includes an array of pixels configured to display an image with regions of different resolutions, and display driver circuitry that includes, first circuitry configured to supply row selection signals to rows of pixels in the pixel array based at least partly on a resolution mode selection signal, and second circuitry configured to receive image data for the image, the second circuitry is coupled to data lines in the pixel array and distributes the received image data to the data lines based at least partly on the resolution mode selection signal, the second circuitry has a first buffer and a second buffer, the first buffer is configured to expand a first number of data input lines that receive the image data into a second number of first buffer output lines, the second buffer receives the image data from the second number of first buffer output lines and distributes the image data to a third number of the data lines in the pixel array, the third number is greater than the second number. 
     In accordance with another embodiment, the electronic device includes display controller circuitry configured to provide an enable signal to the second buffer, where the enable signal controls which of the data lines in the pixel receives the image data from the first buffer output lines. 
     In accordance with another embodiment, the second buffer includes multiplexers configured to receive the image data from the first buffer output lines. 
     In accordance with another embodiment, the second buffer includes registers that receive output from the multiplexers. 
     In accordance with an embodiment, an electronic device is provided that includes a pixel array configured to display an image that has areas of different resolutions, a gaze tracking system configured to produce point of gaze information associated with a point of gaze on the pixel array, control circuitry configured to produce image data for the image based on the point of gaze information, and display driver circuitry including, first circuitry that is controlled by a resolution mode selection signal and that produces row selection signals for rows of pixels in the pixel array, and second circuitry coupled to data lines in the pixel array, the second circuitry includes a first buffer and a second buffer, the first buffer provides image data received from the control circuitry on a first number of data input lines to a second number of output lines based on the resolution mode selection signal, the second buffer receives the second number of output lines and provides image data to a third number of data lines in the pixel array based on an enable signal, the second number is greater than the first number, and the third number is greater than the second number. 
     The foregoing is merely illustrative and various modifications can be made to the described embodiments. The foregoing embodiments may be implemented individually or in any combination.

Metadata:
Filing Date: 20180118
Publication Date: 20210105
Grant Date: 20210105
Priority Date: 20170125
Inventors: KNEZ, IVAN
HUANG, CHUN-YAO
Assignee: APPLE INC
CPC Classifications: [{"code": "G09G2310/0291", "inventive": false, "first": false, "tree": "[]"}, {"code": "G09G2320/0261", "inventive": false, "first": false, "tree": "[]"}, {"code": "G09G2370/08", "inventive": false, "first": false, "tree": "[]"}, {"code": "G09G2354/00", "inventive": false, "first": false, "tree": "[]"}, {"code": "G09G5/391", "inventive": true, "first": false, "tree": "[]"}, {"code": "G09G2350/00", "inventive": false, "first": false, "tree": "[]"}, {"code": "G09G2310/0297", "inventive": false, "first": false, "tree": "[]"}, {"code": "G09G2340/0407", "inventive": false, "first": false, "tree": "[]"}, {"code": "G09G3/20", "inventive": true, "first": false, "tree": "[]"}, {"code": "G09G2354/00", "inventive": false, "first": false, "tree": "[]"}, {"code": "G09G2350/00", "inventive": false, "first": false, "tree": "[]"}, {"code": "G06F3/013", "inventive": true, "first": false, "tree": "[]"}, {"code": "G09G2340/0407", "inventive": false, "first": false, "tree": "[]"}, {"code": "G09G2310/027", "inventive": false, "first": false, "tree": "[]"}, {"code": "G09G2320/0261", "inventive": false, "first": false, "tree": "[]"}, {"code": "G09G3/2092", "inventive": true, "first": true, "tree": "[]"}, {"code": "G09G5/391", "inventive": true, "first": true, "tree": "[]"}, {"code": "G09G2310/0297", "inventive": false, "first": false, "tree": "[]"}, {"code": "G09G2360/08", "inventive": false, "first": false, "tree": "[]"}, {"code": "G09G2310/0291", "inventive": false, "first": false, "tree": "[]"}, {"code": "G09G2354/00", "inventive": false, "first": false, "tree": "[]"}, {"code": "G06F3/013", "inventive": true, "first": false, "tree": "[]"}, {"code": "G09G5/391", "inventive": true, "first": true, "tree": "[]"}, {"code": "G09G3/20", "inventive": true, "first": false, "tree": "[]"}, {"code": "G09G2310/0297", "inventive": false, "first": false, "tree": "[]"}, {"code": "G09G2340/0407", "inventive": false, "first": false, "tree": "[]"}, {"code": "G09G2350/00", "inventive": false, "first": false, "tree": "[]"}, {"code": "G09G2320/0261", "inventive": false, "first": false, "tree": "[]"}]
Family ID: 61157336