PATENT DOCUMENT

Publication Number: US-9727185-B2
Application Number: US-201514640958-A
Country: US
Kind Code: B2

Title: Dynamic artifact compensation systems and methods

Abstract:
One embodiment describes an electronic display. The electronic display includes display driver circuitry that displays at least a first image frame and a second image frame on the electronic device using a first display pixel and a second display pixel. The electronic display also includes touch sensing circuitry that detects user interaction with the electronic display. A timing controller of the electronic display determines at least a first insertion time for a first intra-frame pause for the first image frame and a second insertion time for a second intra-frame pause for the second image frame. The first and second intra-frame pauses are periods where the display driver circuitry is pauses rendering of image data to allow the touch sensing circuitry to detect user interaction. The insertion times for the first and second intra-frame pauses are varied from one another. The timing controller inserts the first intra-frame pause during rendering of the first image frame at the first insertion time and inserts the second intra-frame pause during rendering of the second image frame at the second insertion time.

Claims:
What is claimed is: 
     
       1. An electronic display, comprising:
 display driver circuitry configured to display at least a first image frame and a second image frame on the electronic display using a first display pixel and a second display pixel; 
 touch sensing circuitry configured to detect user interaction with the electronic display; and 
 a timing controller configured to:
 determine at least a first insertion time for a first intra-frame pause for the first image frame and a second insertion time for a second intra-frame pause for the second image frame, the insertion times for the first and second intra-frame pauses being varied from one another, wherein the first and second intra-frame pauses are periods where the display driver circuitry is configured to pause rendering of image data to allow the touch sensing circuitry to detect the user interaction; and 
 insert the first intra-frame pause during rendering of the first image frame at the first insertion time; and 
 insert the second intra-frame pause during rendering of the second image frame at the second insertion time; 
 wherein the timing controller is configured to determine the first insertion time, the second insertion time, or both randomly. 
 
 
     
     
       2. The electronic display of  claim 1 , comprising a random number generator, wherein the timing controller is configured to determine the first insertion time, the second insertion time, or both based at least in part upon an output of the random number generator. 
     
     
       3. The electronic display of  claim 1 , wherein the timing controller is configured to instruct the display circuitry to stop writing image data to display pixels while the touch sensing circuitry determines whether the user touch is present on a surface of the electronic display. 
     
     
       4. The electronic display of  claim 1 , wherein the touch sensing circuitry comprise a touch sensing pixel configured to detect presences and position of the user touch based at least in part on an impedance change at the touch sensing pixel. 
     
     
       5. The electronic display of  claim 1 , wherein the timing controller is configured to:
 determine a first set of insertion times for at least two intra-frame pauses for a single image frame; and 
 insert the at least two intra-frame pauses during rendering of the single image frame at the first set of insertion times. 
 
     
     
       6. The electronic display of  claim 5 , wherein the timing controller is configured to:
 determine a second set of insertion times for at least two subsequent intra-frame pauses for a subsequent single image frame; and 
 insert the at least two subsequent intra-frame pauses during rendering of the subsequent single image frame at the second set of insertion times; wherein the first set of insertion times and the second set of insertion times are varied from one another. 
 
     
     
       7. A tangible, non-transitory, computer readable medium storing instructions executable by a processor of an electronic display configured to display an image frame, wherein the instructions comprise instructions to:
 determine a first insertion time for an intra-frame pause from rendering the image frame, wherein the first insertion time is varied from a subsequent insertion time for at least one subsequent frame and wherein the first insertion time comprises a random first insertion time; 
 receive, using the processor, image data corresponding with the image frame; 
 instruct, using the processor, the electronic display to write a portion of the image data to display pixels in the electronic display to display a portion of the image frame; 
 upon reaching the first insertion time, instruct, using the processor, the electronic display to pause writing the image data once the portion of the image frame is displayed; 
 upon completion of the intra-frame pause, instruct, using the processor, the electronic display to write a remaining portion of the image data to the display pixels. 
 
     
     
       8. The computer readable medium of  claim 7 , comprising instructions to instruct, using the processor, the electronic display to determine whether a user touch is present on a surface of the electronic display during the pause. 
     
     
       9. The computer readable medium of  claim 7 , wherein the intra-frame pause causes a delay between writing the portion of the image data and the subsequent portion of the image data to the display pixels. 
     
     
       10. The computer readable medium of  claim 7 , wherein the instructions to determine the random first insertion time comprise:
 obtaining a random seed from a random number generator of the electronic display; and 
 basing the random first insertion time at least in part upon the random seed. 
 
     
     
       11. The computer readable medium of  claim 7 , comprising instructions to:
 upon completion of rendering of the first frame:
 determine the subsequent insertion time for the at least one subsequent frame; 
 instruct, using the processor, the electronic display to write a portion of the image data of the subsequent frame to display pixels in the electronic display to display a portion of the image frame; 
 upon reaching the subsequent insertion time, instruct, using the processor, the electronic display to pause writing the image data of the subsequent frame once the portion of the subsequent frame is displayed; 
 upon completion of the intra-frame pause, instruct, using the processor, the electronic display to write a remaining portion of the image data of the subsequent image frame to the display pixels. 
 
 
     
     
       12. The computer readable medium of  claim 7 , comprising instructions to:
 determine a second insertion time for a second intra-frame pause in the image frame; and 
 insert the second intra-frame pause at the second insertion time during rendering of the image frame. 
 
     
     
       13. The computer readable medium of  claim 12 , comprising instructions to:
 determine a second set of first and second insertion times for at least two subsequent intra-frame pauses for the subsequent image frame; and 
 insert the at least two subsequent intra-frame pauses during rendering of the subsequent image frame at the second set of insertion times; wherein the first insertion time of the first image frame varies from the first insertion time of the subsequent frame and the second insertion time of the second intra-frame pause in the image frame varies from the second insertion time of the subsequent intra-frame pause for the subsequent image frame. 
 
     
     
       14. A method, comprising:
 writing a first portion of display pixels on an electronic display to display the first portion of a first image frame; 
 pausing writing of the display pixels at a first intra-frame pause time; 
 receiving a first user input during the first intra-frame pause time via a user input device; 
 upon completion of the first intra-frame pause time, writing a second portion of the display pixels to display a second portion of the first image frame; 
 upon completion of writing the first image frame, writing a first portion of the display pixels to display a first portion of the second image frame; 
 pausing writing of the display pixels at a second intra-frame pause time; 
 receiving a second user input during the second intra-frame pause time via the user input device; and 
 upon completion of the second intra-frame pause time, writing a second portion of the display pixels to display a second portion of the second image frame; 
 wherein the first intra-frame pause time and the second intra-frame pause time are different; and 
 wherein the first intra-frame pause time, the second intra-frame pause time, or both are generated based upon a seed from a random number generator. 
 
     
     
       15. The method of  claim 14 , wherein the user input device comprises a touch sensing pixel in the electronic display, wherein the touch sensing pixel is configured to determine occurrence and position of the first user input on a surface of the electronic display. 
     
     
       16. The method of  claim 14 , comprising:
 pausing writing of the display pixels at a third intra-frame pause time, during rendering of the first image frame; 
 receiving a third user input during the third intra-frame pause time via the user input device; 
 upon completion of the third intra-frame pause time, writing a third portion of the display pixels to display a third portion of the first image frame; 
 pausing writing of the display pixels at a fourth intra-frame pause time, during rendering of the second image frame; 
 receiving a fourth user input during the fourth intra-frame pause time via the user input device; and 
 upon completion of the fourth intra-frame pause time, writing a third portion of the display pixels to display a third portion of the second image frame; 
 wherein the third and fourth intra-frame pauses vary from one another. 
 
     
     
       17. The method of  claim 16 , comprising:
 generating the third intra-frame pause time, the fourth intra-frame pause time, or both using a random number generator.

Description:
BACKGROUND 
     The present disclosure relates generally to electronic displays, and more particularly, to artifact compensation for touch-sensitive electronic displays. 
     This section is intended to introduce the reader to various aspects of art that may be related to various aspects of the present techniques, which are described and/or claimed below. This discussion is believed to be helpful in providing the reader with background information to facilitate a better understanding of the various aspects of the present disclosure. Accordingly, it should be understood that these statements are to be read in this light, and not as admissions of prior art. 
     Generally, an electronic display may enable information to be communicated to a user by displaying visual representations of the information, for example, as pictures, text, or videos. More specifically, the visual representations may be displayed as successive static image frames. In some embodiments, each image frame may be displayed by successively writing image data to rows of pixels in the electronic display. 
     In addition to outputting information, the electronic display may enable the user to communicate information to the electronic display and/or a computing system that includes the electronic display. For example, the electronic display may be a touch-sensitive display, which may detect a user touch on the surface of the electronic display. More specifically, the electronic display may detect occurrence and/or position of the user touch based at least in part on an impedance (e.g., capacitance) change in the electronic display caused by the user touch. 
     However, at any given time, the electronic display may generally either write image data to the display pixels or check for an impedance change via touch sensing, but not both. Thus, when image data is being written to the pixels, a user touch may be undetected. Similarly, when checking for impedance changes via touch sensing, the electronic display may stop writing image data. As such, in operation, the electronic display may alternate between writing image data to the pixels and checking for a user touch. Moreover, the touch detection accuracy may depend at least in part on frequency the electronic display checks for impedance changes. However, punctuating the writing of the image data with a greater number of touch sensing impedance checks could introduce perceivable artifacts. 
     SUMMARY 
     A summary of certain embodiments disclosed herein is set forth below. It should be understood that these aspects are presented merely to provide the reader with a brief summary of these certain embodiments and that these aspects are not intended to limit the scope of this disclosure. Indeed, this disclosure may encompass a variety of aspects that may not be set forth below. 
     The present disclosure generally relates to improving touch detection accuracy of touch-sensitive electronic displays. More specifically, the touch detection accuracy may be improved by increasing frequency an electronic display checks for a user touch. In fact, the electronic display may alternate between writing portions of image frames with checking for user touch. For example, the electronic display may write a first portion of an image frame to pixels in the electronic display, pause the writing of the image frame, check for a user touch, and write a second portion of the image frame to the pixels. As used herein, pausing the writing of an image frame to check for a user touch is generally referred to as an “intra-frame pause.” 
     However, pausing in the middle writing of an image frame may cause perceivable visual artifacts on the electronic display, particularly when the desired brightness level (e.g., grayscale value) between successively displayed image frames is changing. More specifically, an intra-frame pause may cause a small delay between writing the first portion of the image frame and writing the second portion of the image frame. In some embodiments, when the successively displayed image frames are changing brightness level (e.g., grayscale value), the delay may cause the brightness of the second portion to be perceptively different from the first portion even when both portions are supposed to be displaying the same brightness level. For example, when the brightness is increasing, the second portion may be displayed darker than desired. On the other hand, when the brightness is decreasing, the second portion may be displayed brighter than desired. The perceptibility of these artifacts may be accentuated when the delay occurs at the same frame rendering position for successive frames. 
     Accordingly, in some embodiments, timing of the intra-frame pauses may be randomly implemented during the rendering of each frame. In this manner, since a human eye generally averages the brightness level of a pixel over short durations of time (e.g., time to write one image frame), adjusting the location of intra-frame pauses may reduce the perception of intra-frame pause induced artifacts. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       Various aspects of this disclosure may be better understood upon reading the following detailed description and upon reference to the drawings in which: 
         FIG. 1  is a block diagram of a computing device, in accordance with an embodiment; 
         FIG. 2  is an example of the computing device of  FIG. 1 , in accordance with an embodiment; 
         FIG. 3  is an example of the computing device of  FIG. 1 , in accordance with an embodiment; 
         FIG. 4  is an example of the computing device of  FIG. 1 , in accordance with an embodiment; 
         FIG. 5  is block diagram of a portion of the computing device of  FIG. 1  used to display images and sense user touch, in accordance with an embodiment; 
         FIG. 6  is a schematic diagram of display components of an electronic display, in accordance with an embodiment; 
         FIG. 7  is a schematic diagram of touch sensing components of the electronic display, in accordance with an embodiment; 
         FIG. 8A  is a plot of grayscale value of a pixel before an intra-frame pause compared to a pixel after the intra-frame pause, in accordance with an embodiment; 
         FIG. 8B  is a plot of the difference between a target grayscale value and an actual grayscale value for the pixel before the intra-frame pause and the pixel after the intra-frame pause, in accordance with an embodiment; 
         FIG. 9  is a timing diagram illustrating random intra-frame pause timing within the rendering of successive frames, in accordance with an embodiment; 
         FIG. 10  illustrates the placement of intra-frame pauses in the successive frames, based upon the timings of  FIG. 9 , in accordance with an embodiment; 
         FIG. 11  illustrates an exemplary comparison of artifacts produced from fixed intra-frame pauses vs. varied intra-frame pauses, in accordance with an embodiment; 
         FIG. 12  is a results data diagram, illustrating a perceptibility comparison of artifacts generated from fixed vs. varied intra-frame pauses, in accordance with an embodiment; and 
         FIG. 13  is a flowchart illustrating a process for using varied-time intra-frame pauses, in accordance with an embodiment. 
     
    
    
     DETAILED DESCRIPTION 
     One or more specific embodiments of the present disclosure will be described below. These described embodiments are only examples of the presently disclosed techniques. Additionally, in an effort to provide a concise description of these embodiments, all features of an actual implementation may not be described in the specification. It should be appreciated that in the development of any such actual implementation, as in any engineering or design project, numerous implementation-specific decisions must be made to achieve the developers&#39; specific goals, such as compliance with system-related and business-related constraints, which may vary from one implementation to another. Moreover, it should be appreciated that such a development effort might be complex and time consuming, but may nevertheless be a routine undertaking of design, fabrication, and manufacture for those of ordinary skill having the benefit of this disclosure. 
     When introducing elements of various embodiments of the present disclosure, the articles “a,” “an,” and “the” are intended to mean that there are one or more of the elements. The terms “comprising,” “including,” and “having” are intended to be inclusive and mean that there may be additional elements other than the listed elements. Additionally, it should be understood that references to “one embodiment” or “an embodiment” of the present disclosure are not intended to be interpreted as excluding the existence of additional embodiments that also incorporate the recited features. 
     As mentioned above, a touch-sensitive electronic display may enable both the output of information to a user from a computing system as well as the input of control commands from the user to the computing system. More specifically, the electronic display may write image data to pixels to display visual representations of the information. Additionally, the electronic display may detect a user touch by checking for changes in impedance (e.g., capacitance) caused by the user touch on the surface of the electronic display. 
     Generally, an electronic display may alternate between writing image data and checking for a user touch. For example, the electronic display may write an entire image frame to the display pixels, check for a user touch, and repeat. However, at one time, the electronic display may only perform one of writing image data or checking for impedance changes, but not both. In other words, when the electronic display is writing image data to the pixels, a user touch during that period may go undetected. 
     Accordingly, to improve the user touch detection, the frequency the electronic display checks for a user touch may be increased. For example, the electronic display may write a first portion of an image frame, pause the writing of the image frame, check for an impedance change, write a second portion of the image frame, pause the writing of the image frame, check for an impedance change, and so on. As used herein, pausing the writing of an image frame to check for a user touch is generally referred to as an “intra-frame pause.” In this manner, intra-frame pauses may enable the frequency the electronic display checks for a user touch to be increased, which may improve user touch detection accuracy. 
     However, the intra-frame pause between writing portions of an image frame may cause perceivable visual artifacts. As will be described in more detail below, artifacts may be more likely perceivable when placed at the same render time of successively displayed image frames. More specifically, artifacts may be accentuated by appearing in the same location of a frame, due to the intra-frame pause occurring at a common time during frame rendering of successive frames. 
     Accordingly, one embodiment of the present disclosure describes an electronic display that reduces the likelihood of perceivable visual artifacts by compensating for an intra-frame pause. As will be described in more detail below, the electronic display may include a timing controller (TCON) that varies a timing of the intra-frame pauses for successive frames. Accordingly, any artifacts resulting from the intra-frame pauses will also be displayed in varied locations of successive frames. 
     In perceiving the visual representations displayed on the electronic display, a user&#39;s eyes generally averages the brightness level of a pixel across a short period of time (e.g., time used to write an image frame). As such, the user&#39;s eyes may average out undesired brightness levels caused by the intra-frame pause with the adjusted brightness level to perceive the desired brightness levels, especially when the artifacts are provided in varying areas of successive frames. Thus, the techniques described herein may enable an electronic display to improve user touch detection accuracy while minimizing the likelihood of perceivable visual artifacts. To help illustrate, a computing device  10  that utilizes a touch-sensitive electronic display  12  is described in  FIG. 1 . As will be described in more detail below, the computing device  10  may be any suitable computing device, such as a handheld computing device, a tablet computing device, a notebook computer, and the like. 
     Accordingly, as depicted, the computing device  10  includes the display  12 , input structures  14 , input/output (I/O) ports  16 , one or more processor(s)  18 , memory  20 , nonvolatile storage  22 , a network interface  24 , and a power source  26 . The various components described in  FIG. 1  may include hardware elements (including circuitry), software elements (including computer code stored on a non-transitory computer-readable medium), or a combination of both hardware and software elements. It should be noted that  FIG. 1  is merely one example of a particular implementation and is intended to illustrate the types of components that may be present in the computing device  10 . Additionally, it should be noted that the various depicted components may be combined into fewer components or separated into additional components. For example, the one or more processors  18  may include a graphical processing unit (GPU). 
     As depicted, the processor  18  is operably coupled with memory  20  and/or nonvolatile storage device  22 . More specifically, the processor  18  may execute instruction stored in memory  20  and/or non-volatile storage device  22  to perform operations in the computing device  10 , such as outputting image data to the display  12 . As such, the processor  18  may include one or more general purpose microprocessors, one or more application specific processors (ASICs), one or more field programmable logic arrays (FPGAs), or any combination thereof. Additionally, memory  20  and/or non volatile storage device  22  may be a tangible, non-transitory, computer-readable medium that stores instructions executable by and data to be processed by the processor  18 . In other words, the memory  20  may include random access memory (RAM) and the non-volatile storage device  22  may include read only memory (ROM), rewritable flash memory, hard drives, optical discs, and the like. By way of example, a computer program product containing the instructions may include an operating system or an application program. 
     Additionally, as depicted, the processor  18  is operably coupled with the network interface  24  to communicatively couple the computing device  10  to a network. For example, the network interface  24  may connect the computing device  10  to a personal area network (PAN), such as a Bluetooth network, a local area network (LAN), such as an 802.11x Wi-Fi network, and/or a wide area network (WAN), such as a 4G or LTE cellular network. Furthermore, as depicted, the processor  18  is operably coupled to the power source  26 , which provides power to the various components in the computing device  10 . As such, the power source  26  may include any suitable source of energy, such as a rechargeable lithium polymer (Li-poly) battery and/or an alternating current (AC) power converter. 
     As depicted, the processor  18  is also operably coupled with I/O ports  16 , which may enable the computing device  10  to interface with various other electronic devices, and input structures  14 , which may enable user interaction with the computing device  10 . Accordingly, the inputs structures  14  may include buttons, keyboards, mice, trackpads, and the like. In addition to the input structures  14 , the display  12  may include touch sensing components to enable user inputs via user touches to the surface of the display  12 . In fact, in some embodiments, the electronic display  12  may detect multiple user touches at once. 
     In addition to enabling user inputs, the display  12  may display visual representations via one or more static image frames. In some embodiments, the visual representations may be a graphical user interface (GUI) for an operating system, an application interface, text, a still image, or a video. As depicted, the display  12  is operably coupled to the processor  18 , which may enable the processor  18  (e.g., image source) to output image data to the display  12 . 
     Based on the received image data, the display  12  may then write image frames to the display pixels in the display  12  to display a visual representation. As will be described in more detail below, once the display  12  receives the image data, additional processing may be performed on the image data to further improve the accuracy of the viewed visual representation. For example, the display  12  (or other component controlling visualizations of the display  12 ) may vary timing of intra-frame pauses to reduce the likelihood of perceivable visual artifacts caused by these intra-frame pauses. 
     As described above, the computing device  10  may be any suitable electronic device. To help illustrate, one example of a handheld device  10 A is described in  FIG. 2 , which may be a portable phone, a media player, a personal data organizer, a handheld game platform, or any combination of such devices. For example, the handheld device  10 A may be any iPhone model from Apple Inc. of Cupertino, Calif. 
     As depicted, the handheld device  10 A includes an enclosure  28 , which may protect interior components from physical damage and to shield them from electromagnetic interference. The enclosure  28  may surround the display  12 , which, in the depicted embodiment, displays a graphical user interface (GUI)  30  having an array of icons  32 . By way of example, when an icon  32  is selected either by an input structure  14  or a touch sensing component of the display, an application program may launch. 
     Additionally, as depicted, input structure  14  may open through the enclosure  28 . As described above, the input structures  14  may enable a user to interact with the handheld device  10 A. For example, the input structures  14  may activate or deactivate the handheld device  10 A, navigate a user interface to a home screen, navigate a user interface to a user-configurable application screen, activate a voice-recognition feature, provide volume control, and toggle between vibrate and ring modes. Furthermore, as depicted, the I/O ports  16  open through the enclosure  28 . In some embodiments, the I/O ports  16  may include, for example, an audio jack to connect to external devices. 
     To further illustrate a suitable computing device  10 , a tablet device  10 B is described in  FIG. 3 , such as any iPad model available from Apple Inc. Additionally, in other embodiments, the computing device  10  may take the form of a computer  10 C as described in  FIG. 4 , such as any MacBook or iMac model available from Apple Inc. As depicted, the computer  10 C also includes a display  12 , input structures  14 , I/O ports  16 , and a housing  28 . 
     As described above, the display  12  may facilitate communication of information between the computing system  10  and a user, for example, by displaying visual representations based on image data received from the processor  18  and detecting user touch on the surface of the display  12 . To help illustrate, a portion  34  of the computing device  10  is described in  FIG. 5 . As depicted, the processor  18  and the display  12  are communicatively coupled via a data bus  36 , which may enable the processor  18  to transmit image data to the display  12  indicating occurrence and/or position of a user touch to the processor  18 . 
     To facilitate such operations, the display  12  may include display components (e.g., display driver circuitry)  38  and touch sensing components (e.g., touch sensing circuitry)  40 . More specifically, the display components  38  may include any suitable components used to display an image frame on the display  12 . For example, when the display  12  is a liquid crystal display, the display components  38  may include a thin film transistor (TFT) layer and a liquid crystal layer organized as display pixels. To help illustrate, operation of display components  38  used in a liquid crystal display are described in  FIG. 6 . 
     In the depicted embodiment, the display components  38  include a number of display pixels  42  disposed in a pixel array or matrix. More specifically, each display pixel  42  may be defined at the intersection of a gate line  44  (e.g. scanning line) and a source lines  106  (e.g., data line). Although only six display pixels  42 , referred to individually by the reference numbers  42 A- 42 F, are shown for purposes of simplicity, it should be understood that in an actual implementation, each source line  46  and gate line  44  may include hundreds or thousands of such display pixels  42 . 
     As described above, image data may be written to each of the display pixels  42  to display an image frame. More specifically, image data may be written to a display pixel  42  by using a thin film transistor  48  to selectively store an electrical potential (e.g., voltage) on a respective pixel electrode  50 . Accordingly, in the depicted embodiment, each thing film transistor  48  includes a source  54 , which is electrically connected to a source line  46 , a drain  56 , which is electrically connected to a pixel electrode  50 , and a gate  58 , which is electrically connected to a gate line  44 . 
     Thus, to write image data to a row of display pixels  42  (e.g.,  42 A- 42 C), the corresponding gates  48  may be activated (e.g., turned on) by a scanning signal on the gate line  44 . Image data may then be written to the row of display pixels by storing an electrical potential corresponding with the grayscale value of the image data from the source lines  46  to the pixel electrode  50 . The potential stored on the pixel electrode  50  relative to a potential of a common electrode  52  may then generate an electrical field sufficient to alter the arrangement of the liquid crystal layer (not shown). More specifically, this electrical field may align the liquid crystal molecules within the liquid crystal layer to modulate light transmission through the display pixel  42 . In other words, as the electrical field changes, the amount of light passing through the display pixel  42  may increase or decrease. As such, the perceived brightness level of the display pixel  42  may be varied by adjusting the grayscale value of the image data. In this manner, an image frame may be displayed by successively writing image data the rows of display pixels  42 . 
     To facilitate writing image data to the display pixels  42 , the display components  38  may also include a source driver  60 , a gate driver  62 , and a common voltage (Vcom) source  64 . More specifically, the source driver  60  may output the image data (e.g., as an electrical potential) on the source lines  46  to control electrical potential stored in the pixel electrodes  50 . Additionally, the gate driver  62  may output a gate signal (e.g., as an electrical potential) on the gate lines  44  to control activation of rows of the display pixels  42 . Furthermore, the Vcom source  64  may provide a common voltage to the common electrodes  52 . 
     Similarly, the touch sensing components  40  may include any suitable components used to detect occurrence and/or presence of a user touch on the surface of the display  12 . To help illustrate, operation of touch sensing components  40  that may be used in a capacitive touch sensitive display  12  are described in  FIG. 7 . 
     In the depicted embodiment, the touch sensing components  40  include a number of touch pixels  66  disposed in a pixel array or matrix. More specifically, each touch pixel  66  may be defined at the intersection of a touch drive line  68  and a touch sense line  70 . Although only six touch pixels  66  are shown for purposes of simplicity, it should be understood that in an actual implementation, each touch drive line  68  and touch sense line  70  may include hundreds or thousands of such touch pixels  66 . 
     As described above, occurrence and/or position of a user touch may be detected based on impedance changes caused by the user touch. To facilitate detecting impedance changes, the touch sensing components  40  may include touch drive logic  72  and touch sense logic  74 . More specifically, the touch drive logic  72  may output touch drive signals at various frequencies and/or phases on the touch drive lines  68 . When an object, such as a user finger, contacts the surface of the display  12 , the touch sense lines  70  may respond differently to the touch drive signals, for example by changing impendence (e.g., capacitance). More specifically, the touch sense lines  70  may generate touch sense signals to enable the touch sense logic  74  to determine occurrence and/or position of the object on the surface of the display  12 . 
     In some embodiments, the touch sensing components  40  may utilize dedicated touch drive lines  68 , dedicated touch sense lines  70 , or both. Additionally or alternatively, the touch drive lines  68  and/or the touch sense lines  70  may utilize one or more of the display components  38 . For example, the touch drive lines  68  and/or the touch sense lines  70  may be formed from one or more gate lines  44 , one or more pixel electrodes  50 , one or more common electrodes  52 , one or more source lines  46 , or any combination thereof. 
     To facilitate controlling operation of both the display components  38  and the touch sensing components  40 , the display  12  may include a timing controller (TCON)  76  as depicted in  FIG. 5 . Accordingly, the timing controller  76  may include a processor  78  and memory  80 . More specifically, the processor  78  may execute instruction stored in memory  80  to perform operations in the display  12 . Additionally, memory  80  may be a tangible, non-transitory, computer-readable medium that stores instructions executable by and data to be processed by the processor  78 . 
     For example, the timing controller  76  may instruct the display components  38  to write image data to the display pixels  42  and instruct the touch sensing components  40  to check for a user touch. As described above, the frequency the touch sensing components  40  detects whether a user touch is present may be increased to improve the user touch detection accuracy. In fact, the timing controller  76  may utilize intra-frame pauses by alternating between instructing the display components  38  to write a portion of an image frame and instructing the touch sensing components  40  to check for a user touch. 
     However, as described above, an intra-frame pause may also potentially cause a perceivable visual artifact, such as pixels being perceived at an undesired brightness level. To help illustrate, the grayscale value (e.g., brightness level) for two display pixels  42  in display  12  is described in  FIG. 8A . More specifically,  FIG. 8A  is a plot that describes the grayscale value of a first pixel, which is written before an intra-frame pause, with a first grayscale curve  82  and the grayscale value of a second pixel, which is written after an intra-frame pause, with a second grayscale curve  84  over a period of operation between 0 to 150 ms, in which time is shown on the X-axis and relative grayscale value (e.g., brightness level) is shown on the Y-axis. 
     To simplify the following discussion, the techniques are described using image frames with a single intra-frame pause and with the same desired brightness level for each display pixel  42 . In the depicted embodiment, the image frames displayed from 0 to 50 ms have a brightness level of 5% (e.g., grayscale value of 64). At 50 ms, the desired brightness level of the image frame is increased from the previously displayed image frames. As such, the grayscale value of the image data written to the first pixel and the second pixel are increased. 
     More specifically, as described by the first grayscale curve  82 , the grayscale value of the image data written to the first pixel begins to be increased at 50 ms or very shortly thereafter. On the other hand, as described by the second grayscale curve  84 , the grayscale value of the image data written to the second pixel begins to increase after an intra-frame pause  86 , which may enable the display  12  to check for a user touch between writing image data to the first pixel and the second pixel. As depicted, the intra-frame pause  86  causes the grayscale value change of the second pixel to lag behind the first pixel. As such, the perceived brightness level of the second pixel may be less than the perceived brightness level of the first pixel. 
     Since an intra-frame pause is generally very short in duration (e.g., 1 or 2 ms), the intra-frame pause may only cause a perceivable brightness difference (e.g., visual artifact) when the desired brightness level is changing by more than a threshold amount. For example, in the depicted embodiment, another intra-frame pause  88  occurs at approximately 25 ms. However, since the desired brightness level is not changing, the intra-frame pause  88  does not cause a perceivable visual artifact. On the other hand, since the desired brightness level is changing at 50 ms, the intra-frame pause  86  may cause a perceivable visual artifact. 
     To help illustrate, the grayscale value difference between the first pixel and the second pixel is described in  FIG. 8B . More specifically,  FIG. 8B  is a plot that describes the difference between a target grayscale value and the grayscale value of the first pixel with a first difference curve  90  and difference between the target grayscale value and the grayscale value of the second pixel with a second difference curve  92  over the period of operation, in which time is shown on the X-axis and grayscale difference is shown on the Y-axis. As used herein, the “target grayscale value” may be the grayscale value of image data received from the image source. In other words, when the target grayscale value is written to a display pixel without an intra-frame pause, the display pixel may be at its desired brightness level. 
     As depicted, between 0 to 50 ms, the grayscale value of the first pixel and the second pixel are generally as desired. As such, the first pixel and the second pixel may be perceived at the desired brightness level. However, at 50 ms, both the first pixel and the second pixel vary from the target grayscale value. More specifically, as described by the first difference curve  90 , even though the first pixel is written before the intra-frame pause  86 , the grayscale value of the first pixel still varies slightly from the target grayscale value. In some embodiments, the difference may result from the less than instantaneous switching of the TFT  48  in the first pixel and/or that the first pixel is not in the first row of display pixels  42 . Nonetheless, the difference in grayscale value of the first pixel will generally not be perceivable by a user&#39;s eyes since it varies by less than 0.1% from the target grayscale value. 
     On the other hand, as described by the second difference curve  92 , the intra-frame pause  86  causes the grayscale value of the second pixel to vary from the target grayscale value by approximately 1.2%. In other words, the second pixel may be more than 1.1% darker than the first pixel, which may be perceivable by a user&#39;s eyes. Furthermore, since the grayscale value of the second pixel does not catch up to the grayscale value of the first pixel until the grayscale values stop substantially changing (e.g., at 100 ms), the intra-frame pause  86  may cause a perceivable artifact at the second pixel from 50 ms until close to 100 ms. 
     As discussed above, a visual artifact may be perceivable when an intra-frame pause causes the perceived brightness level of a pixel after the intra-frame pause (e.g., the second pixel) to vary from the desired brightness level by a perceivable amount. These artifacts may be accentuated when the intra-frame pauses for successive frames occur in common location in the image frame. Accordingly, intra-frame pauses may be introduced at varied times during the frame rendering process. 
     To help illustrate an embodiment of varied insertion of intra-frame pauses, a timing diagram illustrating the operation of the electronic display  12  is described in  FIG. 9 . More specifically,  FIG. 9  describes operation of the display components  38  with a write timing plot  94  and operation of the touch sensing components  40  with a touch sensing timing plot  96  during a period when three image frames are displayed. 
     For the purpose of description, a first image frame is written between t 0  and t 3 , a second image frame is written between t 3  and t 6 , a third image frame is written between t 6  and t 9 , and a fourth image frame is written between t 9  and  12 . In the depicted embodiment, each image frame may be written in two separate portions separated by an intra-frame pause. For example, between t 0  and t 1 , the timing controller  76  may instruct the display components  38  to write a first portion of the first image frame to a corresponding portion of the pixels  42 . Between t 1  and t 2 , the timing controller  76  may instruct the display components  38  to pause writing the first image frame and instruct the touch sensing components  40  to check for a user touch. Between t 2  and t 3 , the timing controller  76  may instruct the display components  38  to write the second portion of the first image frame to the corresponding pixels  42 , resulting in the completed rendering of the first image frame. Between t 3  and t 4 , the timing controller  76  may instruct the display components  38  to write a first portion of the second image frame to a corresponding set of the pixels  42 . Between t 4  and t 5 , the timing controller  76  may again instruct the display components  38  to pause writing the second image frame and instruct the touch sensing components  40  to check for a user touch. Between t 5  and t 6 , the timing controller  76  may instruct the display components  38  to write the remaining portion of the second image frame to a corresponding set of the pixels  42 . Between t 6  and t 7 , the timing controller  76  may instruct the display components  38  to write a first portion of the third image frame to a corresponding portion of the pixels  42 . Between t 7  and t 8 , the timing controller  76  may instruct the display components  38  to pause writing the third image frame and instruct the touch sensing components  40  to check for a user touch. Between t 8  and t 9 , the timing controller  76  may instruct the display components  38  to write the second portion of the third image frame to the corresponding pixels  42 , resulting in the completed rendering of the third image frame. Between t 9  and t 10 , the timing controller  76  may instruct the display components  38  to write a first portion of the fourth image frame to a corresponding portion of the pixels  42 . Between t 10  and t 11 , the timing controller  76  may instruct the display components  38  to pause writing the fourth image frame and instruct the touch sensing components  40  to check for a user touch. Between t 11  and t 12 , the timing controller  76  may instruct the display components  38  to write the second portion of the fourth image frame to the corresponding pixels  42 , resulting in the completed rendering of the fourth image frame 
     As described above, an intra-frame pause may occur when the display  12  pauses writing an image frame to check for a user touch. Accordingly, the time periods from t 1  to t 2 , t 4  to t 5 , t 7  to t 8 , and t 10  to t 11  may each be referred to as an intra-frame pause. As depicted, the intra-frame pauses enable the frequency with which the display  12  checks for a user touch to be increased. For example, in the depicted embodiment, the display  12  may check for a user touch one time during the time period one image frame is written. Generally, the number intra-frame pauses used may be based on desired user touch detection accuracy. For example, the number of intra-frame pauses may be increased to improve user touch detection accuracy. As such, writing of an image frame may be divided by any suitable manner (e.g., 1, 2, 3, 4, 5, or more intra-frame pauses). 
     As mentioned herein, the perception of artifacts resulting from intra-frame pauses may be reduced by varying insertion time of the intra-frame pauses. The insertion of the intra-frame pauses may be varied from pause-to-pause and/or frame-to-frame. For example, in some embodiments, a randomizer may dictate random times to insert intra-frame pauses into the frame renderings. The varied timing of the insertion of intra-frame pauses may reduce the likelihood of a visual artifact being perceived, by distributing these artifacts to varied locations in the successive frames. 
       FIG. 10  illustrates a positioning of the intra-frame pauses  100 A-D within successive frames  102 A-D, where the intra-frame pause insertion point is varied to reduce perceptibility of artifacts associated with these pauses, in accordance with an embodiment. As illustrated, because the intra-frame pause timings (e.g., t 1  to t 2 , t 4  to t 5 , t 7  to t 8 , and t 10  to  11  in  FIG. 9 ) are varied, the intra-frame pauses  100 A-D are inserted at varied locations in the rendering of the frames  102 A- 102 . In some embodiments, the varied timing may be varied based upon random insertion timings (e.g., provided via a random number generator). As will be illustrated in more detail with regard to  FIGS. 11 and 12 , by varying the timing of the intra-frame pause insertions, artifacts from these insertions may be less perceivable by the human eye. 
     To provide an illustration of the effects of intra-frame pause insertion,  FIG. 11  illustrates a scenario  110 , where successive frames  102 A-D switch between dark frames  112  to light frames  114 . Progression  116  illustrates a scenario where common intra-frame pause insertion times are used. As illustrated, frame artifacts  118  occur at a common location within the frames  102 A-D, when the common intra-frame pause insertion time is used. In contrast, when varied (e.g., randomized) intra-frame pause insertion times are used, as in scenario  120 , the location of the artifacts  118  will also be varied. 
       FIG. 12  is a results chart  130 , illustrating a contrast between the scenarios  116  and  120  of  FIG. 11 . The x-axis  132  represents a duration time of the intra-frame pauses in microseconds. The y-axis  134  represents a perceptibility score (e.g., how visible intra-field pause artifacts are). Line  136  provides a data line representing a function of visibility of artifacts for particular fixed-location intra-frame pause durations. Line  138  provides a data line representing a function of visibility of artifacts for particular varied-location intra-frame pause durations. As may be appreciated, the varied-location intra-frame pauses result in reduced perceptibility. For example, varied location intra-frame pauses with a duration of approximately 700 microseconds have a perceptibility score of less than one (e.g. minor perceptibility). In contrast, a fixed-location intra-frame pause with the same duration of approximately 700 microseconds has a perceptibility score of greater than three (e.g., major perceptibility). Accordingly, it may be appreciated that the varied location intra-frame pauses are highly effective in reducing perceptibility of artifacts resulting from intra-frame pauses. 
     Referring now to  FIG. 13 , a process  140  for varied time insertion of intra-frame pauses is provided. The process  140  may be implemented via one or more processors that receive processor-readable instructions from a tangible, non-transitory machine-readable medium. The process  140  begins with determining a varied intra-frame pause insertion time (block  142 ). For example, varied intra-frame pause insertion times may be determined by an algorithm or other hardware. In one example, varied intra-frame pause insertion times may be determined based upon a random number generator seed. For example, a seed from the following Verilog-based random number generator may be used to generate insertion times for an intra-frame pause. 
     
       
         
           
               
             
               
                   
               
               
                 Verilog code: 
               
               
                   
               
             
            
               
                   
               
            
           
           
               
               
            
               
                   
                 ! 
               
               
                   
                 rand_next(0)&lt;=rand_current(1); 
               
               
                   
                 rand_next(1)&lt;=rand_current(2) xor rand_current(0); 
               
               
                   
                 rand_next(2)&lt;=rand_current(3) xor rand_current(0); 
               
               
                   
                 rand_next(3)&lt;=rand_current(4) xor rand_current(0); 
               
               
                   
                 rand_next(4)&lt;=rand_current(5); 
               
               
                   
                 rand_next(5)&lt;=rand_current(6); 
               
               
                   
                 rand_next(6)&lt;=rand_current(7); 
               
               
                   
                 rand_next(7)&lt;=rand_current(8); 
               
               
                   
                 rand_next(8)&lt;=rand_current(0); 
               
               
                   
                   
               
            
           
         
       
     
     Once the intra-frame pause insertion times are determined, frame rendering may begin by rendering a portion of the frame (block  144 ). Once the intra-frame pause insertion time is reached, frame rendering is paused, resulting in the insertion of the intra-frame pause (block  146 ). Upon completion of the intra-frame pause, a subsequent portion of the frame may be rendered (block  144 ). This process may continue until each of the intra-frame pauses is complete and the entire frame is rendered. At that point, the process  140  may continue, rendering the next successive frame and the corresponding intra-frame pauses. 
     As described herein, the technical effects of the present disclosure include improving user touch detection accuracy in an electronic display through the use of intra-frame pauses while reducing perceivable visual artifacts. More specifically, the electronic display may use intra-frame pauses to check for a user touch at a higher frequency. Additionally, the likelihood of perceivable artifacts being generated from the intra-frame pauses may be reduced by varying the insertion timing of the intra-frame pauses. 
     The specific embodiments described above have been shown by way of example, and it should be understood that these embodiments may be susceptible to various modifications and alternative forms. It should be further understood that the claims are not intended to be limited to the particular forms disclosed, but rather to cover all modifications, equivalents, and alternatives falling within the spirit and scope of this disclosure.

Metadata:
Filing Date: 20150306
Publication Date: 20170808
Grant Date: 20170808
Priority Date: 20150306
Inventors: WANG CHAOHAO
SACCHETTO PAOLO
PINTZ SANDRO H.
TANN CHRISTOPHER P.
JIANG JUN
ZHANG LU
Assignee: APPLE INC
CPC Classifications: [{"code": "G06F3/0412", "inventive": true, "first": false, "tree": "[]"}, {"code": "G06F3/044", "inventive": true, "first": true, "tree": "[]"}, {"code": "G09G5/12", "inventive": true, "first": false, "tree": "[]"}, {"code": "G09G2310/08", "inventive": false, "first": false, "tree": "[]"}, {"code": "G09G3/20", "inventive": true, "first": false, "tree": "[]"}, {"code": "G09G3/3611", "inventive": false, "first": false, "tree": "[]"}, {"code": "G09G3/20", "inventive": true, "first": true, "tree": "[]"}, {"code": "G06F3/04184", "inventive": true, "first": false, "tree": "[]"}, {"code": "G06F3/0446", "inventive": true, "first": false, "tree": "[]"}, {"code": "G06F3/0412", "inventive": true, "first": false, "tree": "[]"}, {"code": "G09G3/3611", "inventive": false, "first": false, "tree": "[]"}, {"code": "G09G2310/08", "inventive": false, "first": false, "tree": "[]"}, {"code": "G09G5/12", "inventive": true, "first": false, "tree": "[]"}, {"code": "G09G3/3611", "inventive": false, "first": false, "tree": "[]"}, {"code": "G09G3/20", "inventive": true, "first": true, "tree": "[]"}, {"code": "G09G2310/08", "inventive": false, "first": false, "tree": "[]"}, {"code": "G09G5/12", "inventive": true, "first": false, "tree": "[]"}, {"code": "G06F3/0446", "inventive": true, "first": false, "tree": "[]"}, {"code": "G06F3/0412", "inventive": true, "first": false, "tree": "[]"}]
Family ID: 56850651