Abstract:
Repeating image content in a graphics image may be detected by identifying certain commands, known generally as “tile commands”. If a tile command is detected, the fact that a portion of an image was created with a tile command may be stored along with the portion of the image. This allows for well-informed decision making when transmission of the image is to be performed. For example, this allows for the transmission of a single tile, and subsequent transmission of local copy commands to repeat the single tile. This can be very useful in speeding transmission of background images or other repeating images.

Description:
CROSS-REFERENCE TO RELATED APPLICATION  
       [0001]    The present application is a continuation-in-part of co-pending application Ser. No. 10/247,907, filed on Sep. 20, 2002, by Thomas G. O&#39;Neill and Jordan M. Slott, entitled “TAGGING SINGLE-COLOR IMAGES FOR IMPROVED COMPRESSION”, attorney docket no. SUN-P7083, which is a continuation in part of co-pending application Ser. no. 10/194,123, filed on Jul. 11, 2002, by Thomas G. O&#39;Neill and Jordan M. Slott, entitled “TAGGING MULTICOLOR IMAGES FOR IMPROVED COMPRESSION”, attorney docket no. SUN-P7082. 
     
    
     
       FIELD OF THE INVENTION  
         [0002]    The present invention relates to the field of computer graphics. More particularly, the present invention relates to the tagging of repeating images in order to improve compression.  
         BACKGROUND OF THE INVENTION  
         [0003]    Remote computing applications where screen information is generated on one computer (a “host”) and transmitted for display on another computer (“a display”) are growing in popularity. Examples of some display computers include multipurpose PCs, thin-clients, and Personal Digital Assistants (PDAs).  
           [0004]    The X Windows System is a standard that utilizes X client software and X server software to enable the updating of displays at the requests of X client applications. The X server software runs on the host computer. An X client application, which may be running on a different computer, communicates with the X server software by utilizing a low-level library of routines known as Xlib. Xlib provides the functions required to connect to display servers, create windows, render graphics, respond to events, etc. The X server software then may interface with display drivers to actually render the graphics on the display.  
           [0005]    X is frequently used as a “local display application”, where the X-server and display are both on the same computer. That is, the same computer is both the “host” and “display” computer. Examples of this usage include running an X-server on a workstation or on an X-terminal. An X-terminal typically has a computer processor, graphics subsystem and display, but no hard drive. Applications running on other computers use the Xlib routines to communicate with the X-server software running on the X-terminal.  
           [0006]    While in some contexts it is advantageous to have the X server and the display on the same computer, this is not necessarily the case in other contexts. One specific context that will be discussed is a remote display application. In such a design, client applications make requests of a centralized server or servers (here known collectively as the “host computer”). The host computer then manages one or more “display computers”, which are typically simple terminal devices.  
           [0007]    The Sun Ray™ appliance from Sun Microsystems, Inc. of Palo Alto, Calif. is an example of a thin client which serves as a “display computer” in a remote computing application. A Sun Ray™ appliance has a processor, graphics subsystem and display, but no hard drive. A Sun Ray™ appliance is a “display” computer and runs its own proprietary software. The Sun Ray™ server is the “host” computer and runs the X-server software. The full screen image is maintained both in RAM on the host computer as well as in the frame buffer of the Sun Ray™ appliance&#39;s video card. In order to synchronize the displays, the host computer sends screen update information to the Sun Ray™ appliance via a network protocol known as NewT. The Sun Ray™ appliance uses the protocol commands to update the state of its hardware frame buffer.  
           [0008]    In remote display applications, an increased burden is placed on the network as more information is transmitted from the host computer to the display computers. It is desirable to reduce network bandwidth used by remote computing applications. Doing so provides shorter transmission times between the host and display computers, reduced load on the network (and resulting improvement in network performance), and the capability to utilize more devices on a single network.  
           [0009]    For many typical computing functions such as web browsing, the network bandwidth between the host computer and the display computer is dominated by the transmission of images with a large number of colors, such as photographs or computer generated images which include anti-aliased text or graphics, as well as single-color regions, such as “fill” regions. Furthermore, even once these multi-color and single-color images are addressed, there can still exist a large amount of screen space dedicated to repeating images. These typically include background “tiles”, which are repeated over and over on the screen.  
           [0010]    In particular, the root window in many window managers is a tile of a repeating M×N image, with M indicating horizontal pixels and N indicating vertical. Stipples, which are essentially two-color tiles, are another source of repeating images.  
           [0011]    Sending multiple copies of the tile image wastes bandwidth, host computer processing power, and display computer processing power. The resulting delay can be noticeable even over a 100 Mbps connection. What is needed is an efficient mechanism for the host computer to identify and utilize the identification of tiled regions so that the tile image may only be sent once.  
         BRIEF DESCRIPTION  
         [0012]    Repeating image content in a graphics image may be detected by identifying certain commands, known generally as “tile commands”. If a tile command is detected, the fact that a portion of an image was created with a tile command may be stored along with the portion of the image. This allows for well-informed decision making when transmission of the image is to be performed. For example, this allows for the transmission of a single tile, and subsequent transmission of local copy commands to repeat the single tile. This can be very useful in speeding transmission of background images or other repeating images.  
       
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0013]    The accompanying drawings, which are incorporated into and constitute a part of this specification, illustrate one or more embodiments of the present invention and, together with the detailed description, serve to explain the principles and implementations of the invention.  
         [0014]    In the drawings:  
         [0015]    [0015]FIG. 1 is a block diagram illustrating a remote display application such as a Sun Ray™ network.  
         [0016]    [0016]FIG. 2 is a block diagram illustrating a remote display application in accordance with an embodiment of the present invention.  
         [0017]    [0017]FIG. 3 is a diagram illustrating how when one of the tile commands is used on a realized window, the new repeating image may be sent as a display update to the display computer, in accordance with an embodiment of the present invention.  
         [0018]    [0018]FIG. 4 is a diagram illustrate how local copies can be used to paint part or all of the upper-left width*height pixels on the display computer in accordance with another embodiment of the present invention.  
         [0019]    FIGS.  5 A- 5 C are flow diagrams illustrating a method for reducing the size of a graphics image in a computer system in accordance with an embodiment of the present invention.  
         [0020]    [0020]FIG. 6 is a flow diagram illustrating the sending of the screen update resulting from copying a pixmap to a realized window in accordance with an embodiment of the present invention.  
         [0021]    [0021]FIG. 7 is a flow diagram illustrating a method for subtracting the drawn-to region from the fill and tile linked lists in accordance with an embodiment of the present invention.  
         [0022]    [0022]FIG. 8 is a flow diagram illustrating a method for submitting a region to a fill linked list in accordance with an embodiment of the present invention.  
         [0023]    [0023]FIG. 9 is a flow diagram illustrating a method for submitting a region to a tile linked list in accordance with an embodiment of the present invention.  
         [0024]    [0024]FIG. 10 is a flow diagram illustrating a method for submitting a region to the miscellaneous region under the “disjoint” embodiment.  
         [0025]    [0025]FIG. 11 is a flow diagram illustrating performing tag copy from a source to a destination in accordance with an embodiment of the present invention.  
         [0026]    [0026]FIG. 12 is a flow diagram illustrating a method for submitting a region to the miscellaneous region under the “underlay” embodiment.  
         [0027]    [0027]FIG. 13 is a flow diagram illustrating a method for reducing the size of a graphics image in a computer system in accordance with an embodiment of the present invention.  
         [0028]    [0028]FIG. 14 is a flow diagram illustrating a method for reducing the size of a graphics image in a computer system in accordance with another embodiment of the present invention.  
         [0029]    [0029]FIG. 15 is a block diagram illustrating an apparatus for reducing the size of a graphics image in a computer system in accordance with an embodiment of the present invention.  
         [0030]    [0030]FIG. 16 is a block diagram illustrating an apparatus for reducing the size of a graphics image in a computer system in accordance with another embodiment of the present invention.  
     
    
     DETAILED DESCRIPTION  
       [0031]    Embodiments of the present invention are described herein in the context of a system of computers, servers, and software. Those of ordinary skill in the art will realize that the following detailed description of the present invention is illustrative only and is not intended to be in any way limiting. Other embodiments of the present invention will readily suggest themselves to such skilled persons having the benefit of this disclosure. Reference will now be made in detail to implementations of the present invention as illustrated in the accompanying drawings. The same reference indicators will be used throughout the drawings and the following detailed description to refer to the same or like parts.  
         [0032]    In the interest of clarity, not all of the routine features of the implementations described herein are shown and described. It will, of course, be appreciated that in the development of any such actual implementation, numerous implementation-specific decisions must be made in order to achieve the developer&#39;s specific goals, such as compliance with application- and business-related constraints, and that these specific goals will vary from one implementation to another and from one developer to another. Moreover, it will be appreciated that such a development effort might be complex and time-consuming, but would nevertheless be a routine undertaking of engineering for those of ordinary skill in the art having the benefit of this disclosure.  
         [0033]    In accordance with the present invention, the components, process steps, and/or data structures may be implemented using various types of operating systems, computing platforms, computer programs, and/or general purpose machines. In addition, those of ordinary skill in the art will recognize that devices of a less general purpose nature, such as hardwired devices, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), or the like, may also be used without departing from the scope and spirit of the inventive concepts disclosed herein.  
         [0034]    One example use for the present invention would be to improve compression of images created in pixel-based (as opposed to vector-based) drawing programs. Here, one might keep track of the drawing commands (circle, filled circle, square, image importation, etc.) used to draw the different portion of the image and use that information to influence compression decisions when saving the results to a file.  
         [0035]    In the present application, an embodiment of the present invention is described in the context of a modification of traditional X Windows technology for use with remote computing applications. However, one of ordinary skill in the art will recognize that other embodiments are possible and the present invention should not be limited to X Windows, Sun Ray™, or remote computing technology.  
         [0036]    In remote display applications based on an X server, the Device Dependent X (DDX) layer of the X server may be modified to include possible compression of the screen data and transmission of remote display protocol commands to the display computer. The DDX provides a software interface to a conceptual hardware device. In an embodiment of the present invention, the desired results can be accomplished by modifying the DDX, without changing the DIX.  
         [0037]    [0037]FIG. 1 is a block diagram illustrating a remote display application such as a Sun Ray™ network. Client applications  100   a - 100   c  send information to and receive information from the host computer  102  over the X Windows Protocol. Some of the information comprises repeating images. The host computer utilizes a device independent layer (DIX)  104  to act as a controlling layer, which handles initialization procedures and all device-independent handling of requests and events. A device dependent layer (DDX)  106  then is used to handle device dependent functionality. The DDX  106  then communicates with display computer  108  using the NewT protocol. The DIX and the DDX essentially make up an X-server.  
         [0038]    The present invention provides a mechanism to identify repeating images for special treatment. As will be seen, this results in the ability to dramatically reduce CPU and network bandwidth utilization without affecting image quality. The present application will focus on this identification and handling of repeating images, however the identification and handling of multicolor and singlecolor non-repeating images as well as other, miscellaneous images will also be discussed. The solution for identifying and handling repeating images may be used in conjunction with the solutions for identifying and handling single-color, multi-color images and miscellaneous images, but does not have to be. The identification and handling of repeating images alone will still improve CPU and bandwidth utilization and overall efficiency.  
         [0039]    [0039]FIG. 2 is a block diagram illustrating a remote display application in accordance with an embodiment of the present invention. Client applications  200   a - 200   c  send information to and receive information from the host computer  202  over the X Windows Protocol. Some of the information is repeating image information. The host computer utilizes a device independent layer (DIX)  204  to act as a controlling layer, which handles initialization procedures and all device-independent handling of requests and events. A device dependent layer (DDX)  206  then is used to handle device dependent functionality. An image compression layer  208  may then be provided which performs the additional image compression techniques discussed in the present document. Alternatively, this image compression layer may simply be part of a modified DDX. The image compression layer  208  then communicates with display computer  210  using an enhanced version of the NewT protocol.  
         [0040]    For purposes of this application, a “tile command” may be defined as a drawing command that creates a horizontally and/or vertically repeating image. In the X-server, this is typically accomplished via one of several DDX commands:  
         [0041]    1) A PaintWindowBackground for a window with backgroundState==BackgroundPixmap.  
         [0042]    2) A PaintWindowBorder for a window with borderIsPixel==FALSE  
         [0043]    3) A PolyFillRect with a full planemask and an alu of GXcopy, if:  
         [0044]    a) fillStyle is FillTiled or FillOpaqueStippled, or  
         [0045]    b) fillStyle is FillStippled, and the drawing occurs on top of a single-colored region.  
         [0046]    4) CopyArea or CopyWindow where the source drawable includes a repeating image.  
         [0047]    5) RestoreAreas from a backing store pixmap containing a repeating image.  
         [0048]    6) SaveAreas to a backing store pixmap from a tagged window containing a repeating image.  
         [0049]    Commands 1-3 above can be used to create a repeating image directly in a realized (on-screen) window. Commands 4-6 above can be used to draw a repeating image indirectly, by first creating it in the source drawable.  
         [0050]    In an embodiment of the present invention, the tile regions that are created using one of these tile commands may be tagged. In an embodiment of the present invention, the tagging may be accomplished by attaching a data structure to each pixmap via the pixmap devPrivates facility. In another embodiment of the present invention, the tagging of windows may be accomplished by having a single data structure for the entire screen image.  
         [0051]    In an embodiment of the present invention, the data structure may include a list of tile entries. For example, a linked list could be used. One entry may exist for each tile region that is not compatible with any other tile region. For purposes of the present invention, two regions are compatible if they are drawn with the same tile image and the tile edges occur at the same multiples of x and y. Each linked list entry may contain the following information:  
         [0052]    1) Width of the tile image  
         [0053]    2) Height of the tile image  
         [0054]    3) Origin x of the tile image  
         [0055]    4) Origin y of the tile image  
         [0056]    5) A hash value for the tile image (e.g., MD5), to allow quick comparison of whether two tile images are identical  
         [0057]    6) X region struct identifying the portion of the pixmap or screen image that contains this tile image.  
         [0058]    In this embodiment, tile regions are compatible if they have the same values of width, height, hash value, x modulo width, and y modulo height. The modulo computations take into account the fact that the image repeats every width pixels horizontally and height pixels vertically. Thus, in one embodiment, it is convenient to store x modulo width and y modulo height instead of just x and y.  
         [0059]    In another embodiment, the tile image may be used in place of the hash value. For example, a pointer to the tile image pixmap could be used instead of the hash. In this case, the pixmap&#39;s reference count would have to be incremented to ensure that the pixmap does not get deleted while the pointer is still set.  
         [0060]    The linked list may be associated with the pixmap through the use of the DDX pixmap devPrivates facility, which allows an X extension to attach a private data structure to each X pixmap. In X, a region resource may be used to describe an arbitrarily shaped piece of real estate in XY pixel space. I.e., it can be used to describe an arbitrary collection of pixels by characterizing their XY positions. In X, the positions are characterized by an array of rectangles for which every pixel is in the region. This representation is efficient for regularly-shaped regions, which occur frequently in X, but inefficient for regions with a lot of isolated pixels, which occur much less frequently in X. The rectangle representation is also very convenient for remote display applications such as Sun Ray™, where each protocol command specifies the screen update for a single screen rectangle.  
         [0061]    X also provides a series of routines for performing set operations (union, subtraction, intersection, etc.) on the region resources, which are very convenient in the practice of the invention. The region resources and routines were originally developed to assist with the computation of clipping and drawn-to regions. The composite clip list for a drawable (window or pixmap) represents the region that can currently be drawn to. It includes the effects of clipping requested by the client application (the client clip list) and, if the drawable is a window, by other windows that are on top of the window in question. The drawn-to region for a particular DDX command is found by intersecting the composite clip list with the region that the command would render in the absence of clipping. It is this region to which the command actually renders.  
         [0062]    In X Windows, stipples are often used to tile a region with a two-color tile. The stipple is specified as a bit mask, where a 0 bit represents a pixel with the current background color and a 1 bit represents a pixel with the current foreground color. Thus, the same stipple produces incompatible tile images if either the background or foreground color is changed.  
         [0063]    The tile entry should contain enough information to distinguish between such incompatible tiles. In one embodiment, the hash value of a stipple&#39;s tile image may be computed using the resulting full-color image. If the drawn-to region does not include a full copy of the tile image, this would require an additional computational step where the stipple is converted to a full-color image. In another embodiment, the hash value may instead be computed directly from the stipple bit mask. In this case, three additional pieces of data may be required in the tile list entry:  
         [0064]    1) bStipple: true if the tile image was drawn as a one-bit-per-pixel stipple, e.g., using PolyFillRect in the FillOpaqueStippled mode; false if it was drawn as a 24-bit-per pixel pixmap.  
         [0065]    2) colorFg: foreground color  
         [0066]    3) colorBg: background color  
         [0067]    If bStipple is false, then color Fg and colorBg may be ignored because the tile image contains the complete color information. If bStipple is true, compatibility of tiles requires that the two tiles have the same foreground and background colors, in addition to the aforementioned conditions.  
         [0068]    When a tile command creates a repeating image region in a drawable, the host computer computes the repeating image portion of the drawn-to region. For most of the DDX tile commands, the repeating-image portion of the drawn-to region is just the drawn to region, but several commands require special handling: PolyFillRect in the FillStippled mode, CopyArea, CopyWindow, SaveAreas, and RestoreAreas.  
         [0069]    For PolyFillRect in the FillStippled mode, foreground pixels may be rendered and background pixels left unchanged. Thus, a repeating image may be generated if the stipple is drawn on top of a single-color fill region.  
         [0070]    For CopyArea, CopyWindow, SaveAreas, and RestoreAreas, the repeating-image portion of the drawn-to region may be computed as follows (compare FIG. 12):  
         [0071]    1) A repeating-image portion of the source drawable is the region in one of the source drawable&#39;s tile linked list entries.  
         [0072]    2) This portion is translated to the coordinates of the destination drawable.  
         [0073]    3) The translated portion is intersected with the drawn-to region of the DDX command.  
         [0074]    This process may then be repeated for each tile linked list entry in the source drawable.  
         [0075]    When a tile command creates a repeating-image in a drawable, the host may check to see if the new repeating image is compatible with any of the existing tile entries in the tile linked list. If a compatible entry is found, then the repeating-image portion of the drawn-to region for the DDX command may be added to the entry&#39;s tile region. If no compatible entry is found, a new entry may be created with the parameters of the new repeating image and the region may be set to the repeating-image portion of the drawn-to region. For more details, see FIG. 9 and its corresponding description.  
         [0076]    An embodiment of the present invention may be used in conjunction with an “RGB region” which identifies the portions of the drawable composed of RGB sub-images. For purposes of this application, an RGB image may be defined as one that is rendered from a list of pixel values (in contrast to an image rendered from a specification of a geometric figure or text characters to draw). In a drawing program, such an image may be an imported JPEG image where the pixel values are computed from the compressed JPEG data. In an X Server, this may be an image drawn with the XPutImage command, where the pixel values are provided directly by the client application. XPutImage copies the image data in an XImage structure into an area of a window or pixmap. This is the typical way to create non-repeating, many-colored images on an X server, as the other drawing primitives are typically used to draw only one or two colors at a time.  
         [0077]    While XPutImage is a specific command in the X Windows System, the term put image command may be used in the present application to refer to any rendering command used to create an RGB image.  
         [0078]    Additionally, an embodiment of the present invention may also be used in conjunction with a “fill region”, which identifies the portions of the-drawable composed of single-color regions.  
         [0079]    Repeating images are often used to create the background portion of an image. Thus it is frequent that other drawing commands create sub-images on top of the repeating image. In one embodiment of the present invention, when drawing occurs on top of an existing tile region, the drawn-to region is subtracted from the tile region. This may be termed the “disjoint” embodiment. If the present invention is used in conjunction with single-color and multi-color region tagging, then the region removed from the tile region should be added to a “miscellaneous” region or to the RGB region. This miscellaneous region tracks the region of the drawable that contains non-RGB image data drawn on top of a fill region and/or tile region. The miscellaneous region should be used when the tile image is low color (e.g. if bStipple==TRUE). Otherwise, the RGB region should be used. When a tile region becomes empty, the corresponding linked list entry can be removed from the tile linked list.  
         [0080]    In another embodiment of the present invention, the tile images are treated as underlays of the subsequent drawing. This may be termed the “underlay” embodiment. In this embodiment, only drawing commands which affect large, rectangular portions of the drawable have their drawn-to regions subtracted from the tile regions. Examples of such commands would be PolyFillRect and PutImage. The drawn-to regions of other commands are instead added to the miscellaneous region. When the image in a drawable is sent to the display computer, correct display may be ensured by sending the tile regions first, followed by the fill, miscellaneous and RGB regions.  
         [0081]    When one of the tile commands is used on a realized window, the new repeating image may be sent as a display update to the display computer. FIG. 3 is a diagram illustrating this process. In one embodiment of the present invention, each rectangle in the repeating-image portion is sent separately. For each rectangle:  
         [0082]    1) The upper left width*height pixels in the rectangles are sent to the client. This produces the hashed rectangle  300  show in FIG. 3. Although the left and top edges of these pixels may not correspond to the left and top edges of the original tile image, the image pattern will still repeat every width*height pixels. That is, every dashed rectangle  302  in FIG. 3 should contain the same image after the update is complete. If these pixels contain many colors, then an RGB codec may be used.  
         [0083]    2) A sequence of local copy commands is sent to the display computer to repeat the tile image. For best efficiency, the source of the local copy can grow by powers of two each copy step. For example, the source width*height pixels can be copied immediately to the width*height pixels to their right. This is illustrated by the upper-left drawing  304  of FIG. 3, where the hashed rectangle  300  is the source of the local copy command and the bricked rectangle  302  is the destination. In the next step, the upper-left (2*width)*height pixels can be copied to their right. This is illustrated by the upper-right drawing  306  of FIG. 3. Once the right edge of the rectangle is reached (drawing  308  in FIG. 3), the entire width*height (here width is the width of the rectangle) upper-left pixels can be used as the source for a vertical local copy (drawing  310  in FIG. 3). The vertical copies continue by powers of two until the bottom edge of the rectangle is reached (drawing  312  in FIG. 3).  
         [0084]    In another embodiment of the present invention, the host computer can use the tile tags of the screen image to determine whether some or all of the tile image already exists on the display computer&#39;s screen. If it does, then local copies can be used to paint part or all of the upper-left width*height pixels on the display computer. This is illustrated in FIG. 4, where part (the bricked rectangle  400 ) of the upper-left width*height pixels of a rectangle  402  is drawn using a local copy from a tiled rectangle (hashed rectangle  404 ) in a rectangle  406  with a compatible repeating image that was already on screen.  
         [0085]    The screen image tile tag may be updated after each rectangle is drawn. Thus, subsequent rectangles can take advantage of tile images created in drawing the previous rectangles. That is, the tile image will never need to be sent more than once no matter how many rectangles exist in the repeating-image portion of the drawn-to region. For example, FIG. 4 could illustrate a case where there are no pre-existing tiled images and a single tile command creates two tiled rectangles. In this case, drawing of the second rectangle would take advantage of the fact that the first rectangle contained part of the tile image.  
         [0086]    If the complete tile image already exists on screen, which is quite common, then the entire tile region update will comprise local copies commands that use very little bandwidth. This represents a huge savings in bandwidth over prior techniques.  
         [0087]    Thus, the present invention provides for the quick, low-cost identification of repeating images in cases where prior techniques failed to identify (e.g., screen image refreshes and CopyArea from a pixmap with a repeating image). The present invention also provides for the extremely low bandwidth transmission of the first width*height pixels of a repeating-image rectangle in cases where the tile image already exists on screen. This advantage is present even for tiled images drawn directly to a realized window, where prior techniques would recognize the new image as being tiled, but would still have to send the first width*height pixels from scratch.  
         [0088]    Additionally, in another embodiment of the present invention, a cache of tile images may be maintained on the display computer. To minimize latency, the server may manage the cache entries based on the size of the cache reported by the display computer at the time the host-client connection is formed. When a repeating image is first drawn to the screen, the server may tell the display computer to add the new tile image to its cache with a given ID number. Tile images may only be removed from the cache when cache space is needed for newer tile images (as opposed to because the tile no longer appears on the screen).  
         [0089]    Any time a repeating image is drawn to the screen, the server may tell the display computer to make the required copies of the appropriate cached tile image. For a sufficiently large tile image cache, there will frequently be situations where a cached tile image can be used even though the tile image is not present anywhere on the screen (e.g., when switching between workspaces with two different tile backgrounds). In these situations, the cache embodiment will yield lower bandwidth and faster performance than the embodiment that tries to use an existing on-screen copy of the tile image as the source for the first width*height pixels.  
         [0090]    FIGS.  5 A- 5 C are flow diagrams illustrating a method for reducing the size of a graphics image in a computer system in accordance with an embodiment of the present invention. These figures show the processing of a single DDX command in accordance with an embodiment of the present invention. These figures refer specifically to an embodiment where the Sun Ray™ server software is modified to practice the invention. One of ordinary skill in the art will recognize that this method may be modified for use with other remote display applications. In this embodiment, pixmap and screen sub-images are identified with fill, tile, and miscellaneous region tags using either the underlay or disjoint model. For better results, RGB and text tags may also be used. For simplicity, use of these regions is not shown in the figure, but one of ordinary skill in the art would recognize that they could easily be added. Also in this embodiment, several DDX commands may require special handling. Referring first to FIG. 5A, at  500 , it is determined if the command is a DestroyPixmap command. It should be noted that one of ordinary skill in the art will recognize that a destroy pixmap command, any command utilized to destroy or remove a pixmap or pixmap-like structure, could be substituted for DestroyPixmap. If it is a DestroyPixmap command, then at  502 , the regions in the pixmap&#39;s fill and tile linked lists are uninitialized and the fill and tile linked lists are destroyed at  504 . At  506 , the miscellaneous region is uninitialized and at  508 , standard DDX pixmap destruction may be performed. If it is not a DestroyPixmap command, then at  510  the standard DDX routine for the command is called. For DDX commands that draw repeating images to a realized window (e.g., PaintWindowBackground, PaintWindowBorder, opaque PolyFillRect with fillStyle equal to FillTiled or FillOpaqueStippled), the standard processing can be enhanced using the methods of FIGS.  3 - 4  to send the repeating images.  
         [0091]    At  512  it is determined if the command is a CopyArea from a pixmap to a realized window. If not, then it is determined at  514  if the command is a RestoreAreas from a backing store pixmap to a realized window. If neither is true, the system may simply perform standard Sun Ray™ post-processing of the command at  516 . For example, if the command renders to a realized window, then the Sun Ray™ software sends the appropriate screen update information to the display computer.  
         [0092]    If the command is a CopyArea or RestoreAreas from a pixmap to a realized window, then at  518  the update resulting from the pixmap to window copy may be sent. This is described in more detail in FIG. 6. FIG. 6 is a flow diagram illustrating the sending of an update resulting from copying a pixmap to a window in accordance with an embodiment of the present invention. At  600 , the drawn-to region is computed. At  602 , it is determined if there are more entries in the pixmap&#39;s tile linked list. If so, then at  604  the next entry&#39;s tile region is used to compute the portion of the drawn-to region that contains tiles compatible with that which is specified in the linked list entry. The region will contain zero rectangles if empty or more than zero if it is non-empty. Then it is determined if there are any more rectangles in the tile portion at  606 . If so, then the upper left width * height pixels in the rectangle are sent to the display computer at  608  (also shown in FIGS.  3 - 4 ). Then at  610 , a sequence of local copy commands are sent to the display computer to repeat the tile image (also shown in FIG. 3). This repeats until each rectangle is sent for each region defined by each entry in the fill linked list.  
         [0093]    Once that is complete, at  612  it is determined if there are more entries in the pixmap&#39;s fill linked list. If so, then at  614  the next entry&#39;s fill region is used to compute the portion of the drawn-to region that contains fills with the color specified in the linked list entry. The region will contain zero rectangles if empty or more than zero if it is non-empty. Then it is determined if there are any more rectangles in the fill portion at  616 . If so, then the 1-color protocol command for the next rectangle is sent, using the color of the linked list entry at  618 . This repeats until each rectangle is sent for each region defined by each entry in the fill linked list. Once that is complete, at  620  the pixmap&#39;s miscellaneous region may be used to compute the miscellaneous portion of the drawn-to region.  
         [0094]    At  622 , it is determined if the miscellaneous portion is empty. If not, it may be sent using the ordinary Sun Ray™ method at  624 .  
         [0095]    Referring back to FIG. 5A, at  520 , it is determined if the command is a CreatePixmap command. It should be noted that one of ordinary skill in the art will recognize that a create pixmap command, any command utilized to create a pixmap or pixmap-like structure, could be substituted for CreatePixmap. If the command is a CreatePixmap command, then at  523  an empty fill linked list and an empty tile linked list are attached to the pixmap and at  524 , the miscellaneous region is attached to the pixmap and initialized to empty.  
         [0096]    If the command is not a CreatePixmap command, then at  526  it is determined if the command is an opaque PolyFllRect to a tagged drawable. This may represent case 3 of the commands requiring special treatment as described above (opaque for PolyFillRect means full planemask and an alu of GXcopy). It should be noted that one of ordinary skill in the art will recognize that an opaque poly fill rectangle command, any command utilized to overwrite a rectangle with a single color or a tiled pattern, could be substituted for PolyFillRect. PolyFillRect commands are frequently used to set a region to the background color or pattern. If the command is an opaque PolyFillRect command, then at  528  the drawn-to region is computed.  
         [0097]    Then at  530 , the drawn-to region is subtracted from the miscellaneous region. At  532 , the drawn-to region is subtracted from the fill and tile linked lists. This is described in more detail in FIG. 7. FIG. 7 is a flow diagram illustrating a method for subtracting the drawn-to region from the fill and tile linked lists in accordance with an embodiment of the present invention. At  700 , it is determined if there are more fill linked list entries. If not, then the process continues at step  710 . If so, then at  702 , the drawn-to region is subtracted from the next linked list entry&#39;s region. At  704 , it is determined if the entry&#39;s region is empty. If so, the entry&#39;s region is uninitialized at  706  and removed from the linked list at  708 . Then, at  710 , it is determined if there are more tile linked list entries. If not, then the process ends and it returns to FIG. 5B. If so, then at  712 , the drawn-to region is subtracted from the next entry&#39;s region. At  714 , it is determined if the region&#39;s entry is empty. If so, the entry&#39;s region is uninitialized at  716  and removed from the linked list at  718 .  
         [0098]    Referring back to FIG. 5B, at  534 , it is determined if the fill style is FillSolid. If so, then at  536 , the drawn-to region is submitted to the fill linked list. This is described in more detail in FIG. 8. FIG. 8 is a flow diagram illustrating a method for submitting a region to a fill linked list in accordance with an embodiment of the present invention. At  800 , it is determined if an existing linked list entry has the same color as the submitted region. If so, then at  802 , the submitted region is added to that linked list entry. If not, a new entry is created at the end of the linked list at  804 . Then at  806 , the color of the new entry is set to the color of the submitted region. At  808 , the region of the new entry is set to the submitted region. Consider an opaque PolyFillRect in the FillSolid fill style in the same color as an existing fill linked list entry: the drawn-to region is subtracted from the entry&#39;s region at  532  and added back in at  802 . This may appear somewhat redundant, but is preferred for programming simplicity in that for the implementation described, the processing in  532  does not depend on the fill style or fill color.  
         [0099]    Referring back to FIG. 5B, if the fill style is not FillSolid, then at  538  it is determined if FillStyle is equal to FillTiled ot FillOpaqueStippled. If so, then this represents case 3a of the commands requiring special treatment as described above, and at  540  the drawn-to region is submitted to the tile linked list. This is described in more detail in FIG. 9. FIG. 9 is a flow diagram illustrating a method for submitting a region to a tile linked list in accordance with an embodiment of the present invention. At  900 , it is determined if the repeating image region is compatible with any existing tile entries in the tile linked list. If so, then at  902 , the repeating image region is added to that linked list entry. If not, a new entry is created at the end of the linked list at  904 . Then at  906 , the parameters of the new entry is set to the parameters of the repeating image region. At  908 , the region of the new entry is set to the repeating image region.  
         [0100]    Referring back to FIG. 5B, if FillStyle is not equal to either FillTiled or FillOpaqueStippled, then at  542  the drawn-to region is added to the miscellaneous region. This handles case 3b of the commands requiring special treatment as described above. Alternatively, one could specifically identify FillStippled on top of a single-color fill as a repeating image using the fill tags of the destination drawable. In this case, the drawn-to region intersected with an existing fill tag for the destination drawable yields a single repeating-image region. Each different fill tag with a non-zero intersection results in a different tile tag. For simplicity, this alternative is not illustrated in the figure.  
         [0101]    If the command is not an opaque PolyFillRect to a tagged drawable, then it may represent case 4 of the commands requiring special treatment as described above, and at  544  it is determined if the command is an opaque CopyArea from a tagged drawable to a tagged drawable. If so, then at  546 , a tag copy from source to destination is performed. This is described in more detail in FIG. 10. FIG. 10 is a flow diagram illustrating performing tag copy from a source to a destination in accordance with an embodiment of the present invention. At  1000 , if necessary, an unmodified copy of the source drawable&#39;s region tags is made. This is necessary if the source and destination drawable are the same, and if the source and destination rectangles overlap. Use of the unmodified copy in the remainder of the execution prevents confusion resulting from changes to the destination drawable&#39;s region tags during the execution of FIG. 10. At  1002 , the drawn-to region is computed. Then at  1004 , the drawn-to region is subtracted from the destination drawable miscellaneous region. Then at  1006 , the drawn-to region is subtracted from the destination drawable&#39;s fill linked list according to the procedure in FIG. 7.  
         [0102]    At  1008 , it is determined if there are more entries in the source drawable&#39;s fill linked list. If so, then at  1010 , the next entry&#39;s fill region is used to compute the fill portion of the drawn-to region for the corresponding fill color. Then at  1012 , the fill portion is submitted to the destination drawable&#39;s fill linked list according to the procedure in FIG. 8. Then back to  1008  until there are no more entries in the fill linked list. At  1014 , it is determined if there are more entries in the source drawable&#39;s tile linked list. If so, then at  1016 , the next entry&#39;s tile region is used to compute the repeating image portion of the drawn-to region for the corresponding tile image. Then at  1018 , the tile portion is submitted to the destination drawable&#39;s tile linked list according to the procedure in FIG. 9. This is repeated until there are no more entries in the tile linked list.  
         [0103]    At  1020 , the source drawable&#39;s miscellaneous region is used to compute the miscellaneous portion of the drawn-to region. At  1022 , the miscellaneous portion is added to the destination drawable&#39;s miscellaneous region.  
         [0104]    Referring back to FIG. 5B, 546 may also be performed if it is determined if that the command is a CopyWindow at  548  (which also represents case 4 of the commands requiring special treatment as described above), a RestoreArea from a backing store pixmap to a tagged window at  550  (which represents case 5 of the commands requiring special treatment as described above), or a SaveAreas from a tagged window to a backing store pixmap at  552  (which represents case 6 of the commands requiring special treatment as described above).  
         [0105]    At  554 , it is determined if the command is a PaintWindowBackground to a tagged window. If so, then at  556  it is determined if the window has backgroundState equal to BackgroundPixmap. If so, then this represents case 1 of the commands requiring special treatment as described above, and at  562  the drawn-to region is computed. Then at  564 , the drawn-to region is subtracted from the miscellaneous region. At  566 , the drawn-to region may be subtracted from the fill and tile linked lists according to the procedure outlined in FIG. 7 and the corresponding text. Then, at  568 , the repeating image region may be submitted to the tile linked list according to the procedure outlined in FIG. 9 and the corresponding text.  562 - 568  may also be performed if at  558  it is determined that the command is a PaintWindowBorder for a tagged window, and at  560  it is determined that the window has borderIsPixel equal to false, which represents case 2 of the command requiring special treatment as described above.  
         [0106]    At  570 , it is determined if the command draws to a tagged drawable. If so, then at  572 , the drawn-to region is computed. Then at  574 , the drawn-to region is submitted to the miscellaneous region according to the procedure in FIG. 11 or FIG. 12. FIG. 11 is a flow diagram illustrating a method for submitting a region to the miscellaneous region under the “underlay” embodiment. At  1100 , if the region tags include an RGB region, the destination drawable&#39;s RGB region is first subtracted from the submitted region. This is included for the convenience of implementers using an RGB region as a component of the region tags. If an RGB region is not being used,  1100  need not be executed. Then at  1102 , the remainder is added to the destination drawable&#39;s miscellaneous region. FIG. 12 is a flow diagram illustrating a method for submitting a region to the miscellaneous region under the “disjoint” embodiment. At  1200 , the submitted region is first subtracted from the fill and tile linked lists according to the procedure of FIG. 7. Then at  1202 , in implementations including an RGB region, the destination drawable&#39;s RGB region is subtracted from the submitted region. Then the remainder is added to the destination drawable&#39;s miscellaneous region at  1204 .  
         [0107]    [0107]FIG. 13 is a flow diagram illustrating a method for reducing the size of a graphics image in a computer system in accordance with an embodiment of the present invention. In this embodiment, a certain class of image data (e.g., tile regions) is singled out for special compression, using a compression mode appropriate for the class of image data. A set of one or more drawing commands (e.g. PaintWindowBackground for a window with borderIsPixel==FALSE) may be identified as creating sub-images of this class.  
         [0108]    At  1300 , a tile command executed to create a portion of the graphics image in a destination drawable is recorded. Command recording may or may not be limited to commands from the drawing commands identified above. This drawing command may also be one that is executed during the rendering of the graphics image. The graphics image may be created in a destination drawable. At  1302 , the tile command may be used to update a data structure associated with the destination drawable with information. This may include using information about the region drawn to by the drawing command to update the data structure. This data structure may be associated with the destination drawable. A private data field, such as a devPrivates field, may be used to attach the data structure to the destination drawable. While use of the X Windows devPrivates facility would be one method of associating a data structure with the drawable, there are other possible choices for the method of associating the two.  
         [0109]    The information in the data structure may enable determination of what portion of the image contains sub-images of the specified image class. One example data structure would be a linked list identifying regions of repeating images. In one embodiment of the present invention, an X region resource is used to store the region of the destination drawable which contains sub-images of the specified image class. At  1304 , one or more sub-images of the graphics image may be compressed using a compression scheme, each of the sub-images having a location, the location and the compression scheme for each of the sub-images based on the information in the data structure. It should be noted that the compressor does not necessarily need to know exactly which drawing commands were executed. It only needs to know what sub-images of the image should be compressed with a given compression scheme.  
         [0110]    The compressor may use the information in the data structure to find the location within the image of sub-images of the specified image class. The compressor may compress these sub-images using a compression scheme selected for the specified image class. If the region covered by the sub-images is represented as a set of one or more rectangles, as is the case for the X region resource, each rectangle may be compressed and sent separately.  
         [0111]    [0111]FIG. 14 is a flow diagram illustrating a method for reducing the size of a graphics image in a computer system in accordance with another embodiment of the present invention. At  1400 , a tile command executed to create a portion of the graphics image in a destination drawable may be detected. Command detection may or may not be limited to commands from the drawing commands identified above. This drawing command may also be one that is executed during the rendering of the graphics image. The graphics image may be created in a destination drawable. At  1402 , a drawn-to region may be computed from the tile command. At  1404 , the drawn-to region may be subtracted from a miscellaneous region. At  1406 , the drawn-to region may be subtracted from one or more fill and/or tile regions. At  1408 , the drawn-to region may be added to one of one or more tile regions, each of the tile regions corresponding to a repeating image. At  1410 , the tile image (i.e., the upper width*height pixels in the rectangle) may be compressed. At  1412 , the tile image may be transmitted for display or storage. At  1414 , a sequence of local copy commands may be transmitted to repeat the tile image.  
         [0112]    [0112]FIG. 15 is a block diagram illustrating an apparatus for reducing the size of a graphics image in a computer system in accordance with an embodiment of the present invention. In this embodiment, a certain class of image data (e.g., tile regions) is singled out for special compression, using a compression mode appropriate for the class of image data. A set of one or more drawing commands (e.g. PaintWindowBackground for a window with borderIsPixel==FALSE) may be identified as creating sub-images of this class.  
         [0113]    A tile command recorder  1500  may record a tile command executed to create a portion of the graphics image in a destination drawable. Command recording may or may not be limited to commands from the drawing commands identified above. This drawing command may also be one that is executed during the rendering of the graphics image. The graphics image may be created in a destination drawable. A data structure updater  1502  coupled to the tile command recorder  1500  may use the tile command to update a data structure associated with the destination drawable with information. This may include using information about the region drawn to by the drawing command to update the data structure. This data structure may be associated with the destination drawable. A private data field, such as a devPrivates field, may be used to attach the data structure to the destination drawable. While use of the X Windows devPrivates facility would be one method of associating a data structure with the drawable, there are other possible choices for the method of associating the two.  
         [0114]    The information in the data structure may enable determination of what portion of the image contains sub-images of the specified image class. One example data structure would be a linked list identifying regions of repeating images. In one embodiment of the present invention, an X region resource is used to store the region of the destination drawable which contains sub-images of the specified image class. An image compressor  1504  coupled to the data structure updater  1502  may compress one or more sub-images of the graphics image using a compression scheme, each of the sub-images having a location, the location and the compression scheme for each of the sub-images based on the information in the data structure. It should be noted that the compressor does not necessarily need to know exactly which drawing commands were executed. It only needs to know what sub-images of the image should be compressed with a given compression scheme.  
         [0115]    The compressor may use the information in the data structure to find the location within the image of sub-images of the specified image class. The compressor may compress these sub-images using a compression scheme selected for the specified image class. If the region covered by the sub-images is represented as a set of one or more rectangles, as is the case for the X region resource, each rectangle may be compressed and sent separately.  
         [0116]    [0116]FIG. 16 is a block diagram illustrating an apparatus for reducing the size of a graphics image in a computer system in accordance with another embodiment of the present invention. A tile command detector  1600  may detect a tile command executed to create a portion of the graphics image in a destination drawable. Command detection may or may not be limited to commands from the drawing commands identified above. This drawing command may also be one that is executed during the rendering of the graphics image. The graphics image may be created in a destination drawable. A drawn-to region computer  1602  coupled to the tile command detector  1600  may compute a drawn-to region from the tile command. A drawn-to region to miscellaneous region remover  1604  coupled to the tile command detector  1600  and to the drawn-to region computer  1600  may subtract the drawn-to region from a miscellaneous region. A drawn-to region to fill and tile region remover  1606  coupled to the tile command detector may subtract the drawn-to region from one or more fill and/or tile regions. A tile region adder  1608  coupled to the tile command detector  1600  and to the drawn-to region computer  1602  may add the drawn-to region to one of one or more tile regions, each of the tile regions corresponding to a repeating image. A memory or memories  1610  coupled to the tile region adder  1608 , the drawn-to region to miscellaneous region remover  1604  and to the drawn-to region to fill and tile region remover  1606  may store the various regions. A tile image transmitter  1612  coupled to the memory  1610  may transmit the tile image for display or storage. A local copy command transmitter  1614  coupled to the tile image transmitter  1612  may transmit a sequence of local copy commands to repeat the tile image.  
         [0117]    While embodiments and applications of this invention have been shown and described, it would be apparent to those skilled in the art having the benefit of this disclosure that many more modifications than mentioned above are possible without departing from the inventive concepts herein. The invention, therefore, is not to be restricted except in the spirit of the appended claims.