Abstract:
A method and apparatus are provided for implementing reduced video stream bandwidth requirements when remotely rendering a complex computer graphics scene. Complexity of a scene is reduced at a server, prior to rendering a video stream that comprises the scene and transmitting the video stream to a client. Reducing the complexity of a scene at the server includes adjusting predefined scene configuration parameters. The order and degree to which predefined scene configuration parameters are adjusted is based upon a required stream bandwidth reduction to be made.

Description:
[0001]    This application is a continuation application of Ser. No. 14/177,436 filed Feb. 11, 2014. 
     
    
     FIELD OF THE INVENTION 
       [0002]    The present invention relates generally to the data processing field, and more particularly, relates to method and apparatus for implementing reduced video stream bandwidth requirements when remotely rendering a complex computer graphics scene. 
       DESCRIPTION OF THE RELATED ART 
       [0003]    Cloud gaming is a growing business where much of the processing that was historically done on individual computers is now hosted instead in a cloud environment. In this environment, scene rendering is done by the cloud and then streamed to a host system where a user interacts with the video stream as though it were rendered locally. 
         [0004]    The network bandwidth required for such an environment is significant, and any disruptions to the video stream can be catastrophic for the game appeal. Therefore, maintaining a consistent video stream is crucial for the success of any cloud-based game. 
         [0005]    Typically when a network or cloud resource bottleneck is reached, the stream is compressed using standard video compression algorithms to reduce the amount of data required to stream. This results in a pixilated, degraded image, reducing the quality of the gaming experience. 
         [0006]    A need exists for an efficient and effective method and apparatus for implementing reduced video stream bandwidth requirements when remotely rendering a complex computer graphics scene. 
       SUMMARY OF THE INVENTION 
       [0007]    Principal aspects of the present invention are to provide a method and apparatus for implementing reduced video stream bandwidth requirements when remotely rendering a complex computer graphics scene. Other important aspects of the present invention are to provide such method and apparatus substantially without negative effects and that overcome many of the disadvantages of prior art arrangements. 
         [0008]    In brief, a method and apparatus are provided for implementing reduced video stream bandwidth requirements when remotely rendering a complex computer graphics scene. Complexity of a scene is reduced at a server, prior to rendering a video stream that comprises the scene and transmitting the video stream to a client. Reducing the complexity of a scene at the server includes adjusting predefined scene configuration parameters. The order and degree to which predefined scene configuration parameters are adjusted is based upon a required stream bandwidth reduction to be made. 
         [0009]    In accordance with features of the invention, the order and degree to which predefined scene configuration parameters adjustments are learned based upon history data, or are based upon preferences by each user. 
         [0010]    In accordance with features of the invention, the predefined scene configuration parameters include removing background objects to reduce complexity. Other predefined scene configuration parameters include one or more of disabling water ripple and reflections; disabling clouds; reducing draw distance to limit number of scene objects; adjusting anti-aliasing and texture complexity. 
         [0011]    In accordance with features of the invention, a controller determines the performance of the transmission of the video stream for the previous packet and stores the change in performance to history data. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0012]    The present invention together with the above and other objects and advantages may best be understood from the following detailed description of the preferred embodiments of the invention illustrated in the drawings, wherein: 
           [0013]      FIG. 1  is a block diagram of an example computer system for implementing reduced video stream bandwidth requirements when remotely rendering a complex computer graphics scene in accordance with a preferred embodiment; 
           [0014]      FIG. 2  is a block diagram of the system of  FIG. 1  illustrating system operations with example input scene data and example output scene data in accordance with a preferred embodiment; 
           [0015]      FIG. 3  is a block diagram of a memory of the system of  FIG. 1  illustrating system operations with example input scene data and example action data in accordance with a preferred embodiment; 
           [0016]      FIG. 4  illustrates system operations with example history data in accordance with a preferred embodiment; 
           [0017]      FIG. 5  is a flow chart illustrating example operations for implementing reduced video stream bandwidth requirements when remotely rendering a complex computer graphics scene in accordance with the preferred embodiment; 
           [0018]      FIG. 6  is a block diagram illustrating a computer program product in accordance with the preferred embodiment. 
       
    
    
     DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS  
       [0019]    In the following detailed description of embodiments of the invention, reference is made to the accompanying drawings, which illustrate example embodiments by which the invention may be practiced. It is to be understood that other embodiments may be utilized and structural changes may be made without departing from the scope of the invention. 
         [0020]    The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises” and/or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. 
         [0021]    In accordance with features of the invention, a method and apparatus are provided for implementing reduced video stream bandwidth requirements when remotely rendering a complex computer graphics scene. 
         [0022]    Having reference now to the drawings, in  FIG. 1 , there is shown a computer system embodying the present invention generally designated by the reference character  100  for implementing reduced video stream bandwidth requirements when remotely rendering a complex computer graphics scene in accordance with the preferred embodiment. Computer system  100  includes one or more processors  102  or general-purpose programmable central processing units (CPUs)  102 , #1-N. As shown, computer system  100  includes multiple processors  102  typical of a relatively large system; however, system  100  can include a single CPU  102 . Computer system  100  includes a cache memory  104  connected to each processor  102 . 
         [0023]    Computer system  100  includes a system memory  106 . System memory  106  is a random-access semiconductor memory for storing data, including programs. System memory  106  is comprised of, for example, a dynamic random access memory (DRAM), a synchronous direct random access memory (SDRAM), a current double data rate (DDRx) SDRAM, non-volatile memory, optical storage, and other storage devices. 
         [0024]    I/O bus interface  114 , and buses  116 ,  118  provide communication paths among the various system components. Bus  116  is a processor/memory bus, often referred to as front-side bus, providing a data communication path for transferring data among CPUs  102  and caches  104 , system memory  106  and I/O bus interface unit  114 . I/O bus interface  114  is further coupled to system I/O bus  118  for transferring data to and from various I/O units. 
         [0025]    As shown, computer system  100  includes a storage interface  120  coupled to storage devices, such as, a direct access storage device (DASD)  122 , and a CD-ROM  124 . Computer system  100  includes a terminal interface  126  coupled to a plurality of terminals  128 , #1-M, a network interface  130  coupled to a network  132 , such as the Internet, local area or other networks, shown connected to another separate computer system  133 , and a I/O device interface  134  coupled to I/O devices, such as a first printer/fax  136 A, and a second printer  136 B. 
         [0026]    I/O bus interface  114  communicates with multiple I/O interface units  120 ,  126 ,  130 ,  134 , which are also known as I/O processors (IOPs) or I/O adapters (IOAs), through system I/O bus  116 . System I/O bus  116  is, for example, an industry standard PCI bus, or other appropriate bus technology. 
         [0027]    System memory  106  stores input scene data  140 , output scene data  142 , action data  144 , history data  146 , video stream  148 , and a controller  150  for implementing reduced video stream bandwidth requirements when remotely rendering a complex computer graphics scene in accordance with the preferred embodiments. 
         [0028]    In accordance with features of the invention, by taking into account the game settings and scene makeup, a number of options advantageously are utilized to simplify the scene prior to rendering. For example, a densely populated forest scene might include hundreds of highly-detailed trees. Most compression algorithms are not generally successful in reducing the size of a scene like this significantly, due to the detail of each tree. However, if the trees are removed or replaced with much simpler trees, the rendered image would be far simpler without reducing the resolution, color depth, or failing to render in a necessary amount of time. 
         [0029]    Referring to  FIG. 2 , there are shown system operations designated by the reference character  200  with more detailed example input scene data  140  and example output scene data  142  in the computer system  100  in accordance with a preferred embodiment. System operations  200  illustrate example input scene data  140  including a plurality of objects including object A,  210 , object B,  212 , object C,  214 , and object D,  216 . Each object A,  210 , object B,  212 , object C,  214 , and object D,  216  includes a respective flag  220 ,  222 ,  224 , and  226  produced by the controller  150 , for example, using action data  144 , and history data  146 . System operations  200  show simplified example output scene data  142  including object A,  210 , and object B,  212 . The simplified example output scene data  142  are produced by the controller  150  using the flags  220 ,  222 ,  224 , and  226 . In the illustrated simplified example output scene data  142 , object C,  214 , and object D,  216  have been removed so that the video stream  148  being generated is a fundamentally simpler scene, requiring less bandwidth to be transmitted over network  132  in accordance with the preferred embodiments. 
         [0030]    As shown in  FIG. 2 , the client computer  133  includes a memory  250 , a processor  252 , and a display device  254 . The memory  250  of client computer  133  includes a display controller and receives and stores the video stream  148 . The client computer  133  receives and displays the video stream  148 , which required less bandwidth and was produced without relying on harsher compression or reduced color depth in accordance with the preferred embodiments. 
         [0031]    In accordance with features of the invention, a fundamentally simpler scene requiring less bandwidth for the video stream  148  to be transmitted is generated, for example, by removing background objects from the input scene data  140  to reduce complexity. In addition other options optionally utilized to simplify the scene prior to rendering include various possible scene characteristics can be selectively reduced or removed to simplify the scene and reduce complexity, such as disabling water ripple and reflections, disabling clouds, reducing draw distance to limit number of scene objects, and adjusting anti-aliasing and texture complexity. 
         [0032]    Referring to  FIG. 3 , there are shown system operations designated by the reference character  300  of the memory  106  with example input scene data  140  and example action data  144 . The example input scene data  140  including the plurality of objects including object A,  210 , object B,  212 , object C,  214 , and object D,  216  with the respective flag  220 ,  222 ,  224 , and  226  is shown with example action data  144  in accordance with a preferred embodiment. System operations  300  illustrate example action data  144  including a respective background lever  302  together with a respective performance level  304  including performance 1 less than 1 Mb/Sec; performance 2 is between 1 Mb/Sec and 3 Mb/Sec, and performance 3 is greater than 3 Mb/Sec. For example, as shown with a background level 1 and at performance 1, the action equals anti-aliasing; at performance 2, the action equals texturing; and at performance 3, the action equals none. For example, as shown with a background level 2 and at performance 1, the action equals removal; at performance 2, the action equals anti-aliasing; and at performance 3, the action equals none. For example, as shown with a background level 3 and at performance 1, the action equals removal; at performance 2, the action equals removal; and at performance 3, the action equals none. 
         [0033]    Referring also to  FIG. 4 , there are shown system operations designated by the reference character  400  of the example history data  146  in accordance with a preferred embodiment. The example history data  146  includes a packet ID  402 , an action  404 , and a result  406 . For example, as shown with the packet ID  402  of a first packet 1, and a second packet 2, the action equal none for performance levels 1, 2, and 3 with the result  406  of a performance decrease 5% for packet 1 and of a performance decrease 25% for packet 2. For example, as shown with the packet ID  402  of a first packet 3, and the action  404  equals anti-aliasing for performance level 1, and equals removal for performance levels 2, and 3 with the result  406  of a performance increase 15%. For example, as shown with the packet ID  402  of a first packet 4, and the action equal none for performance levels 1, 2, and 3 with the result  406  of a performance decrease 35%. For example, as shown with the packet ID  402  of a first packet 5, and the action  404  equal texturing for performance level 1, and equals anti-aliasing for performance level 2, and equals removal for performance level 3 with the result  406  of a performance increase 40%. 
         [0034]    Referring also to  FIG. 5 , there are shown example operations for implementing reduced video stream bandwidth requirements when remotely rendering a complex computer graphics scene in accordance with the preferred embodiment starting at a block  500 . As indicated in a block  502 , the controller  150  receives action data  144  and identification of input scene data  140  from the client, for example to configure preferences by each user. As indicated in a decision block  504 , for all packets to be transmitted, a current packet is set to the next packet as indicated in a block  506 , until the operation return with all packets transmitted as indicated in a block  508 . For processing each packet, the controller  150  determines the performance of the previous packet and stores the change in performance to the history data  146  as indicated in a block  510 . As indicated in a block  512 , for each background level  302  of each object in the input scene data  140 , the controller selects the action  404  from the action data specified by the combination of performance and the background level or selects the actions that had the highest historical performance increase; and the controller stores the background levels and assigned actions  404  to the history data  146 . As indicated in a block  514 , the controller  150  creates output scene data  142 , for each object from the input scene data  140 ; the controller  150  performs the selected action that is assigned to the flag of the object to create the objects in the output scene data  142 . As indicated in a block  516 , the controller renders the output scene data into the video stream  148  and transmits the video stream to the client  133 , which receives and displays the video stream. 
         [0035]    Referring now to  FIG. 6 , an article of manufacture or a computer program product  600  of the invention is illustrated. The computer program product  600  is tangibly embodied on a non-transitory computer readable storage medium that includes a recording medium  602 , such as, a floppy disk, a high capacity read only memory in the form of an optically read compact disk or CD-ROM, a tape, or another similar computer program product. Recording medium  602  stores program means  604 ,  606 ,  608 , and  610  on the medium  602  for carrying out the methods for implementing reduced video stream bandwidth requirements when remotely rendering a complex computer graphics scene of the preferred embodiment in the system  100  of  FIG. 1 . 
         [0036]    A sequence of program instructions or a logical assembly of one or more interrelated modules defined by the recorded program means  604 ,  606 ,  608 , and  610 , direct the system  100  for implementing reduced video stream bandwidth requirements when remotely rendering a complex computer graphics scene of the preferred embodiment. 
         [0037]    While the present invention has been described with reference to the details of the embodiments of the invention shown in the drawing, these details are not intended to limit the scope of the invention as claimed in the appended claims.