SUPPLY OF VIDEO PRESENTATIONS OVER NETWORKS

Abstract: The present invention relates to a method of producing a presentation for transmission over a data network using streaming. The presentation comprises a relatively low bandwidth first clip followed by a relatively high bandwidth second clip. The method of producing the presentation comprises the steps of establishing a nominal maximum transmission bitrate, determining the bandwidth requirement of the first clip and allocating spare bandwidth between this and the nominal maximum bitrate for buffering of the second clip. The values of the spare bandwidth for the first clip, the duration of the first clip and the duration of the second clip are used to calculate an indication of the optimum number of bits into which the second clip can be encoded for streaming.

Description: Supply Of Video Presentations Over Networks This invention relates to the supply of video presentations over networks such as the Internet or corporate intranets. The invention is of particular relevance to the supply of video presentations over networks where bandwidth is limited. It is now commonplace for video presentations to be made available over computer networks for display on personal computers and the like. Presentations may be viewed, for example, in browsers such as Microsoft's Internet Explorer™ or Netscape's Navigator™ which access information on the Internet or on an intranet. However, the bandwidths of networks are frequently limited. Domestic users may access the Internet through a modem which may achieve a 28k to 5βk connection. Even dual channel ISDN connections will only achieve 128k. It is impossible to provide live, high quality, video using such bandwidth. One way of dealing with the problem is to download a video file over the network to the client terminal before playing it. Video files can be compressed using techniques in accordance with the MPEG standard, for example, to decrease download time. However, the file will still be very large even for less than a minute of video and can take a very long time to download. In some cases, using a domestic modem connection to the Internet, it can take several minutes to download a file that will play for only a few seconds. An alternative approach is to stream the video presentation over the network to the client terminal, for example using technology from RealNetworks such as RealPlayer™. This enables the presentation to be viewed in approximately real time. There are however problems with streaming technology. One of these is that before a video clip can be started, there is usually a delay whilst buffering of data takes place during a "preroll" phase. This causes difficulties with multimedia presentations, where a video clip is only one of a number of types of media that will be played. The delay whilst buffering of the video takes place will lead to a disjointed presentation. For this reason it has been proposed that a streaming presentation can include a text, audio or other clip with a low bandwidth requirement, before the video clip. The spare bandwidth whilst the initial clip is being played can be used for the preroll of the video clip, so that when the initial clip ends there will be a seamless transition to the video clip. In the design of a typical streaming presentation, the target bandwidth is chosen and then a bitrate suitable for transmission. For example, if the target bandwidth is 28k for a presentation suitable for domestic users accessing the Internet over a modem, then the bitrate suitable for transmission may be set at 20k to ensure satisfactory streaming. The presentation may be organised using an authoring tool which enables multimedia clips to be combined. Typically, this would use SMIL, Synchronised Multimedia Integration Language. By grouping clips played in sequence within a SMIL <par> tag, it is possible to create smooth transitions between the clips. The system takes advantage of unused bandwidth to send a clip's preroll before playback. An aspect of the present invention is concerned with improving the efficiency of using available bandwidth in a streaming presentation. In accordance with this aspect of the present invention, there is provided a method of producing a presentation for transmission over a data network using streaming, the presentation comprising a relatively low bandwidth first clip followed by a relatively high bandwidth second clip, comprising the steps of establishing a nominal maximum transmission bitrate, determining the bandwidth requirement of the first clip and allocating spare bandwidth between this and the nominal maximum bitrate for buffering of the second clip, and using the values of the spare bandwidth for the first clip, the duration of the first clip and the duration of the second clip to calculate an indication of the optimum-number of bits into which the second clip can be encoded for streaming. With existing systems, the bitrate for transmitting a high bandwidth clip such as video would be the nominal maximum bitrate. The number of bits into which the video clip can be encoded is that determined from the length of the video clip and the nominal maximum bitrate. Preroll of the second clip whilst the first clip is transmitted will smooth the transition between clips. In accordance with the invention, the amount of the second clip which is transmitted during the preroll is taken into account to determine the total amount of information that can be transmitted, taking into account the duration of the second clip. This can be expressed as the amount of data that will be transmitted, and / or as a pseudo increased bandwidth to be allocated to the video clip. In practice, a target bandwidth would be set, depending upon the facilities of the intended recipient. For example, if a presentation was intended for domestic users with a 28k modem connection, then that would be the target bandwidth. There would then be calculated a bitrate suitable for transmission, such as 20 kbps, and this would be the nominal maximum bitrate for the purposes of the invention. If a first clip is text, then a low bitrate is required, such as 1 kbps. That leaves spare bandwidth of 19 kbps. If the duration of the text clip is ten seconds, then there is a total of 190 kb available for data in respect of the second clip. If the duration of the second clip is thirty seconds, then the total number of bits that can be transmitted is 20 kbps multiplied by 30 seconds, i.e. 600 kb, plus 190 kb from the preroll period, making a total of 790 kb. This can be expressed in terms of a "virtual" increased coding bitrate of 790 / 30 , i.e. 26.3 kbps for the video clip. This represents an increase of over 30% compared to the normal 20 kbps bitrate that would be allocated to the video clip By means of the invention, the advantages of preroll during an initial low bandwidth clip are maintained, but the bandwidth available for transmission of the second clip is effectively increased dynamically to optimise the design of the presentation. The higher the bandwidth available, the more information that can be transmitted and hence increased quality of the clip that can be played. Of course, the bitrate used does not increase when averaged throughout an entire transmission, but it is possible to allocate a higher than normal bitrate to the video clip for the portion of the transmission that contains the video clip. In practice this will be done dynamically and automatically by a software system used to organise the clips in the design of the presentation . The "first clip" referred to may not be a single clip. It could comprise a number of clip components streamed sequentially or simultaneously. For example a text clip component could be transmitted together with an audio clip component. In such a case the spare bandwidth would be reduced. Alternatively, the text could be followed by the audio, thus providing a longer total duration and an overall increase in the preroll data that can be transmitted, although the spare bandwidth during audio streaming would be less than during text streaming. There may be any combination of clips provided that there is a period of spare bandwidth which can be used to preroll the second clip and to provide an increased "virtual" bitrate for the second clip above the nominal maximum. Viewed from another aspect, the present invention provides computer software which when running on data processing means will establish an environment for producing a presentation for transmission over a data network using streaming, the presentation comprising a relatively low bandwidth first clip followed by a relatively high bandwidth second clip, -the software enabling the establishment of a nominal maximum transmission bitrate, and determining the bandwidth requirement of the first clip, allocating spare bandwidth between this and the nominal maximum bitrate for buffering of the second clip, and using the values of the spare bandwidth for the first clip, the duration of the first clip and the duration of the second clip to calculate an indication of the optimum number of bits into which the second clip can be encoded for streaming. In use of the software, an environment is created in which a user selects the clips for the presentation. The various clips are arranged in the desired order using a graphical user interface. The clips could be of any type, such as relatively low bandwidth text, audio or still images. However, the advantage of the software in accordance with the invention is that when a high bandwidth video clip is combined with a preceding low bandwidth clip, the bitrate for the video clip can be optimised. The software prepares the necessary codes to enable preroll of the video clip and calculates dynamically and automatically the "virtual" bitrate that can be used for the video clip. Thus, the calculation of the video encoding bitrate is dynamic to the environment of the clip, thus taking advantage of the virtual spare bandwidth rather than relying upon a static average value. The software calculates the bitrate automatically as a feature of the clip's position in a presentation. The variables used are the clip durations and positions, and the target bandwidth for the presentation. The technique may simply average out the spare bandwidth to determine the virtual bandwidth for the video clip. However, it may be desirable to introduce some safety margins and to limit the virtual bandwidth available to a somewhat lower level which would be optimal. It should be noted that this approach could be taken with other video encoding systems and not just products using technology from technology from Real Networks. Apple QuickTime™ now supports SMIL, and Internet Explorer™ 5.5 from Microsoft supports the SMIL timing and synchronisation module. Viewed from another aspect of the invention there is provided a method for the streaming of a presentation which has been produced by a method, or using software, as described above. Some embodiments of the invention will now be described by way of example and with reference to the accompanying drawings in which: Figure 1 is a schematic diagram setting out the steps involved in designing a presentation using the system of the invention; Figure 2 is a time line for a first presentation; Figure 3 is a time line for a second presentation; Figure 4 is a time line for a third presentation; and Figure 5 is a schematic representation of how clips may be arranged on screen using software in accordance with the invention. Figure 1 is a block diagram showing the various steps for creating a presentation. These are as follows: User makes clips available for selection, including video clips and low bandwidth introduction and link clips such as text and audio files . User selection of target bandwidth. System selection of nominal maximum bandwidth. User selection of clips and arrangement in desired order. System calculation of spare bandwidth prior to a video clip, and of increased virtual bandwidth for transmission of video clip. System generation of code, including coding for preroll of video clip. The presentation is based upon a SMIL file, which contains information about the spatial and temporal positioning and relation of all media elements. For purposes of bandwidth budgeting, i.e. ensuring the presentation will stream smoothly at the target bandwidth, a presentation is arranged logically into a number of scenes, which translate into a <par> .. </par> block in the SMIL code. This ensures that all media elements within the scene are downloaded in parallel to each other once the scene is started. By default, if a high bandwidth media type (i.e. video) is added to the presentation, it is placed in a new scene being created. Take the case of a presentation consisting of one scene containing one video clip. To stream, this video clip must be encoded to a streaming format, and a SMIL file generated. The system in accordance with the invention uses the target bandwidth of the project (say 28kbps) to set a bitrate suitable for transmission (say 20kbps) . This is the standard behaviour for encoding tools on the market at present. Next, a piece of streaming text is added to the scene, of 10 seconds duration, which will play in sequence before the video clip. This text will have fixed bandwidth requirement of 1 kbps. The resulting timeline for the presentation is shown in Figure 2, with the bandwidth requirements of the two clips. Figures 2, 3 an 4 are not to scale. In a normal streaming scenario, the encoding settings for the video clip would remain the same, based on a 20kbps bandwidth. However, in accordance with the invention software analyses the scene prior to encoding, detecting the duration and file sizes of the text and video clips. Given the average bit rate available (~20kbps) over the total duration of the scene (text clip time + video clip time) , the software concludes that there is approximately 19kbps bandwidth "spare" over the 10 seconds that the text clip is playing, which will be used to start downloading the video clip (as the clips are being downloaded in parallel) . This pseudo-extra bandwidth may be used to calculate the maximum bitrate which the clip may be encoded at and still stream successfully over its duration. In the case of a 36 second video, this is calculated to be 25.2kbps. As a higher bitrate implies more information and hence better quality, this has been improved by approximately 25%. Figure 3 shows the timeline when it is decided to insert an audio clip of 6 seconds in duration into the timeline to play in parallel with the text, before the video plays. This clip will be encoded at 8.5 kbps, determined by the software from the target bandwidth of the project. Audio encoding settings are discrete, fixed values, unlike video encodings where the bitrate may be varied, and the clip will consume 8.5x6 = approximately 45kb. This portion of bandwidth is accounted for in the calculation of the maximum possible bitrate for encoding the video clip. This is calculated to be 23.7kbps. Figure 4 shows the timeline if the presentation is altered so that the three clips are played in sequence; text, audio and video. The text and audio clips will still require the same bandwidth, but over a greater time period. By averaging 20kbps over this duration, the software calculates the bandwidth available for downloading the video clip, and encodes the video at a bitrate of approximately 27kbps. If the order is altered to text, video and then audio, the situation reverts to that of Figure 2, as the audio clip follows the video and is irrelevant for the purposes of calculating the virtual bandwidth available. Figure 5 shows a display as it would appear on a computer screen in use of the software. On the left hand side is a tree of clips available for use in a presentation. In this case a scene is being prepared and a text file "mycra", an audio file "twisted" and a video file "welcomel" have been made available. Clips may be previewed within the application. Using conventional operator techniques, clips can be added to or removed from the tree. On the right hand side, the clips are listed in the order of play. The duration and file size have been determined by the system. The system has already been advised that the target bandwidth is 28k, and using a predetermined rule has set the nominal maximum to 20k. The text and audio clips have predetermined, fixed bandwidths allocated as described above. The software calculates the spare bandwidth available over the period of the text and audio clips, and thus calculates the bitrate that can be allocated to the video clip, in this case 27.2k. The timeline for the presentation is shown at the bottom of the display. The software produces the presentation with the appropriate SMIL codes, in which the bitrate allocated to the video clip is 27.2k and preroll of the video clip takes place whilst the text and audio clips are played. As will be appreciated, the calculation of the video encoding bitrate by the software is dynamic to the environment of the clip, rather than relying upon a static average value, the bitrate being calculated automatically as a feature of its position in a presentation. Thus the variables used are the clip durations, positions and target bandwidth for the presentation. The technique averages to determine the optimal bandwidth for the video clip but it may be desirable to introduce some further safety margins. The above described software is designed to run on a personal computer or workstation under a suitable operating system such as Windows™ or NT™ from Microsoft™, and can be supplied on media such as a CD or from a remote location by downloading. The software preferably produces presentations for use with products from Real Networks such as RealServer™. However, the same approach could be taken with other video encoding systems. QuickTime™ from Apple now supports SMIL, and Internet Explorer™ 5.5 supports the SMIL timing and synchronisation module. Both offer freely available encoding software development kits . In its preferred form, the software is a PC application for the authoring and publication of streaming multimedia presentations for RealNetworks' RealPlayer™. It enables users to assemble, lay out and sequence video, audio, images, animation and text to produce a multi-scene presentation, which will stream successfully over any chosen Internet bandwidth, from dial-up modem to high speed cable modems or corporate networks. This has been achieved by a combination of components such as RealNetworks' RealSystem™ 8 and Macromedia Flash™, and utilisation of open standards (SMIL) . Once the presentation has been assembled, audio and video files (WAV and AVI) are encoded to RealAudio™ or RealVideo™ 8, a highly compressed streaming file format proprietary to RealNetworks. The sequencing and layout details are translated into a SMIL document, analogous to HTML for multimedia. The entire presentation may be previewed locally using a RealPlayer, and be published to any suitable HTTP or network server via a built-in FTP facility, and then previewed remotely. The system in accordance with the invention analyses a video clip's environment within a presentation and automatically encodes the clip at the largest suitable bitrate to stream successfully. This can lead to significant increases in video quality, especially in low bandwidth scenarios such as Internet connections via dial-up modems or mobile phones. The SMIL mark-up language is used to start downloading the video clip whilst the low bandwidth clips are playing, exploiting any unused bandwidth in this period. Thus the clip is already partially downloaded before it is due to start playing in the timeline, reducing the possibility of stalling. Further, depending on the duration of the low bandwidth section of the stream, this unused bandwidth can be used to download and successfully stream a clip which would otherwise be too large (i.e. data rate too high) to stream under the specified network conditions. In a preferred embodiment the system also features a WYSIWYG visual editor for RealText™, an XML-derived language developed by RealNetworks. RealText is currently supported only by RealPlayer. It provides a simple way of producing streaming animated text.

Claims: CLAIMS : 1. A method of producing a presentation for transmission over a data network using streaming, the presentation comprising a relatively low bandwidth first clip followed by a relatively high bandwidth second clip, comprising the steps of establishing a nominal maximum transmission bitrate, determining the bandwidth requirement of the first clip and allocating spare bandwidth between this and the nominal maximum bitrate for buffering of the second clip, and using the values of the spare bandwidth for the first clip, the duration of the first clip and the duration of the second clip to calculate an indication of the optimum number of bits into which the second clip can be encoded for streaming. 2. A method of producing a presentation for transmission over a data network as claimed in claim 1, wherein the indication of the optimum number of bits into which the second clip can be encoded for streaming is calculated by multiplying the spare bandwidth available for the first clip by the duration of said first clip and adding the product of the nominal maximum bitrate and the duration of the second clip. 3. A method of producing a presentation for transmission over a data network as claimed in claim 1 or 2, wherein the nominal maximum bitrate is calculated based on a target bandwidth of the data network. 4. A method of producing a presentation for transmission over a data network as claimed in claim 3, wherein the nominal maximum bitrate is less than the target bandwidth. 5. A method of producing a presentation for transmission over a data network as claimed in any preceding claim, wherein said first clip comprises a number of clip components streamed sequentially or simultaneously. 6. A method of producing a presentation for transmission over a data network as claimed in any preceding claim, wherein said first clip is text, audio and/or still images. 7. A method of producing a presentation for transmission over a data network as claimed in any preceding claim, wherein said second clip is video. 8. Computer software which when installed on a data processing means will cause the data processing means to carry out the method as claimed in any preceding claim. 9. Computer software which when running on data processing means will establish an environment for producing a presentation for transmission over a data network using streaming, the presentation comprising a relatively low bandwidth first clip followed by a relatively high bandwidth second clip, the software enabling the establishment of a nominal maximum transmission bitrate, and determining the bandwidth requirement of the first clip, allocating spare bandwidth between this and the nominal maximum bitrate for buffering of the second clip, and using the values of the spare bandwidth for the first clip, the duration of the first clip and the duration of the second clip to calculate an indication of the optimum number of bits into which the second clip can be encoded for streaming. 10. A method of streaming a presentation which has been produced by a method, or using software, as claimed in any preceding claim.

Inventors: BARNETT STUART, FINCH MICHAEL, JENNINGS ANTHONY, BARNETT, STUART, FINCH, MICHAEL, JENNINGS, ANTHONY
Applicants: ACTIVE WEB TECH LTD, BARNETT STUART, FINCH MICHAEL, JENNINGS ANTHONY, ACTIVE WEB TECHNOLOGIES LIMITED, BARNETT, STUART, FINCH, MICHAEL, JENNINGS, ANTHONY
Application year: 2001
Main classification: H04N7
IPC codes: H04L29, H04N7