Abstract:
A multi-conference unit (MCU) ( 2 ) is arranged to compress a video stream in the horizontal direction to adjust the aspect ratio prior to onward transmission to an endpoint ( 6 - 1  to  6 - 4 ). By arranging for an endpoint to de-compress the video stream by a similar amount, widescreen video streams may be transmitted and viewed using generally legacy equipment. The use of a widescreen transmission provides additional opportunities for effective layouts of windows in a composed video stream with excessive distortion or clipping.

Description:
CROSS-REFERENCE TO RELATED APPLICATIONS  
       [0001]     Not Applicable  
       STATEMENT RE: FEDERALLY SPONSORED RESEARCH/DEVELOPMENT  
       [0002]     Not Applicable  
       BACKGROUND  
       [0003]     This invention relates to a multi-conference unit for connection in a video conference network.  
       BRIEF SUMMARY  
       [0004]     A live multi-way video communication is typically known as a video conference. Each conference location is equipped with a device comprising a display, camera, loudspeaker, microphone and accompanying processing and control units. This combination of equipment is typically known as a “video endpoint”. The video endpoints are connected to a communications network such as ISDN, Internet Protocol (IP) or Third Generation Wireless (3G) networks. The video endpoints encode the video from the camera and the audio from the microphone, and transmit this data on the network. The endpoints also receive encoded data from the network, decode it, and output the audio and video accordingly. The video endpoints typically implement standards such as H.320, H.323 or SIP. These standards define how the video and sound (media) streams and control signals are encoded and decoded.  
         [0005]     When there are more than two parties participating in a particular video communication, then it becomes known as a video conference. A typical topology is shown in  FIG. 1 . In this topology, a video conference network  4  couples together endpoints  6 - 1  to  6 - 4 . In this case, an additional piece of equipment is required, typically known as a video Multi-Conference Unit, or MCU  2 . This is also connected to the communications network  4 . Each individual video endpoint  6 - 1  to  6 - 4  establishes a link to the MCU  2  instead of having to establish links to every other endpoint  6 - 1  to  6 - 4 .  
         [0006]     An MCU is a device that provides a bridge between two or more terminals, or gateways, in order to enable them to participate in a multipoint conference. It is adapted to receive data streams from each endpoint participating in a videoconference, combine the data received from the endpoints and transmit a single data stream to each of the participating endpoints. The data streams sent to the endpoints may encode a composed or voice-switched image of the other participants in the conference call. If desired the data stream sent to each of the participating endpoints may also be customised for each endpoint according to the layout specified by an endpoint&#39;s user. Optionally, the MCU may also be connected to a recording unit which enables a video conference to be recorded and played back.  
         [0007]     The purpose of the MCU  2  is to allow the endpoints to communicate with each other but without each endpoint requiring multiple communications links, and without requiring multiple displays, loudspeakers etc. To achieve this, the MCU  2  sends a “composition” of the video from the other endpoints to each endpoint. Depending on the number of parties in a conference, many different image compositions are possible:  
         [0008]     the screen divided into many equal sized windows  
         [0009]     one or two larger windows plus additional smaller windows  
         [0010]     With an advanced MCU  2 , each participant can make their own choice of composition. A composition can be voice switched, such that the participant(s) currently speaking are displayed in the largest windows.  FIG. 2  shows a selection of possible video compositions.  
         [0011]     Current videoconferencing endpoints invariably produce images which have a width-to-height ratio of 4:3. This is commonly known as standard aspect ratio. This aspect ratio is used whatever data rate, resolution, pixel aspect ratio or frame rate is used.  
         [0012]     However, the large flat screen displays used with modem video endpoints frequently have a width-to-height ratio of 16:9. This is commonly known as widescreen aspect ratio, or widescreen format.  
         [0013]     Because of this difference in aspect ratios, the standard aspect ratio video cannot make best use of the widescreen display. 
        The video can be displayed as if the screen were standard format, but with vertical black bars at each side of the image, or     The display can expand the image so that it fills the full width of the screen, but with the top and bottom of the image lost (clipped), or     The display can stretch the image in the horizontal dimension only, which results in a distorted image.        
 
         [0017]     It is apparent that none of these options is ideal.  
         [0018]     In accordance with a first aspect of the invention there is provided a multi-conference unit for connection in a video conference network, comprising a video input for receiving a video stream to be conferenced and a video output for outputting a video stream to a video endpoint, the unit further comprising a scaler operable to adjust the horizontal dimension of an output video stream before onward transmission to an endpoint, thereby to alter the aspect ratio of images in the output video stream.  
         [0019]     In this way, use of the MCU of the invention in conjunction with aspect ratio adjustment at an endpoint allows widescreen images to be displayed correctly but without alteration to the rest of the video conference network. This provides perfect backwards compatibility and allows best use to be made of modern endpoints with widescreen displays.  
         [0020]     In another aspect, the invention provides a method of transmitting a video stream in a video conference network comprising receiving a video stream, scaling the stream in the horizontal dimension and transmitting the scaled stream to an endpoint 
     
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0021]     Embodiments of the invention will now be described by way of example and with reference to the drawings in which:— 
         [0022]      FIG. 1  is a schematic representation of a typical videoconference arrangement;  
         [0023]      FIG. 2  is a schematic representation of different options for laying out videoconference windows on a standard aspect ratio display;  
         [0024]      FIG. 3  is a screenshot of a user interface in accordance with the invention;  
         [0025]      FIG. 4   a  is a schematic representation of a 3×2 window layout;  
         [0026]      FIG. 4   b  is a schematic representation of a 1+3 window layout;  
         [0027]      FIG. 5   a  is a schematic representation of a 1+4 window layout;  
         [0028]      FIG. 5   b  is a schematic representation of a 4×3 window layout;  
         [0029]      FIGS. 6   a  and  6   b  show a schematic representation of two possible 2×2 window layouts;  
         [0030]      FIG. 7   a  is a schematic representation of a composite window with window clipping carried out before composition;  
         [0031]      FIG. 7   b  is a schematic representation of a composite window with window clipping carried out after composition; and  
         [0032]      FIG. 8  is a schematic diagram of an MCU with a single endpoint in accordance with the invention.  
     
    
     DETAILED DESCRIPTION  
       [0033]     As noted above, one of the functions of an MCU is to compose outgoing video streams from the multiple incoming video streams from conference participants. These outgoing streams are then displayed by the participating endpoints.  
         [0034]     The MCU  2  is arranged to compress the outgoing video streams in the horizontal dimension. At the endpoint  6 - 1  to  6 - 4 , the stream is stretched by a corresponding amount in the horizontal direction thus restoring the aspect ratio ready for display of the stream. In the discussion below, the endpoint is described as carrying out the stretching operation. In practice, however, the stretching may be carried out by the endpoint (which typically does not have a screen but rather a connector for coupling to screen) and/or by a screen connected to the endpoint. The invention is intended to encompass or interoperate with all these possibilities.  
         [0035]     A user interface of the type shown in  FIG. 3  may be used to allow configuration of the widescreen or standard aspect ratio transmissions. A button  8  may select transmission for viewing in a normal aspect ratio (i.e. standard transmission with no dimensional adjustment) and a button  10  may select transmission as a widescreen signal (i.e. horizontal compression is enabled).  
         [0036]     As explained in more detail below, the choice of widescreen transmission also affects how streams are arranged and sized before onward transmission of the outgoing composite video stream.  
         [0037]     It can be seen from  FIGS. 4, 5  and  6  that many of the possible video compositions are much more effectively presented by making use of the extra width available on a widescreen display. There is presently no standardised or established videoconferencing format for widescreen video. So the videoconferencing endpoints cannot themselves make use of this extra display width even though many of the displays used are now widescreen displays.  
         [0038]     The endpoint need have no knowledge of the scaled nature of the video being received from the communications network. Accordingly, if the video scaled as previously described is displayed on a standard display, it would appear distorted. However, if the video is displayed on a widescreen display, then this display device can be configured to take the video input and scale it up (expand it) in the horizontal dimension such that the entire screen is filled. By this means, the compression applied by the MCU is reversed, and the video is properly displayed with no distortion.  
         [0039]     The full area of the wide screen is used; therefore the users see a larger image.  
         [0040]     For many of the screen layouts (compositions), the clipping of the images is much reduced.  
         [0041]     There is no loss of image resolution either vertically or horizontally  
         [0042]     The user benefits from these improvements without needing to upgrade the video endpoint (other than to use a widescreen display)  
         [0043]     The behaviour of the video endpoint is unchanged even though it is providing a widescreen image. It is the MCU which composes the data stream suitable for widescreen display. An aspect of the invention is a control on the MCU to enable or disable the video scaling feature. This control button is shown in  FIG. 3 . Clearly the feature should be disabled if the video endpoint has only a standard display, so that the image is not scaled by the MCU in this case.  
         [0044]      FIG. 4   a  shows a 3×2 layout of windows. On a standard display  9  as shown on the left of the figure, vertical strips  10 - 1  and  1 - 2  on each window have to be clipped. On a widescreen display  12 , it can be seen that the clipping ( 14 - 1 ,  14 - 2 ) is negligible.  
         [0045]      FIG. 4   b  shows a 1+3 layout. On a standard display  9 ′, the images in all the windows are clipped; the small windows requiring clipping  10 ′- 1  and  10 ′- 2  and the large window also requiring clipping  11 ′- and  11 ′- 2 . On a widescreen display  12 ′, no clipping is necessary.  
         [0046]      FIG. 5   a  shows a 1+4 layout. On a standard display  9 ″, the main window is clipped at  10 ″- 1  and  10 ″- 2 . On a widescreen display  12 ″, the clipping is negligible.  
         [0047]      FIG. 5   b  shows a 4×3 layout. On a standard display  9 ′″, the images in all the windows are clipped  10 ′″- 1  and  10 ′″- 2 . On a widescreen display  12 ′″, no clipping is necessary.  
         [0048]      FIGS. 6   a  and  6   b  show a 2×1 layout. This is a commonly used layout being ideal for conferences involving 3 parties.  FIG. 6   a  shows severe clipping  10 ″″- 1  and  10 ″″- 2  on a standard display  9 ″″, with naturally much less on a widescreen display  12 ″″.  FIG. 6   b  shows another possible method for the composition, in which the images are not clipped, but are scaled down and black bars added top and bottom. It can be seen that on the widescreen display  12 ′″″, each window can be much larger. In fact, the image area is increased by a factor of 1.78 compared with the standard display. This is despite the display area being only 1.33 times greater on a widescreen display compared to a standard display, and illustrates the considerable benefit of the use of a widescreen display for composed videoconference images.  
         [0049]     An additional consideration concerns images which need to be clipped or have black bars added in order to fit the display. Ideally this is done to an equal extent for each individual image window. This is illustrated in  FIG. 7   a . This is achieved by scaling the images and then clipping, before composing the image windows  16  into the final output video data stream. The alternative, which is simpler but leads to undesirable results, is shown in  FIG. 7   b . Here the overall image is scaled and composed first, then clipped to fit the screen. It can be seen that as a result the image windows  18  are not clipped equally top and bottom.  
         [0050]      FIG. 8  shows an MCU  2  with a connection  20  into a video conference network  4 . Incoming video streams and at least one (outgoing) composite video stream are transmitted over the link  20  for processing by an endpoint  6 - 1 .  
         [0051]     The MCU  2  includes a composition processor  22  and a scaling processor  24 . The composition processor arranges incoming video streams into a composite video stream as described above. The scaling processor  24  then scales the horizontal dimension (preferably—although scaling in the vertical dimension may also or alternatively be applied) before onward transmission to the endpoint  6 - 1 . A scaling processor  26  in the endpoint  6 - 1  then applies reverse scaling to bring the video stream back into the correct aspect ratio prior to display.  
         [0052]     The scaling processors  24  and  26  are preferably implemented on a digital signal processor which is adapted to execute the step of scaling or reverse scaling an image in a horizontal or vertical dimension. The use of processor to scale images means that a dedicated electronic circuit does not need to be built to scale the images.  
         [0053]     A modified endpoint may be produced which may be configured to drive either a standard or wide screen. Such endpoints may advertise to other endpoints that they are capable of receiving wide format video.  
         [0054]     Some endpoints include a limited-capability MCU function—they can act as an MCU for a conference in which they are a participant along with a small number of additional participants. Such an endpoint may also include an MCU as described above.