Patent Publication Number: US-9426441-B2

Title: Methods for carrying and transmitting 3D z-norm attributes in digital TV closed captioning

Description:
CROSS REFERENCE TO RELATED APPLICATIONS 
     This application claims priority to U.S. Provisional Patent Application No. 61/311,742 filed Mar. 8, 2010, hereby incorporated by reference in its entirety. 
     The present application is related to PCT Application PCT/U.S. 2010/039543 for ‘Perceptual Depth Placement For 3D Objects’ by Richard Welsh and Christian Ralph, filed on Jun. 22, 2010 incorporated herein by reference in its entirety and attached herewith as Appendix A. 
    
    
     FIELD 
     The present disclosure relates to carriage of the z-norm depth information of the overlay. In particular, it relates to methods for carrying and transmitting 3D z-norm attributes in digital tv closed captioning. 
     BACKGROUND 
     The CEA-708-D standard (see, e.g., http://www.ce.org/Standards/browseByCommittee_2525.asp, incorporated herein by reference in its entirety) defines Digital TV closed captioning as used in Digital TV systems (ATSC, DVB, Cable), etc. Newer Digital TV systems provide a 3D rendering capability which provides for depth projection of Video content during playback through use of polarized glass and lens technology and/or shuttered glasses. 
     One problem present in such systems today is that there does not exist a means to specify, using existing CEA-708D captioning technology, the depth for which the on-screen display of caption data shall be rendered. 
     SUMMARY 
     According to a first aspect, a method for carrying data is provided, comprising: providing overlay depth data of an overlay of a stereoscopic image of a display device as a fraction of a viewer distance from the overlay; and allocating the overlay depth data in an available portion of a layer of a multilayer protocol system for transmission of overlays. 
     According to a second aspect, a method to transmit data to a client device is provided, comprising: providing overlay depth data of an overlay of a stereoscopic image of a display device as a fraction of a viewer distance from the overlay; coding the overlay depth data in an available portion of a layer of a multilayer protocol system for transmission of overlays; and transmitting the coded overlay depth data to the client device. 
     Therefore, in accordance with several embodiments of the present disclosure, means to extend the CEA-708-D closed caption standard are to support depth adjustment for the end user system. 
     APPENDIX 
     Appendix A is attached herewith and forms integral part of the specification of the present application. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         FIG. 1  shows a flowchart of an embodiment of the methods in accordance with the disclosure. 
     
    
    
     DESCRIPTION OF EXAMPLE EMBODIMENTS 
     Generation of normalized depth data on the z-axis is described in PCT Application PCT/U.S. 2010/039543 attached herewith as Appendix A. Such z-norm depth information can also apply to overlay and, in particular, to close captioning. 
     Embodiments of the present disclosure are directed to carriage of the z-norm depth information of the overlay. In particular, such information can be conveyed through unallocated elements of a closed captions standard, such as the CEA-708-D standard. 
     The closed captioning standard CEA-708-D defines a closed-caption system for digital television (DTV) using the OSI layered protocol model (see, e.g., http://en.wikipedia.org/wiki/OSI_model, incorporated herein by reference in its entirety). As such, the transport, packet, service, coding, and interpretation layer are all used by the standard. The structure of these layers is known to the person skilled in the art and will not be discussed in detail in the present application. 
     As shown in the examples below, the z-norm depth information can be coded in an available portion of a service layer, coding layer, packet layer etc. of the standard. On the client side, the client device (e.g., set-top box, television, personal computer and the like) can contain a software modification that permits dynamic 3D processing of the overlay. 
     The choice and selection of the layer (e.g., packet, service, coding) by which the depth information is communicated and the type of interpretation provided by the interpretation layer may be application specific. 
     Example 1 
     Packet Layer 
     For the packet layer definition, reference can be made to the cc_data( ) structure as defined in section 4.4 of CEA-708-D standard (DTV cc-data ( ) structure), incorporated herein by reference in its entirety. In accordance with the present disclosure, such a structure can be augmented such that one possible example of such an implementation would result in cc_data.reserved (8 bits) encodes the Z-Norm data as defined below:
         reserved&lt;7&gt;=sign bit   reserved&lt;6-0 &gt;=z norm value
 
The person skilled in the art will understand that other packet layer implementations are possible.
       

     This packet layer embodiment allows for carriage of z-norm depth data on a per-packet basis. It should be noted that these bit definitions are currently reserved and set to all 1&#39;s. Therefore, such modification does not break existing compatibility with legacy systems. 
     Example 2 
     Service Layer 
     The DTV closed captioning standard CEA-708-D provides for up to 63 services. The services are time division multiplexed and inserted sequentially into caption channel packets (coding layer). Service #1 is the primary caption service while Service #2 provides for the secondary language service. Service #0 is not to be used. Service #1-6 are standard services and Services #7-#63 are extended services. 
     According to an embodiment of the present disclosure, a service # can be allocated from one of the extended services #7-#63. 
     For packing of z-norm data within a service channel, the service block definition of section 6.2 in CEA-708-D (6.2 Service Blocks, incorporated herein by reference in its entirety) can be used, with a service channel packet using an extended_service_number and block_size=1. 
     The format of the block_data[0] byte within the extended service packet can be similar to what shown in Example 1 above:
         block_data[0].&lt;7&gt;=sign bit   block_data[0].reserved&lt;6-0 &gt;=z norm value
 
allowing for up to 64 signed and unsigned znorm correction values to be communicated to the DTV rendering device on a per-service basis. The person skilled in the art will understand that the above implementation could be done at different levels of the CC service protocol stack. The exemplary implementation above was chosen by applicants to support the broadcast compatibility across existing (Legacy) set-top box receivers and televisions.
       

     Example 3 
     Coding Layer 
     For packing of Z-Norm data within the coding layer of CEA-708-D, any of the unused codes as defined in section 7 of CEA-708-D (7 DTVCC Coding Layer—Caption Data Services (Services 1-63), incorporated herein by reference in its entirety) can be used. One or more bytes of data can be supported using the 1, 2, or 3-byte character code extensions. 
     For optimal compatibility, the extended code space in section 7.1.1 (7.1.1 Extending The Code Space) can be used with a minimum of a 1-byte code from the C1 Code set defined in 7.1.5 (7.1.5 C1 Code Set—Captioning Command Control Codes). For example, a window command is a single byte that may be followed by several parameter bytes. In a similar fashion, a z-norm depth command may be defined as a one of the undefined window commands 0x93-0x96 which carries the z-norm bits as defined below:
         znorm_direction&lt;7&gt;=sign bit   znorm_value&lt;6-0 &gt;=z norm value       

     It should be noted however, that additional information may be conveyed beyond just the Z-Norm shift offset and sign (positive/negative direction). However, at the time of this writing the above requirements should meet the majority of existing TV platforms supporting stereoscopic display capabilities however, without loss of generality, additional fields may be encapsulated in this proposed schema. 
     The person skilled in the art will understand that, for all of the examples provided above, the bit definition may change to allow for less depth information to be carried since most current displays do not allow for more than 15 pixel offset correction to be performed. The present disclosure is intended to cover any and all definitions for these bits, with the main concepts of the disclosure allowing for any embodiment or derivative of the same to be covered across all layers of the transport medium. 
     Therefore, in accordance with several embodiments of the present disclosure, a method ( 100 ) as disclosed in the flowchart of  FIG. 1 . Overlay depth data are provided ( 110 ) and allocated and/or coded ( 120 ) in an available portion of a protocol system such as a protocol operating according to a CEA-708-D standard. The overlay depth data can then be transmitted ( 130 ) to a client device and processed ( 140 ) at such device. 
     In another embodiment, a normalized 1 meter screen width can be used, such as a 1000 pixel wide screen with 1 millimeter per pixel resolution. Normalization provides an advantage that the playback device need only know its own screen width (w s ) to appropriately render an object at depth, and composition software can virtually render (e.g., no physical screen used to make object depth decision) using the normalized screen width. That is to say, a w s  value need not be communicated since it is known a priori. 
     Additionally, the numerical presentation of z pv  cannot adequately express depth at or beyond the plane of infinity. This shortfall is traversed by appreciating that an object will appear to be at infinity when the visual axes of a viewer&#39;s eyes are parallel. Thus, the plane of infinity can be specified to be at or about the negative value of interocular separation (about −65 millimeters for an adult). For a normalized 1 pixel/millimeter screen, the plane of infinity can be established to have a pixel separation offset at or about −65 pixels. 
     Using 3D positioning layout for a normalized screen, either a playback or compositing device can appropriately insert an object, such as captioning, into a 3D image when provided with three positional values: x as a percent of screen width, y as a percent of screen height, and z n  as a percent of perceived normalized depth. A normalized pixel separation offset, S p , can then be computed at least as follows, without limitation: 
                 S   p     =       z   n     ⁡     (     65     100   -     z   n         )         ;         
where 0≦z n ≦100 (i.e., object lying on or in front of screen plane towards viewer position; and
 
S p =0.65z n , where z n &lt;0 (i.e., object lying behind the screen plane away from the viewer position).
 
     The normalized pixel separation offset allows object placement in 3D space with respect to a viewer&#39;s perceived depth independently of display size or viewer distance. A playback device can use the normalized pixel separation offset (S p ) received, for example as metadata in a bitstream, to compute a device specific pixel separation offset (S ct ) by adjusting with a factor of its own pixel pitch. If the playback device&#39;s pixel pitch is 0.5 millimeters instead of 1 millimeter of the normalized screen, then S ct =S p /0.5, in this example. 
     The present disclosure may suitably comprise, consist of, or consist essentially of, any element (the various parts and/or features of the disclosure) and their equivalents as described herein. Further, embodiments of the present disclosure may be practiced in the absence of any element, whether or not specifically disclosed herein. Numerous modifications and variations of the disclosure are possible in light of the above teachings. It is therefore to be understood that within the scope of the appended claims, the disclosure may be practiced otherwise than as specifically described herein.