Systems and methods for processing timed text in video programming

Systems, devices and methods are described to process timed text that is associated with television or other video programming. Pre-set data that describes the particular format of the timed text that is associated with a program is initially received and stored at the television receiver prior to receiving the program with encoded timed text. The timed text that is associated with the received program is interpreted according to the pre-set data previously stored at the television receiver, and the program with the interpreted timed text is output to a display for presentation to the viewer.

TECHNICAL FIELD

The following discussion generally relates to the processing of closed captioning or other timed text information associated with television or other video programming.

BACKGROUND

Closed captioning refers to any system that receives, formats, and displays text on a television or other display to thereby allow viewers to access additional or interpretive information about programming content. Closed captions often provide textual transcriptions of the audio portions of programs as the programs progress, thereby allowing the hearing impaired and others to follow the program content without listening to the audio track typically associated with the programming. Motion pictures, video disk (e.g., DVD) content, streamed audio/video, video games and the like may similarly incorporate closed captions using any number of standard or non-standard techniques.

Federal law mandates that all broadcast television programming in the United States must include closed caption information. For conventional digital (e.g., ATSC) television broadcasts, caption streams are typically encoded in a well-known format such as EIA-608 format, EIA-708 format, and/or the like. Other closed captioning formats or standards may be implemented in the United States or elsewhere.

As more and more consumers receive their television programming from broadband sources, the Society of Motion Picture and Television Engineers (SMPTE) and others have recommended a transition from the older standards (e.g., EIA 608/708) to more advanced “timed text” formats such as SMPTE-TT. This newer timed text format incorporates syntax, structures and tags similar to the extensible markup language (XML) to provide more advanced features and to improve flexibility in comparison to the more traditional legacy formats. The SMPTE-TT format is described in SMPTE Standard ST 2052-1-2010, which was approved on Dec. 3, 2010 and is incorporated herein by reference.

Transitioning from legacy formats to timed text formats can create a number of issues in practice. First, incorporating legacy data (e.g., EIA 608/708 data) into timed text formats can be a challenge. Although the SMPTE-TT standard indicates a desire to support legacy formats, it does not presently provide guidance as to how this is to be accomplished. Moreover, challenges can arise in providing timed text within live broadcasts for reasons described more fully below.

It is therefore desirable to provide systems, devices and/or processes that ease the transition from legacy formatting to the newer timed text formats, and that support timed text within live television broadcasts. These and other desirable features and characteristics will become apparent from the subsequent detailed description and the appended claims, taken in conjunction with the accompanying drawings and this background section.

BRIEF SUMMARY

According to various exemplary embodiments, systems, devices and methods are described to process timed text that is associated with television or other video programming. Pre-set data that describes the particular format of the timed text that is associated with a program is initially received and stored at the television receiver prior to receiving the program with encoded timed text. The timed text that is associated with the received program is interpreted according to the pre-set data previously stored at the television receiver, and the program with the interpreted timed text is output to a display for presentation to the viewer. The pre-set information may be provided for each program as part of an electronic program guide, for example, or as part of other data transmitted out of band within the broadband distribution system. Other embodiments may provide the pre-set formatting data in response to a request from the receiver device, or otherwise as desired.

Additional embodiments, aspects and other features are described in more detail below.

DETAILED DESCRIPTION

According to various exemplary embodiments, formatting data about timed text associated with a television program, movie or other video content is pre-stored at the video player/receiver prior to receipt of the content itself. The pre-stored data allows the player/receiver to know the formatting of timed text associated with the program in advance so that the player can adequately interpret the timed text while rendering the program for output to the viewer. Each program may have its own unique timed text format in some embodiments, or some programs may share formatting data as appropriate. The pre-set formatting data may be provided according to a batch scheme (e.g., as part of an electronic program guide or other feature in which information about multiple programs is received simultaneously), or pre-set data may be obtained on any sort of as-needed basis, as described more fully below.

The pre-set data associated with a program allows the receiver to obtain a priori knowledge about the format of the timed text so that the timed text can be properly interpreted. To that end, the formatting of the timed text can be flexibly adjusted to accommodate any available information. Legacy data (such as data in conventional CEA-608 or CEA-708 formats), for example, can be simply encapsulated within a more current timed text format, with the pre-set data identifying the particular locations and structures of the legacy data. By processing the pre-set data, the receiver/player can readily locate and reconstruct the data, thereby allowing for convenient backward compatibility with any number of legacy formats.

Moreover, by providing pre-set data in advance of the program itself, the formatting metadata within a live broadcast can be reduced or even eliminated. That is, live broadcasts typically require the broadcaster to periodically (and frequently) transmit files containing header information in the broadcast metadata so that viewers tuning in to the broadcast at any time are able to quickly obtain the information needed to process the timed text associated with the broadcast. Otherwise, viewers tuning in mid-broadcast would be unable to process the timed text until the header information was re-transmitted.

Rather than relying upon the receiver to extract formatting metadata from the broadcast itself, some embodiments provide pre-set formatting data that is transmitted separately and in advance of the programming. Since the receiver has a priori knowledge of the timed text formatting from the pre-set data, there is no need to wait for the information to be broadcast within the program stream itself. This allows the receiver to interpret the timed text on a relatively immediate basis for any program that the viewer may select at anytime. Other embodiments may additionally or alternately retain the header data within the broadcast itself, but the header data could be augmented to provide additional pre-set information describing the format of the timed text.

Turning now to the drawing figures and with initial reference toFIG. 1, an exemplary system100for producing imagery having closed caption information suitably includes a production system102, a distribution system104, and/or a transmission system106as appropriate. Content with formatted timed text information103, once created, is provided over any sort of broadcast or other distribution channel108for rendering at a media receiver no and playing on any sort of display120. As shown inFIG. 1, timed text information103may be inserted into the programming at any stage of production102, distribution104and/or transmission106to allow closed captioning to be presented in a desired format. The format of the timed text information can then be described in pre-set data105that can be provided to the receiver110in advance the program broadcast to allow appropriate interpretation of formatted timed text content103.

Pre-set data105is any sort of template, schema or other data that describes the particular formatting of timed text103. In various embodiments, formatting data105includes conventional header information (as described by the SMPTE-TT or another specification) or other information that allows receiver no to properly interpret the timed text103that accompanies video programming rendered by the receiver110. Such information may include the sizes and locations of data fields, for example, or other information relating to the structure, size, arrangement or other formatting of the timed text103. In various embodiments, timed text105also carries information related to the presentation of caption or other data, such as location of presentation on the screen, scrolling speed or time, pop on/off data, timing data, and/or the like. Such information may be defined within the pre-set data105as desired.

Timed text103and format data105may be created and formatted in any manner. Timed text103may be newly-created in some cases; in other cases, the timed text is extracted from legacy data (e.g., CEA-608 or CEA-708 closed caption data) and encapsulated within a different format (e.g., SMPTE-TT format). This extraction and encapsulation process may be automated, as desired. In such instances, the positions and lengths of the various legacy data fields can be defined in pre-set data105so that the ultimate viewer is able to re-extract and process the encapsulated data as desired. This encapsulation process allows a very high level of flexibility in retaining legacy data even as newer timed text formats are implemented.

In the exemplary embodiment illustrated inFIG. 1, production system102, distribution system104and transmission system106represent any systems, devices and/or organizations capable of producing, distributing and/or transmitting program content, respectively. As noted above, closed caption information103may be inserted into the programming content in any manner at any stage of production, distribution and/or transmission. In various embodiments, timed text103is formatted and associated with the programming at the production stage102by an author, editor, producer or other party. Certain producers, distributors and/or broadcasters may have preferred formats105for timed text103, and each may produce, distribute and/or broadcast content using their own formats105. Format data105may change from program to program, or may be consistent for each producer, distributor, broadcaster or other entity. All of the programs broadcast on a particular channel, for example, may have common format data105that may nevertheless differ from the format data105used by another network.

Timed text103formatted in accordance with any sort of pre-set data105may be encoded into any sort of programming in any manner, such as during a conventional “post production” phase or the like. Insertion of timed text103during or just after production may be appropriate in any sort of programming, including cinema programming, television programming or any other programming that is delivered in any sort of standard format, such as any form of MPEG format that can be rendered by a player/receiver device110. In other embodiments, however, timed text103may be added to programming during intermediate distribution104(e.g., encoding or transcoding of programming that is stored onto DVDs or other portable media, or otherwise distributed to network affiliates, broadcasters and/or any other parties). In still other embodiments, timed text103may be inserted into the program stream just prior to broadcast or other transmission106, much as current caption data is encoded in many current television broadcasts. Timed text103may also be inserted prior to placeshifting or other streaming of a network video stream, or in any other setting as desired.

To that end, distribution channel108may represent any sort of data link, broadcast or other wireless connection, physical media, and/or other avenue for delivering programming content. Examples of distribution channels include, without limitation, broadcast television, very small aperture terminal (VSAT) satellite (e.g., for movie distribution), direct broadcast satellite (DBS), cable television, cellular or other wireless telephone networks, Internet or other data communications networks, and/or the like. Again, closed caption information103may be inserted into or otherwise encoded with programming content in any manner, and at any phase of production, post-production, distribution and/or delivery of the programming content. Indeed, caption data need not be provided with the video content itself, but rather may be retrieved from a separate server (e.g., server132) and/or other source of information, as desired.

Receiver no is any device, component, circuitry or logic capable of receiving and processing video programming content. As shown inFIG. 1, receiver no includes a receiver interface112, a controller113with appropriate processor116and memory118resources, and a display interface114as appropriate. In various embodiments, receiver110is a conventional television receiver (e.g., a set top box or other broadcast, satellite and/or cable television receiver) capable of receiving signals via distribution channel108and providing an output signal115that can be displayed to the viewer. In various embodiments, display120is any sort of television or other monitor that is capable of receiving a program signal115from a set-top box, decoder or other external receiver no as desired. Receiver no may be equivalently implemented as a DVD or other player that is capable of rendering content stored on any optical, magnetic and/or other portable media. In still other embodiments, receiver no is a media player capable of receiving media streams over a communications network (e.g., the Internet, a local or wide area network, and/or any sort of wireless telephony network). In such embodiments, receiver110may be a software program that executes on conventional computer hardware, such as a personal computer, personal digital assistant, mobile telephone, video game player and/or the like.

Receiver interface112is any hardware, firmware and/or software interface capable of receiving programming content. In various embodiments, receiver interface implements a demodulator/decoder feature for receiving and demodulating digital television programming over a broadcast, satellite, and/or cable programming link. In other embodiments, receiver interface112is a conventional network interface to a digital network such as the Internet, or any local area, telephone and/or other network having access to the Internet. As noted above, receiver interface112may equivalently receive programming from a DVD or other portable media, or any other source as desired.

The exemplary television receiver no illustrated inFIG. 1also includes a storage medium119, as appropriate. Storage medium119may be implemented as additional memory, as a disk drive, or in any other manner. Many embodiments may provide a digital video recorder (DVR) or other recording feature that allows content to be stored in storage medium119for later viewing. Such content may be stored with timed text information103to allow for decoding and viewing of content text at the time that the programming stored on the recorder is rendered. AlthoughFIG. 1shows electronic program guide data124and pre-set formatting data105stored within storage medium119, equivalent embodiments could store this data in memory118, and/or in any other memory, disk or other storage medium available to receiver110.

Controller113is any sort of control logic or the like that interacts with receiver interface112and display interface114to output imagery to the viewer on display120. Controller113also controls the reception, storage and processing of video content via receiver interface112. To that end, controller113suitably directs the reception and storage of pre-set data105, as well as the interpretation of timed text information103that is provided within received programming. Additional detail about such processing is provided below with respect toFIG. 2.

In various embodiments, controller113may be implemented using any sort of microprocessor, microcontroller, digital signal processor or other processor116capable of directing the actions and processes of receiver110. Typically, processor116will be associated with any sort of memory118, such as any sort of static, dynamic, flash or other memory capable of storing programming instructions and/or data for processing by processor116. In various embodiments, receiver110is based upon a “system on chip” (SoC) implementation that incorporates a hybrid microcontroller116with memory118, input/output and/or other features to perform the various signal processing and other actions of receiver110. Various SoC and other integrated hardware implementations are available from Texas Instruments, Conexant Systems, Broadcom Inc., and many other suppliers as appropriate. Other embodiments may implement processor116and/or memory118using any sort of application specific integrated circuit (ASIC) or the like. Still other embodiments may implement processor116and/or the other features of receiver no with any number of discrete and/or integrated processing components (e.g., any sort of microprocessor or microcontroller), memories118, input/output features and/or other features as desired.

Display interface114is any physical and/or logical interface to display120. As noted above, in some implementations receiver no and display120are provided in an integrated product (e.g., a conventional television). In other embodiments wherein receiver no provides video output signals115to an external display104, such signals115may be provided in any compatible format. In embodiments wherein display120is a conventional television, for example, display interface114may provide video output signals115in any conventional format, such as component video, composite video, S-video, High-Definition Multimedia Interface (HDMI, e.g., any version of the CEA-861 standards), Digital Visual Interface (DVI), IEEE 1394, universal serial bus (USB) and/or any other formats as desired.

Display120is any sort of television, monitor and/or other display capable of presenting video imagery122to a viewer. In various embodiments, display120operates in conjunction with receiver110to generate video imagery122in any manner. Other embodiments may combine the functions and components of receiver110and display120into a common housing, as in a conventional television with a built-in tuner or other receiver. In the example illustrated inFIG. 1, imagery122is generated with caption text124that is provided within timed text103, as described by format data105.

In practice, then, receiver no suitably receives television-type programming in any format or medium. The programming may be received at receiver interface112from any broadcast or other transmission source, from any sort of streaming or portable media source, and/or any other distribution channel108. The programming is provided with (or at least associated with) timed text information103that includes caption or other information. This information103may be extracted from the programming stream, or from any other source as described herein, and interpreted according to the pre-set data105for the particular program that was previously received. Timed text and program content may be further processed in conjunction with any user preference or other information to generate suitable imagery on display120to thereby present the program and the timed text information to the viewer as desired.

The pre-set formatting data105may be delivered to the receiver no in any manner. In various embodiments, pre-set data105is provided through the same distribution channel108as the program itself, albeit at an earlier time. Data105may be delivered as part of the data associated with an electronic program guide125, for example. Such data may be delivered via an out-of-band signal that is separate from programming signals, but using the same DBS, cable or other distribution channel108used to deliver the programming signals, as desired.

In other embodiments, data105and/or timed text103may be delivered from a separate source via a backchannel such as the Internet, a telephone link, and/or another data communications network. Such embodiments may provide pre-set formatting data105from a server132operating on the network that delivers data105in response to requests placed by receiver110, as desired. Such requests may be posited on a batch or as-needed basis, as described more fully below.

With reference now toFIG. 2, an exemplary method200for processing timed text103at a video receiver such as receiver110suitably includes the broad functions of receiving pre-set data105for the particular program (function202), storing the pre-set data for later use (function204), subsequently receiving and decoding a program as part of a video broadcast or other distribution (function206), and interpreting the timed text associated with the received program in accordance with the previously-stored pre-set formatting data (function208). Method200may be repeated as needed for any number of different programs (function210), as desired.

As noted above, receiver no is able to interpret timed text103associated with a received television program or other video content because it has a priori information about the timed text format in pre-set formatting data105. This pre-set data105may be organized in any logical format, such as a template, an XML-type schema, a SMPTE-TT header format, or the like. In various embodiments, each particular program is associated with its own pre-set data. Alternatively, pre-set data105may be combined for multiple programs in some implementations, and/or individual programs (or groups of programs, e.g., programs appearing on a common channel, or different episodes of a common program) may have more custom and unique pre-set data105for the particular program(s).

Pre-set data may be received at any time, and according to any temporal basis (function202). As shown inFIG. 2, pre-set data105may be obtained in batches, or in any sort of “as-needed” or “on-demand” basis, as appropriate. Batch processing could represent any technique for obtaining pre-set data105for multiple programs at the same time, such as providing the data105as part of a program guide or other regular download to receiver no from a broadcaster106or other source. To that end, pre-set data105may be received via an out-of-band channel separate from the programming content, but still part of a DBS, cable or other distribution channel108. Other implementations may allow receiver no to request the pre-set data105from a service132available via the internet or another back channel, as desired. Still other embodiments could receive pre-set data105as part of a header or other structure provided at the beginning of on-demand or broadcast programming, and/or during any part of the program stream (which may be repeated at any desired frequency). Pre-set data105may therefore be provided automatically (e.g., as part of a regular batch download), in response to requests placed by the receiver110, as part of a broadcast or on demand stream, and/or in any other manner as desired. In some implementations, the receiver no may assume certain default values for pre-set data105if no explicit pre-set data105is received. Such default values may be associated with the content provider, the programming channel, the program itself, the manufacturer of the receiver110, the geographic region of the receiver110, and/or any other factors as desired.

Pre-set data105is stored within receiver110as appropriate (function204). As noted above, data105may be stored within memory118, within storage medium119, and/or in any other storage medium available to receiver110as desired.

Program content is received and decoded in any suitable manner (function206). In various embodiments, broadcast content may be received by controlling a tuner within receiver112to select an appropriate channel containing programming desired by the viewer; this programming is received and decoded as appropriate (function206). In various embodiments, content is received as an MPEG or similar multiplex that includes multiple component streams each containing video, audio, data or other information. In such embodiments, timed text103may be provided within a data stream, as desired, or in any other appropriate manner.

As noted previously, the timed text103associated with the particular program is interpreted using the information in pre-set format data105(function208) that was previously stored within receiver110. The format data105can be used to locate information, to decode information, or for any other purpose. As noted above, some embodiments could use format data105to describe an encapsulation wherein legacy data (e.g., EIA 608/708 data or the like) is contained within a timed text frame, thereby allowing for convenient extraction and decoding using conventional techniques. This encapsulation technique could be used to conveniently support existing programming that already contains caption data in a legacy format, since existing legacy data could simply flow as-is and converge later with the already existing timed text elements, as appropriate.

Generally speaking, the various functions and features of method200may be carried out with any sort of hardware, software and/or firmware logic that is stored and/or executed on any platform. Some or all of method200may be carried out, for example, by logic executing within receiver no inFIG. 1. In one embodiment, processor116executes software logic that performs each of the various functions shown inFIG. 2. Such logic may be stored in memory118or in any other storage available to processor116as desired. Hence, the particular logic and hardware that implements any of the various functions shown inFIG. 2may vary from context to context, implementation to implementation, and embodiment to embodiment in accordance with the various features, scenarios and structures set forth herein. The particular means used to implement each of the various functions shown inFIG. 2, then, could be any sort of processing structures that are capable of executing conventional software logic in any format. Such processing hardware may include processor116or other components of receiver110inFIG. 2, as well as any other processors or other components associated with any conventional television, receiver, media player and/or the like.

FIG. 3is a diagram showing an exemplary process for encoding and transmitting video content with timed text. As noted above, timed text103is formatted as desired by a broadcaster106, distributor104and/or producer102prior to broadcast (function302). The timed text103is described by pre-set data105that defines the various locations, lengths, formats and/or other parameters of the timed text103. Legacy closed caption data in CEA-608 or CEA-708 format, for example, could be encapsulated within a SMPTE-TT or other format. In this example, the pre-set data105would specify the locations of the various data fields in sufficient detail to allow the receiving device no to appropriately extract and process the encapsulated data. In many embodiments, the timed text103is included within the programming304itself, e.g., as part of an MPEG multiplex. Other embodiments could maintain the timed text103separately from the rest of the programming304, as desired.

The pre-set data105is provided to the receiving device110as appropriate.FIG. 3shows broadcaster106transmitting the pre-set data105and the programming content304to the receiver device110. As noted above, pre-set data105may be provided out-of-band as part of a program guide broadcast or the like, or the pre-set data105may be provided in-band at the beginning of a broadcast (or at any other suitable interval). The broadcast programming304and pre-set data105(which may be sent in-band or out-of-band) may be provided as part of a common broadcast across distribution channel108. Although not expressly shown inFIG. 3, an alternate embodiment could provide caption data (including timed text103and/or pre-set data105) from a separate source (e.g., server132inFIG. 1) other than the broadcast source that provides the video programming304itself.

The receiving device110is able to store the pre-set data105(function204) and to receive the broadcast programming304as described above (function206). The receiving device uses the a priori knowledge contained in the pre-set data105to process the timed text data103that is associated with programming304as desired (function208). Timed text may be interpreted to provide closed captioning, subtitles, or any other information as desired when the broadcast programming304is rendered to the viewer (function208).

Note that any number of alternate but equivalent implementations could be formulated in addition to the detailed examples that are set forth herein. Equivalent techniques could be incorporated into a placeshifting device such as any of the SLING products available from Sling Media of Foster City, Calif., for example. In such embodiments, caption data may be received and rendered at a placeshifting device, with the rendered caption data being provided within the placeshifted media stream. Alternately, timed text data103and/or format data105could be provided from the placeshifting device to a remotely-located media player (equivalent to receiver110), as desired. Still other embodiments could allow a personal or other computer system to allocate resources to receiving and processing pre-set data that describes timed text associated with a streaming or downloaded video program. Many other equivalent embodiments could be formulated as desired.

The general systems, structures and techniques described above may be inter-combined, enhanced, modified and/or otherwise implemented to provide any number of different features. Although the preceding discussion focuses primarily on broadcast sources of television, for example, equivalent embodiments could apply the same features in any other context, including any sort of satellite, cable, terrestrial or other transmittable format, as well as any sort of stored media format (e.g., DVD, including conventional DVD formats as well as any sort of BLU-RAY or other high definition disk format), streaming video format (e.g., streaming video over the Internet or another network, as well as streaming over a telephone or other wireless network), cinema format and/or the like.

The term “exemplary” is used herein to represent one example, instance or illustration that may have any number of alternates. Any implementation described herein as “exemplary” should not necessarily be construed as preferred or advantageous over other implementations. While several exemplary embodiments have been presented in the foregoing detailed description, it should be appreciated that a vast number of alternate but equivalent variations exist, and the examples presented herein are not intended to limit the scope, applicability, or configuration of the invention in any way. To the contrary, various changes may be made in the function and arrangement of the various features described herein without departing from the scope of the claims and their legal equivalents.