Method and apparatus for transmitting and receiving a content file including multiple streams

A method and apparatus is provided for transmitting and playing complex content including a plurality of content. A method of playing the complex content includes receiving, from a server, the complex content and classification information corresponding to the complex content, classifying the plurality of content according to the classification information, and playing the complex content based on the classified plurality of content.

PRIORITY

This application claims priority under 35 U.S.C. §119(a) to a Korean Patent Application filed in the Korean Intellectual Property Office on Mar. 5, 2010 and assigned Serial No. 10-2010-0020027, the entire disclosure of which is hereby incorporated by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates generally to a method and apparatus for transmitting and receiving a content file, and more particularly, to a content file transmission and reception method and apparatus for providing semantic configuration and role information of content including a plurality of stream groups.

2. Description of the Related Art

Recently, with the expansion of digital content markets and the diversity of available platforms, there is an increasing demand for complex content rather than single type content. For example, a main service stream and an additional information stream such as an advertisement stream may constitute complex content by being created as one service object rather than individual streams.

The recently produced and distributed content is characterized in that the content can be produced and distributed in real time, can be reconfigured and redistributed, and can be controlled according to user preference. The recent content reflecting these characteristics includes, for example, free content that users can freely use, copy, change, and redistribute without any special restrictions, and open content allowing the users to share creations such as texts, videos, and music.

Using free content and open content, many different types of content may be created by different producers according to their purposes and intentions. For example, using source content A, B, and C, a service provider may create content in which the source content A is its main content, and may also create content in which the source content B or C is its main content. Accordingly, the utilization of content depends on semantic configurations and roles of various source streams constituting the content.

However, conventional content delivery and provision technology have limitations in providing content, taking into account the semantic configurations and roles of content, because the technology merely focuses on providing delivered content in predetermined sizes and shapes at predetermined times.

SUMMARY OF THE INVENTION

Accordingly, the present invention is designed to address at least the problems and/or disadvantages discussed above and to provide at least the advantages described below. An aspect of the present invention is to provide a method and apparatus for recording and playing content including a plurality of stream groups.

Another aspect of the present invention is to provide a content providing method and apparatus for enabling free adjustment and control of content by providing semantic configuration and role information of the provided content.

In accordance with an aspect of the present invention, a method for receiving a plurality of content is provided. The method includes acquiring classification information from a server; and classifying the plurality of content provided from the server according to the classification information.

In accordance with another aspect of the present invention, a method for transmitting a plurality of content is provided. The method includes providing classification information to a reception apparatus; and providing, to the reception apparatus, the plurality of content information, which is classifiable based on the classification information.

In accordance with another aspect of the present invention, an apparatus for receiving a plurality of content is provided. The apparatus includes a receiver for receiving classification information and the plurality of content from a server; and a processor for classifying the plurality of content according to the classification information.

In accordance with another aspect of the present invention, an apparatus for transmitting a plurality of content is provided. The apparatus includes a data processor for generating classification information; and a transmitter for transmitting, to a reception apparatus, the plurality of content, which is classifiable based on the classification information.

DETAILED DESCRIPTION OF EMBODIMENTS OF THE INVENTION

FIG. 1is a diagram illustrating a content providing method for complex content including multiple content types when a plurality of content is provided in the same time period. Specifically,FIG. 1illustrates a content providing method for complex content consisting of multiple content types100, i.e., Content1and Content2, which are to be provided in a same time period102.

Referring toFIG. 1, because Content1and Content2are provided in the same time102or the same service time, a conventional (or legacy) player may not be able to determine how to render them. Especially, for unexpected content that is created and provided in real time, such as newsflashes or emergency messages, the conventional player has no way to control this content, taking into account the ongoing content.

FIG. 2is a diagram illustrating a service configuration for selecting an appropriate source stream from among a plurality of source streams according to a user environment and providing the selected stream to a user.

Referring toFIG. 2, it is assumed that streams1,2,3, and4include associated segments of content A200, and streams5and6include associated segments of content B202, which together, constitute complex content. A user terminal that is to receive the segments located in a time period206can receive 500-Kbit data. In this case, the terminal may not determine whether it should select any one of or both of content A and content B for playback because the terminal has no information about the relationship between the contents A and B. Therefore, the terminal will merely select a data stream corresponding to a size of 500 Kbits, which is its possible capability. Even in this case, however, because there are multiple segments206aand206bbeing transmitted in 500-Kbps data streams, as illustrated inFIG. 2, the terminal may not determine which data stream or segment it should select and receive to provide the service to the user in the time period206.

To address these and other disadvantages, a content providing method and apparatus in accordance with an embodiment of the present invention facilitate free adjustment and control of content according to various content types and user intentions by providing semantic configuration and role information of the content including stream groups consisting of a plurality of streams. These multiple streams may include additional streams that are generated and provided during service of main content. Herein, the terms “stream” and “content” will be used in the same meaning. Additionally, the streams or content may be transmitted in segments.

The information for content control, as will be described herein below, may be configured as a recording and delivery medium that stores and/or carries information. Although it will not be described herein in detail, the control information or the medium may be included in any data formats capable of recording, transmitting, delivering, and storing data.

Further, the complex content consisting of multiple source content types, as will be described herein below, may be configured as a logical object or an aggregate of streams, and an aggregate of complex source content may constitute one new content or object. Herein, the term “complex content” refers to an aggregate of multiple content types.

FIG. 3is a flowchart illustrating a method for transmitting multiple content types, i.e., complex content, according to an embodiment of the present invention.

Referring toFIG. 3, a server apparatus creates a plurality of content in step300, and identifies configuration information between the multiple content types in step302. Based on the identified configuration information, the server apparatus or a transmission apparatus generates classification information in step304, and provides the plurality of content, i.e., complex content, and the classification information to a reception apparatus or a client apparatus in step306.

FIG. 4is a flowchart illustrating a method for receiving complex content according to an embodiment of the present invention.

Referring toFIG. 4, a player or a client apparatus acquires complex content, which includes multiple content types, and acquires classification information for the complex content from a server apparatus in step400. The client apparatus may acquire the complex content and classification information from the server apparatus by first sending a request to the server apparatus, or may receive them from the server apparatus without a request.

After receiving the complex content and classification information, the client apparatus classifies the multiple content types depending on the classification information in step402, and processes the complex content according to the relationship between the identified multiple content types in step404. In step404, depending on the relationship between the multiple content types, the client apparatus may determine which content is main content and which content is additional content, and may also determine the number of additional content among the multiple content types. Based on the relationship between the identified multiple content types, the client apparatus may process the main content and additional content independently.

For example, if the additional content is about an emergency broadcast, the client apparatus may provide the additional content about the emergency broadcast while playing the main content. However, if the additional content is an advertisement, the client apparatus may provide the additional content in the middle of the main content, or after the main content is completed. In step406, the client apparatus plays the complex content.

FIG. 5is a block diagram illustrating a server apparatus500according to an embodiment of the present invention.

Referring toFIG. 5, a data processor502generates classification information, i.e., information for classifying or identifying multiple content types. A transmitter504transmits the classification information and the associated complex content to a client apparatus.

FIG. 6is a block diagram illustrating a client apparatus for receiving complex content according to an embodiment of the present invention.

Referring toFIG. 6, the client apparatus600includes a receiver602that receives classification information and associated complex content from a server apparatus. A processor604classifies multiple content types included in the complex content using the classification information, and provides content to a user on a display606. The processor604may identify a relationship between individual content among the multiple content types using the classification information, distinguish between main content and additional content depending on the identified relationship, and appropriately output the individual content on the display606.

FIGS. 7 to 11illustrate complex content structures according to an embodiment of the present invention.

Referring toFIGS. 7 and 8, a complex content structure704includes a configuration file700and complex content702including multiple streams or multiple content types. Accordingly, a receiving terminal (or client terminal) may identify information about the complex content702and802by receiving and checking the configuration files700and800, respectively, in advance.

FIGS. 9 to 11illustrate complex content structures according to an embodiment of the present invention.

Referring toFIGS. 9 to 11, complex content902,1006,1008,1104, and1106may be different according to their streams or content in terms of a unit/size in which the streams or content are transmitted/received.

The complex content described herein may be configured as an aggregate of multiple complex content1006and1008, as illustrated inFIG. 10, or may be configured as an aggregate of complex content1104and source content or source stream1106, as illustrated inFIG. 11.

More specifically, referring toFIG. 9. a configuration file900includes information about the complex content902including multiple streams, and a reception apparatus identifies semantic configuration and role information of the streams by checking the configuration file900. In accordance with an embodiment of the present invention, the configuration file900includes classification information for classifying the multiple streams, i.e., content types, included in the complex content902.

Referring toFIG. 10, a complex content structure1010includes a hierarchical structure. For example, an upper-level (or upper-layer) configuration file1000has two lower-level configuration files1002and1004, which include classification information for their associated complex content1006and1008, respectively. Upon receiving the complex content structure1010, a reception apparatus first analyzes the upper-level configuration file1000, identifies the configuration files1002and1004existing in a lower level of the configuration file1000as a result of the analysis, and then classifies complex content1006and1008according to the classification information of the identified configuration files1002and1004, respectively. If the upper-level configuration file1000has the configuration files1002and1004in its lower level as inFIG. 10, then the upper-level configuration file1000has reference information for its lower-level configuration files1002and1004as well.

Referring toFIG. 11, a complex content structure1110includes an upper-level configuration file1100having information about its lower-level configuration file1102and information about the source streams1106. Upon receiving the complex content structure1110, a reception apparatus analyzes the configuration file1100, and processes the complex content1106by checking the lower-level configuration file1102according to the classification information or reference information included in the configuration file1100, or processes the source streams1106by directly checking a configuration of the source streams1106.

The configuration files700,800,900,1000, and1100, as described above, providing information, such as information provision for services and setting/initialization, and may be Media Presentation Descriptions (MPDs), manifests, configurations, etc. Additionally, any information having the same meaning or the same role may be determined as the same information as the configuration information.

Although basic field and attribute information have been or will be provided in this specification, configuration file information described in each embodiment of the present invention may further include additional information, such as fields and attributes according to the environments and/or services in use, but it is not intended to limit the scope of the present invention.

Additionally, while logical semantics of information included in the configuration file are illustrated and described in this specification, they may be represented in various information representation structures or formats such as eXtensible Markup Language (XML), binary data, formatted information structures, according to the system, and service production or provision environments.

FIG. 12is a diagram illustrating a configuration file according to an embodiment of the present invention.

Referring toFIG. 12, the configuration file1200includes a top-level element1202having elements1204representing information about complex content, i.e., multiple streams, in its lower level. Detailed information about each stream may be identified by an attribute field1206. As to the stream transmission or provision unit, detailed information about each provision unit may be provided by the attribute field1206according to the system or service production and provision environments.

InFIG. 12, a new element or a stream type (streamType)1208is added in the attribute field1206of the stream element1204in order to provide semantic configuration and role information of streams to information in the configuration file1200. The stream type (streamType)1208represents its role or importance in the content, or a role of the stream itself. The stream type (streamType)1208, as in the example of Table 1 below, may provide information indicating that a stream is a main service in the content provision. The other attributes illustrated inFIG. 12, i.e., type, bitrate, and StartTime, are not directly related to the present invention, so a description thereof will be omitted.

Table 1 shows an example of the stream type (streamType)1208indicating whether the stream is a main service/content stream or an additional content stream of the content according to an embodiment of the present invention.

The stream type (streamType)1208may also describe a role of the content stream in detail, as shown in the example of Table 2. Table 2 shows an example of the stream type (streamType)1208indicating whether the stream is a content stream for an emergency message service, a stream for an advertisement, or a user-defined stream.

FIG. 13illustrates a configuration file structure1300for providing semantic configuration and role information of streams according to an embodiment of the present invention. However, unlike inFIG. 12, inFIG. 13, an attribute field1306of the stream element1204includes an “is_mainStream” attribute element1308.

Referring toFIG. 13, information provided by the “is_mainStream” element1308has the same meaning as the “streamType” element1208described in conjunction withFIG. 12. If a value of the “is_mainStream” element1308inFIG. 13is set as “true”, it indicates that the stream is a main service/content stream in the service provided. As another example, by setting an “emergency” field (not shown) as “true”, it may be used to indicate that the content stream plays a role of an emergency message.

In accordance with another embodiment of the present invention for providing semantic configuration and role information of complex content, an example of configuring a complex content structure including multiple streams or multiple content types in one file or formatted content storage format will be described below. In the following description, examples of a complex content structure including multiple content types/streams will be described according to the International Organization for Standardization (ISO)-based file format.

The ISO-based media file format is defined in “Information technology-coding of audio-visual objects—part12: ISO base media file format” of the ISO/IEC international standard 14496-12:2005. A file in this format includes media data of streams and metadata including information for playing the streams. A basic building block in the ISO-based media file format is called a “box”, and the box is an objected-oriented basic building block in the ISO-based media file format. Each box includes a header and a payload. A header represents a type of the box and a size of the box in bytes. A plurality of specified boxes are derived from a “full box” structure including a version number and a flag in the header. A box may include another box, and the ISO file format designates types of boxes allowable in specific types of boxes.

Media data of a stream is stored in a media data (mdat) box as an Audio/Video (A/V) file, metadata thereof is stored in a MOOV box, and a file type thereof is stored in a file type (ftyp) box. Accordingly, the ISO-based media file format includes a plurality of boxes including A/V data and detailed information about the A/V data. In this specification, the term “box” may also be referred to as a “data block” or a “container”.

A more detailed description of the ISO-based media file format will be omitted so as not to obscure the subject matter of the present invention.FIG. 14illustrates a conventional content storage format based on an ISO-based media file format. Although not illustrated inFIG. 14, a file type (ftyp) box includes information about a file type and a compatible file type. The compatible file type makes normal playback possible in a decoder.

Referring toFIG. 14, data in a MOOV box1402is configured in an object-based structure, and includes content information such as a frame rate, a bit rate, and an image size; synchronization (sync) information for supporting playback functions such as Fast forward (FF)/rewind (REW); and detailed information for playing a media file included in an mdat box1404. Because the MOOV box1402includes information about a total number of A/V data frames and a size of each frame, a client apparatus may provide A/V data included in the mdat box1404by parsing the MOOV box1402during playback.

The MOOV box1402includes track (trak) boxes1402aand1402bfor referencing streams that are recorded in the mdat box1404on a track basis.

The mdat box1404includes actual stream data corresponding to each track, and A/V data may be stored in the mdat box1404on a unit data basis, such as a frame and a sample.

Although the ISO/IEC 14496-12 standard-based media file format illustrated inFIG. 14has been considered so far by way of example, the complex content structure described in the present invention may be used in any ISO/International Electrotechnical Commission (IEC) 14496-12 standard-based file formats. In addition, the complex content structure may be applied to any available storage formats and recording media that are configured in the structured format, though not based on the ISO/IEC 14496-12 standard.

FIGS. 15 and 17illustrate storage formats including information for providing semantic configuration and role information of streams constituting content according to embodiments of the present invention.

Each of storage formats1500and1700illustrated inFIGS. 15 and 17includes a main content stream of a service and an additional content stream thereof, such as an advertisement or an emergency message, in one storage format. Herein, the storage format may also be referred to as a complex content structure.

InFIGS. 15 and 17, to provide semantic configuration and role information of streams, an additional media track (addm) is newly defined as “handler_type” of a reference type (reference_type) indicating reference information of each track as in Table 3.

The reference type (reference_type) is information included in track reference (tref) boxes1514band1714, and the tref boxes1514band1714may be used to connect with other tracks that the current tracks reference, or by which the current tracks are referenced, using the reference type (reference_type) and track IDs (track_ID). Therefore, by checking the reference type (reference_type) of the tref box in the complex content structure storage format according to an embodiment of the present invention, a client apparatus may determine that the track that the current track will reference, or by which the current track is referenced, is an additional media track (addm), i.e., a track including content other than the main service, e.g., an emergency message service or an advertisement.

More specifically, referring toFIG. 15, the complex content structure storage format1500includes a MOOV box1510with two track (trak) boxes1512and1514, which designate main service content1520aand additional service content1520bincluded in a mdat box1520, respectively. The track (trak) boxes1512and1514include track header (tkhd) boxes1512aand1514a, respectively, and the tkhd boxes1512aand1514ainclude track IDentifiers (Ids) for identifying their associated tracks1520aand1520bincluded in the mdat box1520, respectively.

InFIG. 15, the track reference box1514bincluding information for referencing the additional service content1520bis included in the track (trak) box1514corresponding to the additional service content1520b. The track reference box1514bincludes a reference type (reference_type) and a track ID. The client apparatus checks the reference type (reference_type) included in the track reference box1514b, and if the reference type (reference_type) is set as “addm”, the client apparatus determines that the track1520bcorresponding to the track (trak) box1514and including additional service content is additional content.

FIG. 16is a flowchart illustrating an operational method of a terminal receiving a complex content storage format, as illustrated inFIG. 15, according to an embodiment of the present invention.

Referring toFIG. 16, a terminal receives the complex content storage format1500in step1610, and checks the MOOV box1510in step1620. Although not illustrated inFIG. 16, a process of analyzing and checking a box to be analyzed in the storage format, such as the file type (ftyp) box1502, may be performed before step1620. The terminal checks the track (trak) boxes1512and1514in a lower level in step1630, and checks the presence/absence of the track reference (tref) box1514bin step1640. For the track with the track reference (tref) box1514b, the terminal checks the track reference (tref) box1514bin step1640.

The terminal checks a reference type (reference_type) of the track reference (tree box1514bin step1650, and determines in step1660whether a value of the reference type (reference_type) is “addm”. If a value of the reference type (reference_type) is “addm”, this indicates that the track is a track including content other than the main service, such as an emergency message service or an advertisement. Accordingly, the terminal may determine which type of track the track containing the main service content is, by checking a value of the track_ID field of the track reference (tree box1514bin step1670. All steps succeeding step1630of checking the track (trak) boxes may be repeated from the first track to the last track of the image file.

Although the process of checking boxes is included in the description of operations of all terminals, the order of checking the boxes should not necessarily follow the order in the drawing. The file format analysis process and its associated terminal operation, which have not been described herein in detail, may follow details in the ISO/IEC 14496-12 standard.

Referring again toFIG. 17, the complex content structure storage format1700includes a MOOV box1710with track (trak) boxes1712and1716, which designate main service content1720aand additional service content1720bincluded in an mdat box1720, respectively. The track (trak) boxes1712and1716include their associated track header (tkhd) boxes, respectively.

A track reference (tref) box1714included in the track (trak) box1712corresponding to the main service content1720aincludes a reference type (reference_type) and a track identifier (track_ID). The client apparatus checks the reference type (reference_type) included in the track reference (tref) box1714, and if the reference type (reference_type) is set as “addm”, the client apparatus determines that the track corresponding to the track (trak) box1712is a track including the main service content1720a.

FIG. 18is a flow chart illustrating an operational method of a terminal receiving a complex content storage format, as illustrated inFIG. 17, according to an embodiment of the present invention.

Referring toFIG. 18, a terminal receives a complex content storage format1700in step1810, and checks a MOOV box1710in step1820. Although not illustrated inFIG. 18, a process of analyzing and checking a box to be analyzed in the storage format1700, such as the file type (ftyp) box, may be performed before step1820. The terminal checks the track (trak) boxes1712and1716in a lower level in step1830, and checks the presence/absence of the track reference (tref) box1714in step1840. For the track with the track reference (tref) box1714, the terminal checks the track reference (tref) box1714in step1840.

The terminal checks the reference type (reference_type) in the track reference (tref) box1714in step1850, and if its value is set as “addm” in step1860, determining that the track is a track including the main service content1720a, the terminal may determine which type of track the track including content other than the main service (such as an emergency message service or an advertisement) is, by checking a value of the track_ID field in the track reference (tref) box1714in step1870. All steps succeeding step1830of checking the track (trak) boxes may be repeated from the first track to the last track of the image file.

Although the process of checking boxes is required in the description of operations of all terminals, the order of checking the boxes should not necessarily follow the order in the drawing. The file format analysis process and its associated terminal operation, which have not been described herein in detail, may follow details in the ISO/IEC 14496-12 standard.FIGS. 19 and 20are diagrams illustrating box configurations including a new moov-level box and a new moov-level meta box, respectively, according to embodiments of the present invention.

Specifically,FIGS. 19 and 20illustrate examples of new box structures1900and2000of a complex content storage format, which provide semantic configuration and role information of streams. Each of newly defined boxes1920and2070may be distinguished by defining its identifier as “uuid”, and setting a value of an ID field of “uuid” as a specific value indicating that the box is a box for including information described in the present invention. Alternatively, the newly defined boxes1920and2070may be defined by setting their unique names.

Referring toFIGS. 19 and 20, the new boxes may include information indicating the number of tracks including content other than the main service, such as an emergency message service or an advertisement, and also indicating the types of tracks. The new boxes1920and2070may be configured to include information representing detailed semantic and role information of content configuration, which indicates that the tracks constitute a main service stream, an advertisement, etc.

The method proposed in the present invention may configure new boxes of the storage format in various forms such as a file-level box, a moov-level box, a track-level box, a moov-level meta box, a track-level meta box, etc., specified in the ISO-based media file format. Although not be configured in boxes as illustrated inFIGS. 15 and 17, the required information may be added in fields.

FIG. 19illustrates an example of configuring a box including new information described herein in the form of a new moov-level box according to the second embodiment of the present invention, andFIG. 20illustrates an example of configuring a box including new information described herein in the form of a new moov-level meta box according to the second embodiment of the present invention.

InFIG. 19, the uuid box1920as newly defined is defined in a MOOV box1910, and an identifier of the uuid box1920is called “addm”. The MOOV box1910includes track (trak) boxes1930and1940, and the track (trak) box1930designates main service content1950awith track_ID=1 in an mdat box1950, while the track (trak) box1940designates additional service content1950bwith track_ID=2 in the mdat box1950.

The uuid box1920includes the number of additional media content types as information indicating how many additional content types are included in the mdat box1950. The number of additional media content types is defined in an “additional media count” field. The uuid box1920includes track IDs of the additional content types.

InFIG. 19, when the additional media count field included in the uuid box1920is set to 1, one additional content is present in the mdat box1950, and when track_ID is set to 2, the additional content is an additional content with track_ID=2.

FIG. 20illustrates a storage format2000configured by including a uuid box2070, not in a MOOV box2010, but in a moov-level meta box2060.

FIGS. 21 to 27illustrate complex content storage formats and meta information according to an embodiment of the present invention. Specifically, inFIGS. 21 to 27illustrate the meta information including classification information and the complex content storage format including complex content and information for playing the complex content may be configured separately, or the meta information may be included in the complex content storage format.

FIGS. 21 to 24illustrate meta information and complex content storage formats that are configured separately according to an embodiment of the present invention.

Referring toFIGS. 21 to 24, meta information or a meta information file may designate or reference the complex content storage format including actual content. For example, inFIG. 21, meta information2102designates a complex content storage format2100; inFIG. 22, meta information2202designates a complex content storage format2200; inFIG. 23, meta information2302designates complex content storage formats2300aand2300b; and inFIG. 24, meta information2402designates complex content storage formats2404and2406. For example, the meta information or meta information file may be set to designate or reference detailed streams or track (trak) boxes as illustrated inFIG. 22or24.

Additionally, the meta information or meta information file may include related information of the complex content configured in multiple content storage formats as illustrated inFIGS. 23 and 24.

When services are configured as illustrated inFIGS. 21 to 24, a terminal checks semantic configuration and role information of content streams by analyzing a meta file, and then analyzes an actual content storage format, streams, and/or tracks.

FIGS. 25 and 26illustrate meta information included in a complex content storage format according to an embodiment of the present invention.

Specifically,FIG. 25illustrates a complex content storage format2500, in which a meta information file2508is included in an XML or binary XML (bXML) box2506in a meta box2504. The meta box2504is in the same level as a MOOV box2502.

FIG. 26illustrates meta information2612in a complex content storage format2600by generating a new “admt” box2610including meta information in a meta box2608. The meta box2608is in the same level as a MOOV box2602.

The new boxes of the complex content storage format described above may be configured in various forms such as a file-level box, a moov-level box, a track-level box, a moov-level meta box, a track-level meta box, etc., specified in the ISO/IEC 14496-12 standard. Each of the newly defined boxes may be distinguished by defining its identifier as “uuid”, and setting a value of an ID field of “uuid” as a specific value indicating that the box is for including information as described in accordance with embodiments of the present invention. Alternatively, a newly defined box may be defined by setting its unique name such as “admt”, as illustrated inFIG. 26.

FIG. 27is a diagram illustrating meta information including classification information and a complex content storage format that are configured separately according to an embodiment of the present invention. Specifically,FIG. 27illustrates meta information2706including classification information and a complex content storage format2700that are configured separately. However, unlike inFIGS. 21 to 24, a terminal first analyzes the complex content storage format2700.

In this case, the terminal may check reference information for meta information or meta information file, using reference information-related boxes for the external data, content, or data source. While an item location box (iloc) and/or item information box (iinf) box2704of a meta box2702is used as a reference information-related box inFIG. 27, by way of example, other boxes can be used as reference information-related boxes in the complex content storage format2700, such as a data information box (dinf) and/or data reference box (dref), to provide reference information for the meta information2706or a meta information file.

If the meta information is included in the content storage format as illustrated inFIGS. 25 and 26, the terminal may analyze in sequence a meta box, XML or bXML in a lower level of the meta box, and meta information or meta information file in order of the box level and the box configuration, through a process of analyzing the storage format according to the details of the ISO/IEC 14496-12 standard.

In the following description, if the meta information2508and2612is included in the complex content storage formats2500and2600, respectively, as illustrated inFIGS. 25 and 26, the information may be analyzed according to a terminal operation inFIG. 28.

FIG. 28is a flow chart illustrating an operational method of a terminal that receives meta information included in a complex content storage format, as illustrated inFIGS. 25 and 26, according to an embodiment of the present invention.

Referring toFIG. 28, upon receiving a content storage format in step2810, a terminal checks a meta box in step2820. Before step2820of checking a meta box, the terminal may check a file type (fytp) box in advance. Using compatibility-related information, such as brand information, in the file type (fytp) box, the terminal may determine if the complex content storage format is a content storage format including the data information or meta information box described in the present invention.

The terminal identifies meta information by checking the meta information or meta information box in step2830, and checks a main media track and an additional media track, or semantic configuration and role of each track in step2840. Thereafter, the terminal may check a moov box and a track (trak) box in steps2850and2860, respectively.

If the meta information or meta information box is configured in a file-level box, a moov-level box, a track-level box, a moov-level meta box, or a track-level meta box, then the terminal may perform a step of analyzing an upper-level box of the relevant box, replacing step2820.

For example, if the meta information or meta information box is configured in a moov-level box, the terminal may receive a content storage format, determine if the received content storage format is a content storage format including meta information or meta information box, check a MOOV box first, and check meta information (admt) configured in its lower level in step2830.

FIG. 29is a diagram illustrating meta information according to an embodiment of the present invention.

Referring toFIG. 29, meta information2900includes, as a top-level element, a content element2902for indicating information about which content is included in the meta information2900, and a track element2904, which includes information about a plurality of tracks existing in the content. For each track element2904, an attribute field2906has a track ID (trackID)2908and a content type (contentType)2910, which is information describing a type of the content. The content type (contentType)2910may provide information indicating that the content is a main service in content provision, as in the example of Table 1. Also, the content type (contentType)2910may provide information indicating whether the stream is a content stream for an emergency message service or a stream for an advertisement, as in the example of Table 2.

The track element2904inFIG. 29may be replaced by information indicating information about the stream or content. For example, if the track is replaced by content, a service provided may include content A, content B, and content C, and attribute information of the content type (contentType)2910may provide information describing a semantic or a role of each of the content A, the content B, and the content C in providing complex content.

FIG. 30illustrates meta information provided in a unit of an “item”, whose semantic configuration and role information is a logical configuration unit, according to an embodiment of the present invention.

Referring toFIG. 30, the meta information3000includes a content element3002indicating information about which content is included in the meta information3000, and an item element3004, which includes information about the items existing in the content. For each item element3004, an attribute field3006has a reference track identifier (ref trackID)3008, an item identifier (item_ID)3010, and an item type (item_Type)3012, which is information describing a type of the item3004.

FIGS. 31 and 32illustrate meta information according to embodiments of the present invention. Specifically, inFIGS. 31 and 32, semantic configuration and role information of content may be provided by offering a specific method for providing content using meta information3100and3200.

Referring toFIG. 31, a terminal may determine whether the track/stream/content is set for mandatory provision, by checking a value of an is_mandatory element3108in an attribute field3106of a stream element3104, which is a lower-level element of a top-level ‘configuration’ element3102of the meta information3100. The is_mandatory element3108is mandatory provision attribute information indicating whether the track/stream/content must be provided, or is provided for an additional service. Based on this attribute information, the terminal may determine whether the track/stream/content is for a main service, or for an additional service such as an advertisement. If a value of the is_mandatory element, or the mandatory provision attribute information, is set as “true”, it indicates that the track/stream/content is for a main service. However, if a value of the is_mandatory element is set as “false”, it indicates that the track/stream/content is for an additional service.

Referring toFIG. 32, a terminal may determine whether the track/stream/content can be skipped, by checking a value of a “can_skip” element3208or “skip” attribute information in an attribute field3206of a lower-level stream element3204of a top-level configuration element3203of the meta information3200. If a value of the can_skip element3208or the skip attribute information is set as “true”, the terminal may determine that the track/stream/content corresponds to an additional service that can be skipped. However, if a value of the can_skip element3208is set as “false”, the terminal may determine that the track/stream/content corresponds to a main service that cannot be skipped. Based on this attribute information, the terminal may determine whether the track/stream/content is for a main service, or for an additional service such as an advertisement.

The information about the content providing method ofFIGS. 31 and 32may be further provided as additional information, together with configuration information and meta information, as in above-described embodiments of the present invention. For example, after a terminal has determined that the track/stream/content is for a main service, if a value of the can_skip attribute field of the track/stream/content is set as “true”, then the terminal may lower priority of the main service being provided, such as stopping the main service and providing another service.

The information about the content providing method ofFIGS. 31 and 32may be used to reflect opinions of content providers or service providers in providing service content, or to reflect terminal settings or user preferences in providing service content. This information may further include information reflecting opinions of content providers or service providers in providing service content, or may further include information reflecting terminal settings or user preferences in providing service content.

The meta information described herein may change a content provision method, a content display method, or a content storage method according to the semantic configuration and role information of content, or may further provide information about the content provision method, the content display method, or the content storage method according to the semantic configuration and role information of content.

As an example of the content display method, information for determining whether to provide data of streams1and2simultaneously, or whether to provide the stream2in the middle of the stream1, may be further added, and the content provision method may be determined based on the meta information described in the present invention.

As another example of the content display method, for video content, detailed information as shown in Table 4 may be provided by providing new attribute information for layout information, and the content provision method may be determined by checking the meta information described in the third embodiment.

As another example of the content display method, for audio content, unlike video content, if two different audio streams are played simultaneously, normal audio services may not be provided. While a service such as an advertisement is being provided according to the semantic configurations and roles of content, detailed service provision-related attribute information, such as reducing the volume for the main media, muting the main media, and skipping data of the main service, may also be provided as meta information. Also, the content provision method may be determined by checking the configuration information and meta information including the above-described semantic configuration and role information of content.

Additionally, the content stored in a receiving terminal may be controlled according to the configuration information and meta information including the above-described semantic configuration and role information of content. After undergoing a content provision process such as playing service content by checking configuration information and meta information including the semantic configurations and roles of content, the receiving terminal may select and store the content/stream/track according to the configuration information and meta information including the semantic configurations and roles of content.

For example, if complex content includes a stream1, a stream2, and a stream3as its main service content, and a stream4and a stream5as advertisement service content, stream1, stream2, and stream3are stored as main service content in a storage of the receiving terminal, and then configured in a separate media file and managed.

While a single media file is transmitted in the foregoing description, one file including a stream1, a stream2, and a stream3constituting the main service content and another file including a stream4and a stream5constituting the advertisement service content may be stored and managed as separate content files independently according to the configuration information and meta information including the semantic configurations and roles of content.

FIG. 33is a block diagram of a complex content file transmission apparatus according to an embodiment of the present invention.

Referring toFIG. 33, the complex content file transmission apparatus3300, such as a server apparatus500, as illustrated inFIG. 5, includes a data generator3310, an input unit3320, an image signal processor3330, an encoder3340, a file/stream creator3350, a content semantic configuration and role information generator3360, and a transmitter3370.

The data generator3310generates raw data such as texts, videos, audios, graphics, and metadata capable of constituting content, including image generation devices (e.g., camera and camcorder), acoustic devices, and text generators. Data output from the data generator3310is input to the input unit3320. The input unit3320provides the data received from the data generator3310to the image signal processor3330, in which the data output from the input unit3320undergoes preprocessing. The preprocessing may include all operations of converting an input image from analog to digital, and transforming input data using necessary external input values to provide the image data.

The data preprocessed by the image signal processor3330is output to the encoder3340. Although a storage structure for buffering is not illustrated separately between the components shown inFIG. 33, the storage structure may be provided depending on the generation devices, systems, and data generation methods. The encoder3340encodes the data provided from the image signal processor3330. The encoding operation of the encoder3340is optional. Additionally, if beneficial for a specific design, two or more encoders3340may be provided, making it possible to provide different types or classes of services.

The file/stream creator3350creates content files or data streams from the data encoded by the encoder3340according to the transmission format or unit, and outputs them to the content semantic configuration and role information generator3360. The content semantic configuration and role information generator3360generates semantic configuration and role information of content files according to an embodiment of the present invention. If a complex content file is configured to include the semantic configuration and role information of a content file, the file/stream creator3350may generate semantic configuration and role information of a content file. The semantic configuration and role information of a content file may be generated by a separate data generator and transmitted to a receiving terminal, when the file created by the file/stream creator3350is transmitted.

The content semantic configuration and role information generator3360generates information about relationships between and roles of the contents included in the complex content according to exemplary embodiments of the present invention. The information about relationships between and roles of the contents may be included in classification information for classifying the contents. The data generator3310, the input unit3320, the image signal processor3330, the encoder3340, the file/stream creator3350, and the content semantic configuration and role information generator3360may be included in a data processor502, as illustrated inFIG. 5.

FIG. 34is a block diagram illustrating a complex content file reception apparatus according to an embodiment of the present invention.

Referring toFIG. 34, the complex content file reception apparatus3400, such as client apparatus600, as illustrated inFIG. 6, includes a receiver3410for receiving complex content files from a server, a content semantic configuration and role information analyzer3420, a file/stream analyzer3430, a decoder3440, a player3450, and a display3460. The content semantic configuration and role information analyzer3420receives and analyzes the information generated by the content semantic configuration and role information generator3360inFIG. 33. The file/stream analyzer3430analyzes the file created by and received from the file/stream creator3350inFIG. 533.

If a complex content file is configured to include the content semantic configuration and role information, the file/stream analyzer3430may include a process of analyzing the content semantic configuration and role information. The decoder3440decodes the content data analyzed and extracted by the file/stream analyzer3430. The decoding process is performed, when the complex content file transmission apparatus3300encodes the file/stream using the encoder3340. The data encoded by more than one encoder, may be decoded by their associated decoder, or may be decoded selectively depending on the systems or terminals, making it possible to provide different types/classes of services.

Although a separate storage is not illustrated inFIG. 34, a storage for buffering or storing data may be provided between the components illustrated inFIG. 34.

The player3450plays the data provided from the decoder3440depending on the information analyzed by the content semantic configuration and role information analyzer3420and the file/stream analyzer3430. The display3460displays the content played by the player3450. This display3460refers to a final output device, or a physical output device, by which actual data is output and provided to the user. The content semantic configuration and role information analyzer3420, the file/stream analyzer3430, the decoder3440, and the player3450may be included in a processor604, as illustrated inFIG. 6.

The information described in embodiments of the present invention and used to classify contents included in complex content is called ‘classification information’. For example, the classification information may include ‘stream_type’; ‘is_mainStream’; reference_type and track_ID included in a track reference (tref) box; ‘additional media count’ and track_ID included in a meta box or metadata; and ‘content_type’, ‘item_Type’, ‘is_mandatory’, and ‘can_skip’.

The foregoing details of the above-described embodiments of the present invention are subject to change according to the system or service environments. For example, upon checking storage format configuration information for real-time services, a terminal may determine that an advertisement, which has not been received at a terminal but is to be provided in the future, will be received in data of an additional media track. When services are provided in real time or an unexpected emergency message is created, to provide the above-described content semantic configuration and role information, a process of updating and delivering configuration information, or retransmitting and re-receiving the configuration information may be performed.

As is apparent from the above-described embodiments of the present invention, in providing complex content including multiple source contents or streams, information about semantic configurations and roles of various source streams constituting the complex content is proved, thereby making it possible to utilize the content according to various content types and user intentions. Such content may be provided as user-adaptive content.