Patent Description:
More specifically, the present invention is directed to methods and devices for generating, encapsulating and parsing timed media data.

The invention is related to encapsulating, parsing and streaming media content, e.g. according to ISO Base Media File Format as defined by the MPEG standardization organization, to provide a flexible and extensible format that facilitates interchange, management, editing, and presentation of a group of media content and to improve its delivery for example over an IP network such as Internet using adaptive http streaming protocol.

The International Standard Organization Base Media File Format (ISO BMFF, ISO/IEC <NUM>-<NUM>) is a well-known flexible and extensible format that describes encoded timed media data bitstreams either for local storage or transmission via a network or via another bitstream delivery mechanism. This file format has several extensions, e.g. Part-<NUM>, ISO/IEC <NUM>-<NUM> that describe encapsulation tools for various NAL (Network Abstraction Layer) unit based video encoding formats. Examples of such encoding formats are AVC (Advanced Video Coding), SVC (Scalable Video Coding), HEVC (High Efficiency Video Coding) or L-HEVC (Layered HEVC). Another example of file format extensions is ISO/IEC <NUM>-<NUM> that describes encapsulation tools for still images or sequence of still images such as HEVC Still Image. This file format is object-oriented. It is composed of building blocks called boxes (or data structures characterized by a four character code) that are sequentially or hierarchically organized and that define descriptive parameters of the encoded timed media data bitstream such as timing and structure parameters. In the file format, the overall presentation over time is called a movie. The movie is described by a movie box (with four character code 'moov') at the top level of the media or presentation file. This movie box represents an initialization information container containing a set of various boxes describing the presentation. It is logically divided into tracks represented by track boxes (with four character code 'trak'). Each track (uniquely identified by a track identifier (track_ID)) represents a timed sequence of media data belonging to the presentation (frames of video, for example). Within each track, each timed unit of data is called a sample; this might be a frame of video, audio or timed metadata. Samples are implicitly numbered in sequence. The actual samples data are in boxes called Media Data Boxes (with four character code 'mdat') at same level as the movie box. The movie can also be organized temporally as a movie box containing information for the whole presentation followed by a list of couple movie fragment and Media Data box. Within a movie fragment (box with four character code `moof') there is a set of track fragments (box with four character code 'traf'), zero or more per movie fragment. The track fragments in turn contain zero or more track run boxes ('trun'), each of which document a contiguous run of samples for that track fragment.

In the file format, a media or presentation file may also contain one or more static items (e.g. one or more still images) described within a meta box (`meta') at same level as the movie box. This meta box may contain descriptive information describing static items, this descriptive information being organized in several boxes (for instance, the list of items in an item information box (`iinf') and the location (in data boxes) of data items in an item location box ('iloc')), each item being uniquely identified by an item identifier (item_ID). The actual items data are stored either in an item data box ('idat') in the meta box or in a media data box (`mdat') at file top level.

An ISOBMFF may contain multiple encoded timed media data bitstreams or sub-parts of encoded timed media data bitstreams forming multiple tracks and/or multiple static items. ISOBMFF and its extensions comprise several grouping mechanisms to group together tracks, static items or samples. A group typically shares common semantic and/or characteristics.

For instance, ISOBMFF comprises an entity group mechanism, a track group mechanism and a sample grouping mechanism. The entity grouping mechanism can be used to indicate that tracks and/or static items are grouped according to an indicated grouping type or semantic. The track grouping mechanism can be used to indicate that tracks are grouped according to an indicated grouping type or semantic. The sample grouping mechanism can be used to indicate that certain properties associated with an indicated grouping type or semantic apply to an indicated group of samples within a track.

The inventors have noticed several problems when describing and signaling a group of components in ISOBMFF using existing grouping mechanisms, a component being either an item (e.g. still image), a track (e.g. audio, video or metadata), or a group thereof (e.g. a track group or an entity group).

For instance, a problem is that the signaling of properties associated with a given component or common to all components belonging to a group of components may generate overhead and complexity.

Another difficulty is that the properties common to a component or to a group of components are not or do not vary dynamically over time.

Also, another problem involves the dynamicity over time of a list of components that belong to a group of components.

Generally speaking, the existing solutions do not allow this dynamicity.

The present invention has been devised to address one or more of the foregoing concerns.

In this context, there is provided a method for generating one or more media files based on timed media data according to claim <NUM>.

Optional features of the invention are further defined in the dependent appended claims.

According to embodiments, at least a part of the plurality of sets of properties is included in SampleGroupDescriptionBox in the descriptive metadata.

According to embodiments, one SampleGroupDescriptionBox is generated per track.

According to embodiments, one or more tracks corresponding to a same track group identifier are associated with the same set of properties from among the plurality of sets of properties.

According to embodiments, said single location is located at a higher level than the track level in the presentation file.

According to embodiments, the track group box indicates a grouping type associated with semantics describing at least one criterion used to form the group of tracks.

According to embodiments, the timed data are partitioned so that at least one of the time samples comprises subsamples.

According to embodiments, at least one tracks group is a composition group comprising subsamples of the different tracks belonging to the group.

According to embodiments, the track group box indicates an identifier of the group, the identifier being unique in the presentation file.

According to embodiments, the list of existing groups of tracks and properties shared by the tracks of each group are defined at movie level in the presentation file.

According to embodiments, for at least one track, further properties specific to this track are defined at track level in the presentation file.

According to embodiments, the group definition at movie level is associated with the group definition at track level by the same unique group identifier.

According to embodiments, for at least one track, further properties specific to this track are defined at said same single location in the presentation file.

According to embodiments, properties specific to said one track are directly defined in the list of existing groups of tracks.

According to embodiments, each property can take different values, and the value of each property specific to a track varies over time.

According to embodiments, the value of each property specific to the track is defined for one or more samples within said track.

According to embodiments, the descriptive metadata further comprise at least one box describing the samples within the tracks belonging to the same group of tracks, the at least one box defining the different possible values that can take the specific properties over time.

According to embodiments, the at least one box indicates a grouping type associated with semantics describing at least one criterion used to form a group of samples.

According to embodiments, the values taken by the properties of consecutive samples follow a repeated pattern and the at least one box defines the repeated pattern itself and its repetition.

According to embodiments, each property can take different values, and the presentation file is fragmented so that the at least one group of tracks is defined at fragment level, and the set of tracks composing a group of tracks varies over time.

According to embodiments, the track group box comprises a flag indicating whether the definition of the at least one group of tracks can vary or not over time.

According to embodiments, the definition of the at least one group of tracks includes: the properties shared by the tracks of the group, the values that can take each property, and/or the tracks composing the group.

According to a second aspect of the invention, there is provided a device for generating one or more media files based on timed media according to claim <NUM>.

The second aspect of the present invention has features and advantages similar to the first above-mentioned aspect.

Since the present invention may be implemented in software, the present invention may be embodied as computer readable code for provision to a programmable apparatus on any suitable carrier medium, and in particular a suitable tangible carrier medium or suitable transient carrier medium. A tangible carrier medium may comprise a storage medium such as a floppy disk, a CD-ROM, a hard disk drive, a magnetic tape device or a solid state memory device or the like. A transient carrier medium may include a signal such as an electrical signal, an electronic signal, an optical signal, an acoustic signal, a magnetic signal or an electromagnetic signal, e.g. a microwave or RF signal.

Further advantages of the present invention will become apparent to those skilled in the art upon examination of the drawings and detailed description. It is intended that any additional advantages be incorporated herein.

Embodiments of the invention are described below, by way of examples only, and with reference to the following drawings in which:.

In the following description, a movie is temporally partitioned into several parts, each part being referred to as a "fragment". Each fragment is included in one or more tracks, with associated descriptive metadata. Therefore, a fragment can be seen as a track portion.

In practice, in an ISOBMFF file, metadata are included in a movie box ('moov') and the bitstream is included in a media data box (`mdat'). The movie box can comprise the description of several tracks (for instance an audio track and a video track).

The use of fragments allows media data to be fragmented into several parts. One fragment is represented by a pair of movie fragment box ('moof') and media data box ('mdat'), the movie fragment box describing the content of the associated media data box.

Thus, in a fragmented ISOBMFF file, there will be a succession of box 'moov', 'moof' and 'mdat', 'moof' and 'mdat', 'moof' and 'mdat', etc. The first box 'moov' comprises data common to the whole ISOBMFF file.

In the following description, the equivalent of a track ('trak') for a fragmented ISOBMFF file is a `traf'.

In the context of the present invention, tracks may be grouped according to common properties, i.e. properties that all the tracks of the group present. Each track may, in addition, have other properties which are specific to it, i.e. which is not shared with all the tracks of its group.

According to a first embodiment, the presentation file comprises at least one track group box describing groups of tracks where all the tracks belonging to a same group share common properties. The list of existing groups of tracks is declared in one single location in the presentation file, and the common properties shared by the tracks of a group are also declared at this same single location in the presentation file.

According to a second embodiment, properties specific to a track (i.e. not shared by all the tracks of a group) take different values over time.

According to a third embodiment, the set of tracks composing a group of tracks varies over time.

According to a fourth embodiment, the track group box comprises a flag indicating whether the definition of the at least one group of tracks can vary or not over time. This definition may vary in terms of properties shared by the tracks of the group (i.e. grouping criterion), in terms of values that can take each property (specific to each track or common to all the tracks of the group), and/or in terms of the tracks composing the group.

<FIG> illustrates a typical client server system in which the invention may be used. The invention is not restricted to such system as it may concern the generation of media files that may be distributed in any way, not only by streaming over a communication network.

The system comprises at a server side: media encoders <NUM>, in particular a video encoder, a media packager <NUM> and a media server <NUM>. The media packager <NUM> comprises a NALU parser <NUM>, a memory <NUM> and an ISOBMFF (ISOBMFF) writer <NUM>. The media server <NUM> can generate a manifest file (also known as a Media Presentation Description (MPD) file)) <NUM> and media segments <NUM>. The system further comprises at client side a media client <NUM> with different modules: an ISOMBFF parser <NUM>, media decoders <NUM> in particular a video decoder, a display <NUM> and a HTTP client <NUM> that supports adaptive HTTP streaming, in particular parsing of streaming manifest <NUM> to control the streaming of media segments <NUM>. It also contains a module capable of performing operations on encoded bitstreams and/or decoded picture: the transformation module <NUM>.

Typically, the media client <NUM> requests the manifest file <NUM> in order to get the description of the different media representations available on the media server <NUM> that compose a media presentation. Accordingly, the media client <NUM> then requests the media segments <NUM> it is interested in. These requests are done via the HTTP module <NUM>. The received media segments are then parsed by the ISOBMFF parser <NUM>, decoded by the video decoder <NUM>, optionally transformed through the transformation unit <NUM> to be played on the display <NUM>.

A video sequence is typically encoded by the video encoder <NUM> to be encapsulated into one or several files by the media packager <NUM>. The generated files are made available to clients by the Media server <NUM>.

According to some embodiments of the invention, another module is included in the system as part of the media packager or outside the media packager: the generation unit <NUM>. This unit allows a user to watch the encapsulated media tracks and to edit and modify them by applying various image operations onto the samples. This generation unit interacts with the media packager when the user wants to insert the result of his composition as an additional track in the media file produced by the ISOBMFF Writer.

The media server is optional in the sense that the invention mainly deals with the media encapsulation and construction/description of tracks resulting from composition, derivation, combination, transformation or aggregation of one or more media tracks. As for the media server, the transmission part (HTTP module and manifest parser) is optional in the sense that the invention also applies for a media client consisting in a simple media player to which the encapsulated media file is provided for rendering. The media file can be provided by full download, by progressive download, by adaptive streaming or just by reading the media file on a disk or from a memory.

The method according to the invention is performed by the media packager module <NUM>, and more specifically by the ISOBMFF writer module <NUM> in cooperation with the composition unit <NUM>, comprising software code, when executed by CPU <NUM> of server apparatus as illustrated in <FIG>.

Typically, the encapsulation module is in charge of reading high-level syntax of encoded timed media data bitstream, e.g. composed of compressed video, audio or metadata, to extract and identify the different elementary units of the bitstream (e.g. NALUs from a video bitstream) and organize encoded data in an ISOBMFF file or ISOBMFF segments <NUM> containing the encoded video bitstream as one or more tracks with descriptive metadata according to the ISOBMFF box hierarchy.

According to particular embodiments, encoded timed media data bitstreams are encapsulated into a file or into small temporal segment files according to an encapsulation file format, for instance ISO Base Media File Format (ISO/IEC <NUM>-<NUM> and ISO/IEC <NUM>-<NUM>), Omnidirectional MediA Format (OMAF) (ISO/IEC <NUM>-<NUM>) and associated specifications, and possibly with one or more still image bitstreams according to the Image File Format (ISO/IEC <NUM>-<NUM>) as defined by the MPEG standardization organization.

An encoded bitstream (e.g. HEVC, AVC) and possibly its sub-bitstreams (e.g. tiled HEVC, MV-HEVC, scalable HEVC), can be encapsulated as one single track. Alternatively multiple encoded bitstreams that are spatially related (i.e. are sub-spatial parts of a panorama image) can be encapsulated as several sub-picture tracks. Alternatively, an encoded bitstream (e.g. tiled HEVC, MV-HEVC, scalable HEVC) comprising several sub-bitstreams (tiles, views, layers) can be encapsulated as multiple sub-picture tracks.

A sub-picture track is a track embedding data for a sub part of a picture or image. A sub-picture track may be related to other sub-picture tracks or to the track describing the full picture (also known as panorama picture) the sub-picture is extracted from. For example a sub-picture track can be a tile track. It can be represented by an AVC track, an HEVC track, an HEVC tile track or any compressed video bitstream encapsulated as a sequence of samples.

A tile track is a sequence of timed video samples corresponding to a spatial part of an image or to a sub-picture of an image or picture. It can be for example a region of interest in an image or an arbitrary region in the image. The data corresponding to a tile track can come from a video bitstream or can come from a sub part of a video bitstream. For example a tile track can be an AVC or HEVC compliant bitstream or can be a sub-part of AVC or HEVC or any encoded bitstream, like for example HEVC tiles. In a preferred embodiment a tile track is independently decodable (encoder took care to remove motion prediction from other tiles). When a tile track corresponds to a video bitstream encoded in HEVC with tiles, it can be encapsulated into an HEVC Tile track denoted as 'hvt1' track as described in ISO/IEC <NUM>-<NUM>4th edition. It can then refer to a tile base track to obtain parameter sets, high level information to set up the video decoder. The tile track can also be encapsulated into an independently decodable HEVC track 'hvc1' or 'hev1' track by including and repeating in each tile track the parameter sets (also known as system tiles). A tile track can be used for spatial composition of sub-pictures into a bigger image or picture.

A tile base track is a track common to one or more tile tracks that contain data or metadata that is shared among these one or more tracks. A tile base track may contain instructions to compose images from one or more tile tracks. Tile tracks may depend on a tile base track for complete decoding or rendering. When a tile base track derives from a video bitstream encoded in HEVC with tiles, it is encapsulated into an HEVC track denoted as 'hvc2' or `hev2' track. In addition it is referenced by HEVC tile tracks via a track reference 'tbas' and it shall indicate the tile ordering using a `sabt' track reference to the HEVC tile tracks as described in ISO/IEC <NUM>-<NUM>4th edition.

A composite track (also denoted reference track) is a track that refers to other tracks to compose an image. One example of composite track is, in case of video tracks, a track composing sub-picture tracks into a bigger image. This can be done by post-decoding operation, for example in a track deriving from video tracks that provides transformation and transformation parameters to compose the images from each video track to a bigger image. A composite track can also be a track with extractor NAL units providing instructions to extract NAL units from other video tracks or tile tracks to form before decoding a bitstream resulting from sub-bitstream concatenation. A composite track can also be a track that implicitly provides composition instructions, for example through track references to other tracks.

<FIG> illustrates a first scenario where the partition of an image (denoted full image) into several sub-spatial parts may vary over time according to an embodiment of the invention. For instance, images <NUM>, <NUM>, <NUM> and <NUM> represent the same full image at different temporal instants. For instance, from time t=N, image <NUM>, to time t=N+M, image <NUM>, the partition grid may be equally divided between four sub-pictures <NUM>, <NUM>, <NUM>, <NUM>, all sub-pictures, having the same height and width sizes. Then from time t=N+M+<NUM>, image <NUM> to time t=N+M+Z, image <NUM>, the partition grid and sizes of each sub-picture <NUM>, <NUM>, <NUM> and <NUM>, are changed to obtain the new sub-pictures respectively <NUM>, <NUM>, <NUM> and <NUM> while the size of the composition picture (indicated by composition_height and composition_width) remains the same. In this scenario, the composition picture is the full image. Each corresponding sub-picture over time (e.g. <NUM> and <NUM>) forms a sub-picture track for which spatial properties (e.g. spatial coordinates: horizontal and vertical positions of the upper left corner (x, y), width and height) may vary at some temporal instants.

<FIG> illustrates a second scenario where the partition grid of an image (full image), denoted over different temporal instants <NUM>, <NUM>, <NUM>, <NUM>, is not varying over time. However, on contrary, the list of sub-pictures composing a given group of sub-pictures denoted G1 is varying over time. For instance, from time t=N, image <NUM>, to time t=N+M, image <NUM>, a group of sub-pictures G1 <NUM>, <NUM>, <NUM>, <NUM> is composed of sub-pictures at the intersection of first and second columns with second and third rows. Then from time t=N+M+<NUM>, image <NUM> to time t=N+M+Z, image <NUM>, the list of sub-pictures composing the group becomes the sub-pictures at the intersection of second and third columns with third and fourth rows. In this <FIG>, composition_ height and composition_width indicate the size of the composition picture G1 that represents a sub-picture composition within the full image. Each corresponding sub-picture over time (e.g. at second column and third row <NUM> and <NUM>) forms a sub-picture track for which spatial properties (horizontal and vertical positions of the upper left corner (x, y), width and height) do not change over time in the scope of the full image, but are varying in the scope of the group of sub-pictures G1. Similarly, in this scenario, the spatial properties (x, y, width and height) of the group of sub-picture G1 may be varying in the scope of the full image <NUM>, <NUM>, <NUM>, <NUM>.

Respective sizes of sub-pictures and composition pictures may be expressed for instance in luma pixels or in normalized coordinates.

In a third scenario, both the partition grid and the list of sub-pictures composing a group of sub-pictures may also vary over time simultaneously.

The present invention has been devised to address one or more of the foregoing scenarios.

ISO/IEC <NUM>-<NUM> provides a box 'trgr' located at track level (i.e. within the 'trak' box in ISOBMFF box hierarchy) to describe groups of tracks, where each group shares a particular characteristic or the tracks within a group have a particular relationship. This track group box is an empty container defined as follows:
<IMG>.

This track group box can contain a set of track group type boxes defined as follows:
aligned(<NUM>) class TrackGroupTypeBox(unsigned int(<NUM>)
track_group_type)
extends FullBox(track_group_type, version = <NUM>, flags
= <NUM>)
{
unsigned int(<NUM>) track_group_id;
II the remaining data may be specified for a particular
track_group_type
}.

The particular characteristic or the relationship declared by an instance of this track group type box is indicated by the box type (track_group_type). This box also includes an identifier (track_group_id), which can be used to determine the tracks belonging to the same track group. All the tracks having a track group box with a track group type box having the same track_group_type and track_group_id values are part of the same track group. The box also allows declaration of specific parameters associated with the track for a particular track group type. For instance, the MPEG OMAF standard (ISO/IEC <NUM>-<NUM>) defines a specific track group for spatial composition as a TrackGroupTypeBox of type 'spco' as defined below:
aligned(<NUM>) class SubPictureCompositionBox extends
TrackGroupTypeBox('spco') {
SubPictureRegionBox();
II optional boxes
}
aligned(<NUM>) class SubPictureRegionBox extends
FullBox('sprg',<NUM>,<NUM>) {
unsigned int(<NUM>) track x;
unsigned int(<NUM>) track_y;
unsigned int(<NUM>) track_width;
unsigned int(<NUM>) track_height;
unsigned int(<NUM>) composition width;
unsigned int(<NUM>) composition_height;
}.

Where track_x, track_y are the coordinates of the track within the spatial composition, track_width and track_height define the size in luma pixels of the track within the spatial composition and composition_width and composition_height indicate the size in luma pixels of the overall spatial composition.

The samples of each track in an 'spco' track group can be spatially composed with samples (at the same composition or decoding time) from other tracks in this same group to produce a bigger image.

Another example of track grouping is the signaling of pair of tracks that composes stereoscopic media content, for instance the TrackGroupTypeBox of type 'ster' as defined below:
aligned(<NUM>) class StereoVideoGroupBox extends
TrackGroupTypeBox('ster') {
unsigned int(<NUM>) left_view_flag;
bit(<NUM>) reserved = <NUM>;
}.

Where left_view_flag signals which track is the left (value <NUM>) or right (value <NUM>) view.

Describing groups of tracks as described above presents several drawbacks:.

According to a first embodiment, the list of existing groups of tracks is declared in one single location at higher level of the media or presentation file and group parameters or properties common to all tracks composing a group of tracks are declared in this same single location instead of repeating them in all tracks.

This embodiment may rely on the entity group mechanism as defined in ISO/IEC <NUM>-<NUM>:<NUM> Amd2. An empty container box (Groups List box ('grpl')) located in a file-level Meta Box (`meta') can contain a list of group description boxes called EntityToGroupBox as defined below:
<IMG>.

Where the box type (grouping_type) indicates the grouping type of the entity group (or component group). An entity is equivalent to a component as previously described.

In an alternative this empty container box (Groups List box ('grpl')) can be defined in a file-level Movie Box ('moov') or file-level Meta Box (`meta') if there is no Movie box.

Similarly to the track_group_type in TrackGroupTypeBox, each grouping_type code is associated with semantics describing the grouping.

Thus a specific spatial composition group of type 'spec' is defined at two different levels by associating an EntityToGroupBox with TrackGroupTypeBoxes when both EntityToGroupBox@grouping_type equal TrackGroupTypeBox@track_group_type and EntityToGroupBox@group_id equal TrackGroupTypeBox@track_group_id.

First, at movie level the dedicated EntityToGroupBox of type 'spec' (called SubPictureCompositionGroupBox) is defined in a GroupsListBox as follows:
aligned(<NUM>) class SubPictureCompositionGroupBox
extends EntityToGroupBox ('spco') {
SubPictureCompositionRegionBox;
II optional boxes
}
aligned(<NUM>) class SubPictureCompositionRegionBox extends
FullBox('scrg',<NUM>,<NUM>) {
unsigned int(<NUM>) composition_width;
unsigned int(<NUM>) composition_height;
}.

The list of tracks composing this spatial composition group is then provided using the list of entity_id of the EntityToGroupBox. The properties common to all tracks in this group are mutualized and defined once by defining a new SubPictureCompositionRegionBox in the SubPictureCompositionGroupBox.

composition_width and composition_height provide the size of the composition in luma pixels.

Second, at track level a track group with a TrackGroupTypeBox of type 'spco' (called SubPictureCompositionBox) is defined in all tracks composing the same spatial group as defined below:
aligned(<NUM>) class SubPictureCompositionBox extends
TrackGroupTypeBox('spco') {
SubPictureRegionBox();
II optional boxes
}
aligned(<NUM>) class SubPictureRegionBox extends
FullBox('sprg',<NUM>,<NUM>) {
unsigned int(<NUM>) track x;
unsigned int(<NUM>) track_y;
unsigned int(<NUM>) track_width;
unsigned int(<NUM>) track_height;
}.

Third, the group definition at movie level and at track level are associated by defining the same group identifier (i.e. a same group is defined by its couple grouping_type/group_id for EntityToGroupBox or track_group_type/track_group_id for a TrackGroupTypeBox). For instance, for the spatial composition group, when the parameter SubPictureCompositionBox@track_group_id equals to the SubPictureCompositionGroupBox@group_id and SubPictureCompositionBox@track_group_type equals to SubPictureCompositionGroupBox@grouping_type, both structures belong to the same group.

As an alternative to the previous example, rather than combining an existing box at movie level (e.g. EntityToGroupBox) with an existing box at track level (e.g. TrackGroupTypeBox), it is also possible to define a new EntityToGroupBox box (e.g. ComponentToGroupBox) that contains all information related to a group of components in one single place. The set of group of components is contained in an empty container box (e.g. Groups List box ('grpl')) located in a high-level Box (e.g. 'moov', `moof', or 'meta' at either file-level, movie-level or movie-fragment-level, or also 'udta'). The ComponentToGroupBox contains the list of components (item, track, group of components) that compose the group of components, the properties associated with the group shared by all members of the group, and for each member, the properties specific to this member within the group of components. The ComponentToGroupBox can be defined as a class GroupOfComponents as follows:
class GroupOfComponents {
groupOfComponent_type;
group_id;
num_component_in_sets;
for(<NUM> to num component in sets) {
referenceToComponent();
optionalComponentProperties();
}
optionalGroupOfComponentProperties();
}.

Where groupOfComponent_type is the type that characterizes the group of components, group _id is a unique identifier of the group in the scope of the presentation, num_component_in_sets is the number of components in the group (e.g. number of tracks in the group), reference To Component () is an information to identify a component belonging to the group (e.g. component's unique identifier), optionalComponentProperties () are optional properties of the component in the group and optionalGroupOfComponentProperties () are optional properties shared by all components of the group.

The ComponentToGroupBox can also be defined according to ISOBMFF syntax as follows:
<IMG>
<IMG>.

where the box type (grouping_type) indicates the type of grouping of the group of components.

group_id and component_id are unique identifiers respectively of the group and of the components that belong to the group in the scope of the whole presentation. group_id shall not be equal to any group_id value of any other ComponentToGroupBox, any item_ID value of the hierarchy level (file, movie, or track) that contains the GroupsListBox, or any track_ID value (when the GroupsListBox is contained in the file level).

The ComponentInComponentToGroupBox allows defining optional specific parameters associated with a given component (identified by component_id) for a group of type grouping_type. The values of these specific parameters apply in the scope of the embedding group with identifier group_id and type grouping_type.

Similarly the ComponentToGroupBox allows defining optional parameters associated with a group of type grouping_type that are shared by all components belonging to this group.

For instance, a group declaring stereoscopic video pair can be described as follows:
aligned(<NUM>) class ComponentToGroupBox ('ster', version, flags)
extends FullBox('ster', version, flags) {
unsigned int(<NUM>) group id;
unsigned int(<NUM>) num_entities_in_group;
for(i=<NUM>; i<num_entities_in_group; iff)
{
ComponentInComponentToGroupBox('ster');
}
}
aligned(<NUM>) class ComponentInComponentToGroupBox ('ster')
extends FullBox('ster', version = <NUM>, flags = <NUM>)
{
unsigned int(<NUM>) component_id;
unsigned int(<NUM>) component_is_left;
}.

Where the property component_is_left is associated with each component belonging to a group of components of type 'ster' to signal if the component is the left or right view.

In another alternative, the ComponentToGroupBox can be defined according to ISOBMFF syntax as follows:
<IMG>
Where:.

According to this alternative of the ComponentToGroupBox, the PropertyContainerBox and embedded properties are defined as follows:
aligned(<NUM>) class ComponentProperty(property_type)
extends Box(property_type)
{
}
aligned(<NUM>) class ComponentFullProperty(property_type,
version, flags)
extends FullBox(property_type, version, flags)
{
}
aligned(<NUM>) class PropertyContainerBox
extends Box('ppco')
{
properties Property() [];
}.

Where Property boxes derive from ComponentProperty or Component Full Property to fill box.

By either defining rules to combine EntityToGroupBox and TrackGroupTypeBox, or defining a new grouping box ComponentToGroupBox containing the group definition, common group properties and component properties within the given group as described above, it is then possible to declare the list of all existing groups at movie level possibly without having to parse all tracks and to mutualize common properties within a group of tracks at a single location without redundancy.

According to the invention, the entity group mechanism and the track group mechanism are combined as described in the previous embodiment and in addition a sample grouping mechanism is used to allow the properties associated with a track within a group of tracks to vary over time.

The sample grouping mechanism allows certain properties corresponding to an indicated grouping type to be associated with a group of samples within a track.

This mechanism relies on the use of two boxes:.

A particular type of sample grouping is defined by the combination of one SampleToGroup box and one SampleGroupDescription box via a type field ('grouping_type').

Multiple sample grouping instances (i.e. pair of SampleToGroup and SampleGroupDescription boxes) may exist based on different grouping criteria.

The SampleToGroup box is defined as follows:
<IMG>.

Where grouping_type identifies the type (i.e. the criterion used to form the sample groups) of the sample grouping and links it to its sample group description table with the same value for grouping type.

In some cases, consecutive samples in sample grouping may follow some repeated patterns, for instance, an Intra frame I followed by N inter-frames P repeatedly, e.g., IPPPIPPP.

In such a case, it may be desirable to use a more compact SampleToGroup box based on the pattern where it is listed as a sample count, and pattern repetition.

Thus, an alternative to the SampleToGroup box, more compact, is defined as follows:
<IMG>
<IMG>
Where:.

In an alternative, the parameter is_traf_sgpd is removed and the following rule is applied to sample_group_description_index values: Within the CompactSampleToGroupBox in a movie fragment, the group description indexes (sample_group_description_index) for groups defined within the same fragment start at 0x10001, i.e. the index value <NUM>, with the value <NUM> in the top <NUM> bits. Otherwise, for value below 0x10001, the group description index refers to index in the SampleGroupDescriptionBox present in the sample table in the MovieBox. This means there must be fewer than <NUM> group definitions for this track and grouping type in the sample table in the MovieBox.

The indicated pattern will be repeated a number of times equal to repeat_count (<NUM> means pattern occurs only once) when mapping to samples. If the repeated pattern exceeds the number of samples in the track/traf, it is truncated. If it is less, the remaining samples are mapped to default group index or not mapped.

The definition of this CompactSampleToGroupBox currently supports/preserves the default sample grouping mechanism, i.e. samples that are not mapped to any sample group description entry of a given grouping_type are mapped to the sample group description entry provided by the index default_sample_description_index defined in the SampleGroupDescriptionBox of this grouping_type.

This new CompactSampleToGroupBox can be advantageously used, for instance, to signal patterns in temporal sub-layers with the generic Temporal Level sample grouping ('tele') defined in ISO/IEC <NUM>-<NUM> or with the HEVC Temporal layer sample grouping ('tscl') defined in ISO/IEC <NUM>-<NUM>.

For instance, the Temporal Level sample grouping ('tele') allows association of a given generic temporal level identifier (in this case using sample group description index, the order in index value providing a level of importance) with a flag 'level_independently_decodable' signaling if the level is independently decodable or not. Thus, the client can know which set of samples can be dropped depending on its associated level and flag.

If necessary, the HEVC Temporal layer sample grouping ('tscl') provides even more precise information in the context of HEVC encoding by assigning a given temporal layer (temporalLayerId) with each sample and thus determining whether a sample can be dropped depending on properties associated with a given temporal layer identifier.

According to the second embodiment, the properties of a track within a group that vary over time are defined as sample group entries in SampleGroupDescription box ('sgpd') with a grouping_type equal to the grouping_type of the associated group (EntityToGroupBox@grouping_type and/or TrackGroupTypeBox@grouping_type). In addition, the corresponding SampleToGroup box has the same grouping_type and the grouping_type_parameter equals the group identifier (EntityToGroupBox@group_id and/or TrackGroupTypeBox@track_group_id) in order to associate the sample grouping with a given group of tracks.

Within a track, when same parameters are defined in both a TrackGroupTypeBox and a VisualSampleGroupEntry that have the same grouping_type and a track_group_id equal to grouping_type_parameter respectively, the parameter values of the VisualSampleGroupEntry supersede the parameter values of the TrackGroupTypeBox.

For instance, to address the scenario of the <FIG>, a new sample group entry SubPictureRegionEntry with a grouping_type equal to 'spco' is defined as follows:
aligned(<NUM>) class SubPictureRegionEntry extends
VisualSampleGroupEntry('spco') {
unsigned int(<NUM>) track x;
unsigned int(<NUM>) track_y;
unsigned int(<NUM>) track_width;
unsigned int(<NUM>) track_height;
}.

This SubPictureRegionEntry provides the coordinates (track_x, track_y) and size (track_width, track_height) that may vary over time of the one or more sub-pictures carried by a track within the spatial composition group described by a SubPictureCompositionGroupBox and/or SubPictureCompositionBox for which EntityToGroupBox@group_id, TrackGroupTypeBox@track_group_id, and SampleToGroup@grouping_type_parameter are equal.

As an alternative, the sample grouping mechanism described above can also be combined with the ComponentToGroupBox to vary along time parameters associated with a component within a group of components. Similarly, within a track, the SampleToGroup box has the same grouping_type and the grouping_type _parameter is equal to the group identifier (ComponentToGroupBox@group_id) in order to associate the sample grouping with a given group of components. When the same parameters are defined in both a ComponentInComponentToGroupBox and a VisualSampleGroupEntry with the same grouping_type in a given track and a component identifier within this group ComponentInComponentToGroupBox@component_id is equal to the track identifier track_ID, and the ComponentInComponentToGroupBox belongs to a group with group _id equal to grouping_type_parameter, then the parameter values of the VisualSampleGroupEntry supersede the parameter values of the ComponentInComponentToGroupBox.

By defining rules to combine EntityToGroupBox, TrackGroupTypeBox and sample grouping mechanism as described above, it is then possible to vary over time some properties associated with a track within a particular group of tracks.

<FIG> illustrates another alternative where it is also possible to describe spatial relationship between multiple tracks using only track grouping <NUM> and sample grouping <NUM> and <NUM> by defining a grouping type 'srd' as follows:
The 'srd ' track_grouping_type indicates that this track belongs to a group of tracks corresponding to spatial parts of a video. The tracks that have the same value of track_group_id within a TrackGroupTypeBox of track_group_type 'srd' are mapped as being originated from the same source. For example, a very high resolution video may have been split into sub-pictures tracks for complexity reason. Each sub-picture track then conveys its position and sizes in the source video. The 'srd ` track group may convey the size of the source video.

The SpatialRelationshipDescriptionBox (or SpatialRelationshipTrackGroupBox <NUM>, name is here as an example) is defined as follows:
aligned(<NUM>) class SpatialRelationshipDescriptionBox
extends TrackGroupTypeBox ('srd ', version, flags)
{
II the track_group_id is inherited from
TrackGroupTypeBox;
II data specified for particular 'srd ' track_group_type
if (version == <NUM>) { // single grouping
unsigned int(<NUM>) total width;
unsigned int(<NUM>) total_height;
}
else if (version == <NUM>) { // several sub-groups
unsigned int(<NUM>) num_sub_groups_minus_one;
for (set=<NUM>; set <= num_sub_groups_minus_one; set++) {
unsigned int(<NUM>) sub_group_id;
unsigned int(<NUM>) total_width;
unsigned int(<NUM>) total_height;
}
}
}
Where.

In addition, a 'srd ' grouping_type for sample grouping is defined for the sub-picture tracks to declare their positions and sizes in a spatial relationship. When version <NUM> of the SpatialRelationshipDescriptionBox is in use, a version <NUM> of the SampleToGroupBox shall be used with the grouping _type_parameter value identifying the corresponding 'srd ' sub_group_id. When version <NUM> of the SpatialRelationshipDescriptionBox is in use, either version <NUM> of the SampleToGroupBox can be used or a version <NUM> with grouping_type_parameter set to <NUM>.

A specific VisualSampleGroupEntry is defined: SpatialRelationshipGroupEntry ('srd ') as follows, and illustrated as <NUM> on <FIG>:
class SpatialRelationshipGroupEntry () extends
VisualSampleGroupEntry ('srd ')
{
unsigned int(<NUM>) object_x;
unsigned int(<NUM>) object_y;
unsigned int(<NUM>) object width;
unsigned int(<NUM>) object_height;
}.

Where object_x, object_y, object_width, object_height indicate the moving coordinates of the track in the referential defined by total_width and total height.

'srd ' track group or sub-group respectively allows the reconstruction of the full video source or a partial video source. The reconstruction is done as follows:.

The sub-picture rectangles of all tracks belonging to the same ' srd ` track group with the same spatail_set_id and belonging to different alternate groups (i.e., having alternate_group equal to <NUM> or different alternate_group values) shall not overlap and shall not have gaps, such that in the above derivation process for the video source each luma sample position (x, y), where x is in the range of <NUM> to total_width - <NUM>, inclusive, and y is in the range of <NUM> to total_height - <NUM>, inclusive, is traversed exactly once.

In another alternative, the SpatialRelationshipDescriptionBox is defined to match the definition of the Spatial Relationship Descriptor `SRD' as defined in Dynamic Adaptive Streaming over HTTP (DASH) protocol (ISO/IEC <NUM>-<NUM><NUM>rd edition) to express spatial relationships between video tracks.

The proposed solution provides direct mapping between spatial relationship descriptions in ISOBMFF and in DASH SRD level as provided in the Table below:.

A TrackGroupTypeBox with 'srd ` track _grouping _type indicates that the track belongs to a group of tracks corresponding to spatial parts of a video. The tracks that have the same value of source_id within a TrackGroupTypeBox of track_group_type 'srd ' are mapped as being originated from the same source. For example, a very high resolution video may have been split into sub-pictures tracks for complexity reason. Each sub-picture track then conveys its position and sizes in the source video. The 'srd ' track group may convey the size of the source video.

The SpatialRelationshipDescriptionBox is defined as follows:
aligned(<NUM>) class SpatialRelationshipDescriptionBox
extends TrackGroupTypeBox('srd ', version, flags)
{
II the track group id is inherited from TrackGroupTypeBox;
II data specified for particular 'srd ' track_group_type
unsigned int(<NUM>) total_width;
unsigned int(<NUM>) total_height;
unsigned int(<NUM>) source_id;
unsigned int(<NUM>) spatial_set_id;
}
Where.

As in the previous alternative, a 'srd ` grouping_type for sample grouping is defined for the sub-picture tracks to declare their positions and sizes in a spatial relationship. A version <NUM> of the SampleToGroupBox shall be used when grouping_type is 'srd ` with the grouping_type _parameter value identifying the corresponding 'srd ` track_group_id.

<FIG> provides an example of use of the 'srd ' track group according to the invention.

A media file describing a single video <NUM> is considered. This single video <NUM>, called source video, is split and encoded into two sets of sub-picture tracks <NUM> and <NUM>. The first set <NUM> is a division of the source video in <NUM> sub-picture tracks (Tracks with ID= <NUM> to <NUM>) while the second set <NUM> is a division of the same source video into <NUM> sub-picture tracks (Tracks with ID=<NUM> to <NUM>). Two spatial relationships track groups <NUM> and <NUM> describe these spatial relationships, one track group per split level. Each track group provides an indication on how tracks can be rendered into the source video <NUM> or <NUM>. Each track contains a 'trgr' box containing the information present in the "Track Group" boxes <NUM> or <NUM>. Each track declares, through sample grouping, in a SpatialRelationshipGroup Entry 'srd ' (inheriting from VisualSampleGroupEntry) its position and size (x, y, w, h) in the source video <NUM>. The 'srd ' track group describes the organization of spatially related tracks. It does not provide composition instructions. It can be seen from <FIG> that most of the track group parameters (in <NUM> or <NUM>) are duplicated in each track group (<NUM>/<NUM> of the parameters). However, each track group can unambiguously be referenced through its track_group_id. An alternative embodiment of the example shown in <FIG> is shown in <FIG>.

In this example, to be used with the track group description at moov level, the two initial track groups <NUM> and <NUM> can be described in the more compact form <NUM> (<NUM>/<NUM> compaction). Since the track group <NUM> describes the spatial decomposition of a single source video <NUM>, the track_group_id is set to the source_ID value. Since this single source video is split into distinct sets of sub picture tracks, two spatial_set_ids are defined in this single track group to identify these different sets of sub-picture tracks.

<FIG> illustrates another example of use for spatial relationship description in track groups.

A media file describes two alternative versions, in terms of quality, of a single video <NUM>. Each alternative is split into <NUM> sub-picture tracks <NUM> and <NUM>. A first set of sub-picture tracks (with IDs= <NUM> to <NUM>) is available in low quality <NUM>, encoded for example in HEVC with a Qp=<NUM>, tracks with ID=<NUM> to <NUM>. A second set of sub-picture tracks <NUM> is available in higher quality, for example Qp=<NUM> in tracks with ID=<NUM> to <NUM>. Two track groups are defined, one per quality level. This leads to similar track groups as in the previous example (<NUM>/<NUM> overhead) shown in <FIG>. The same optimization as in the example of <FIG> can be applied.

<FIG> illustrates another example of use for spatial relationship description in track groups to describe a same source at different resolution levels <NUM> and <NUM> of a single video.

Each alternative is split into <NUM> sub-picture tracks <NUM> and <NUM>. The <NUM> resolution level <NUM> contains the <NUM> tracks with ID=<NUM> to <NUM> while the second resolution level <NUM>, for example in HD, contains <NUM> sub-picture tracks with ID=<NUM> to <NUM><NUM>. For each resolution level, a track group is declared (<NUM> and <NUM>) with spatial relationships description parameters.

In this example, only <NUM>/<NUM> of the parameters is duplicated. Each track group can be uniquely identified using the track_group_id. The track group description can be <NUM>% more compact by defining a single track group <NUM> in the Groups List box with two subsets as shown in <FIG>.

Since the track group describes spatial decomposition of a same video source in terms of content, the track_group_id can be set to the source_ID value.

This leads to a single track group definition in the Groups List box. Since this single source video is split into distinct sets of sub picture tracks <NUM> and <NUM>, two spatial_set_ids are defined in this single track group <NUM> to identify the set of sub-picture tracks. The difference with previous examples is that each spatial set defines its own total_width and total_height in addition to its unique identifier.

<FIG> illustrates a media file <NUM> describing a single video source encoded at different quality levels.

First, the video is split into 3x3 sub-pictures. The central sub-picture is encoded in high quality and surrounding sub-pictures are encoded each in two quality levels.

This allows adaptation in quality while keeping the central part in high quality (for example a region of interest). This leads to two spatial sets of sub-picture tracks with one sub-picture track, the one for the region of interest with ID=<NUM>, being part of these two spatial sets. Tracks with ID = <NUM> to <NUM> are encoded in high quality. Tracks with ID=<NUM> to <NUM> are encoded in low quality.

Again, two track groups can be defined, each providing a track group identifier, a source identifier and a spatial set identifier plus the total width and total_height of the source video. The track with ID=<NUM>, is the one shared in the two track groups. It is recalled that sample grouping 'srd ' is used to provide the positions and sizes for each track. The mutualized track (ID=<NUM>) contains the two 'srd ` track group boxes 'trgr', the other tracks contain on track group box: tracks with ID = <NUM> to <NUM> and <NUM> to <NUM> contain the 'srd ' track group with ID=<NUM>. The track groups with ID=<NUM> to <NUM> and <NUM> to <NUM> contain the 'srd ' track group with ID=<NUM>. This description <NUM> and <NUM> can be optimized into a single track group <NUM> in the GroupList Box as shown in <FIG>.

<FIG> illustrates another example with a media file <NUM> describing a single video. This video is split in two sets of sub-pictures.

Two track groups <NUM> and <NUM> are defined for the split level <NUM> decomposition level. While the track group <NUM> provides the spatial relationships for the sub-picture tracks in the source video, the track group <NUM> describes spatial relationships for a partial reconstruction <NUM> (tracks with ID <NUM>, <NUM>, <NUM> and <NUM> for example) of this source video <NUM> from the same set of sub-picture tracks (track group with ID=<NUM>). This track group introduces a new origin, as such a new source_id value. The spatial_set_id indicates that it provides the same set of sub-picture tracks as in group <NUM>. Only a subset of sub-picture tracks is used here. The tracks with ID <NUM>, <NUM>, <NUM> and <NUM> are part of both groups: they have to define different position and sizes in the different source id, i.e. referential. For this purpose, each combination of position and size parameters has to be associated with a track_group_id (in <NUM>). In the above example it could be either the track_groupid or the source id. This is done with the SampleToGroupBox::grouping_type_parameter. As many SampleToGroupBox of grouping_type 'srd 'as track groups are defined, as well there are as many entries of grouping_type 'srd ' in the SampleGroupDescriptionBox as track groups. Each SampleToGroup of type 'srd ' maps to an entry in the corresponding `sgpd' and in a given SampleToGroupBox, the grouping_type _parameter provides the mapping between the (x,y,w,h) and the track group. For example, the track with ID=<NUM> has x,y,w,h parameters in the group with track_group_id=<NUM> while it has x_prime, y_prime, w_prime, h_prime in the group with track_group_id=lll.

<FIG> illustrates an example of 'srd ' track group where spatial relationship description parameters are expressed in normalized coordinates, instead of luma sample units.

The same Track Groups as shown in <FIG> are presented <NUM> and <NUM> in the file together with a third Track Group <NUM> with track_group_id=<NUM> that allows to produce a partial reconstruction fully covering a new source with source_id=<NUM>. The tracks location of the partial reconstruction is here expressed in normalized coordinates: 'w' and `h' expressed in the 'srd ' box are not the same as the ones expressed in pixels in object_width and object_height. In such a case, the normalized ratio is the (rounded) ratio between the value of 'w' and the track_width and between h and the track_height present in the track header. These ratios shall be equal. The total_width or total_height values in luma samples for the group are respectively deduced by multiplying the ratio with the normalize total_width and total_height declared in the track group <NUM>. In the example shown in <FIG>, the partial reconstruction <NUM> also combines tracks with different spatial shapes (from two different splitting levels), but with the same spatial resolution (in terms of spatial sampling).

<FIG> illustrates the track group <NUM> with track_group_id=<NUM> is also expressed with normalized coordinates (as shown in <FIG>). In this example, the third track group <NUM> uses the same source as the two other track groups.

It indicates that the rendering shall be performed in the same referential as the source (same origin, same dimensions). However, a partial_reconstruction flag <NUM> equal to <NUM> is used to signal that the track group contains only a sub-part of the source: the source will be only partially rendered. This additional parameter is an indication for media players. It can be a text description of the shape, for example: "rectangle", "square", "triangle", etc..

In the examples shown in <FIG>, the x corresponds to the obj ect_x, y corresponds to object_y and w and h respectively to object_width and object_height from the SpatialRelationshipGroupEntry.

In an alternative embodiment, two types of spatial relationships descriptions are distinguished: descriptive and prescriptive. Each type of spatial relationships description has a reserved unique four character code that can be used in track_grouping_type. Having different track grouping_types allows describing spatial relationships from the original source (descriptive) or a spatial composition (prescriptive). Making this distinction allows a lighter spatial relationship description for the composition where the source id parameter may not be needed anymore since the composition may result in a different video from the original source video.

In another embodiment, the track group for spatial relationship descriptions does not contain a spatial_set_id. In this embodiment, the spatial_set_id value is the same as the track_group_id value. When preparing the content for tile-based or region-of-interest streaming (for example DASH), these spatial relationships can be exposed in a streaming manifest (for example a DASH Media Presentation Description with SRD descriptor) or in a Session Description Protocol file, the spatial_set_id is set to the value of the track_group_id. As well, when the track properties are dynamic and described with sample group with grouping_type='srd ', the grouping_type parameter uses this track_group_id.

In another embodiment, the SpatialRelationGroupEntry may not contain sizes of the sub-picture tracks (w, h) but only its positions (x and y). In this case, the track width and height in the track header shall be considered. In this case, the coordinates shall be considered expressed in luma sample units, not in arbitrary or normalized units.

The SpatialRelationshipDescriptionGroupEntry and the 'srd ' track can also be described in 'trgr' box or alternatively in the Groups List box ('grpl')). When the 'srd ' track group is static, both 'srd ' track group and SpatialRelationshipGroupEntry 'srd ' may be merged respectively as track property in one entry of the GroupList Box. When the 'srd ' track group is dynamic, only the parameters from the track group are moved to one entry in the GroupList Box while the SpatialDescriptionGroupEntry remains in the sub-picture track.

According to a third embodiment, the Groups List box ('grpl')) can be defined in a fragment-level Meta Box ('meta') in addition to the file-level Meta Box ('meta'), i.e. an EntityToGroupBox can be redefined across ISOBMFF fragments along time.

An EntityToGroupBox defined in a fragment (under a `moof' box) supersedes any EntityToGroupBox with same grouping_type and same group_id previously defined. It also invalidates all previous track group definitions with same grouping _type and track_group_id equal to group _id in all tracks.

By defining an EntityToGroupBox in a fragment, it is possible to proceed to a complete reset of the definition of a group of tracks (including common properties) with a given grouping_type and group _id with the consequence that all properties associated with a track belonging to this updated group of tracks must be redefined at fragment level for each track that composes this group of tracks.

To allow this, it is proposed to extend track group by authorizing track group definition at fragment level as follows:
<IMG>.

Where this box contains the set of TrackGroupTypeBox describing the set of groups of tracks the track fragment is belonging to.

The TrackGroupTypeBox is defined as in previous embodiments.

This is useful for updating parameters associated with a track within a track group along time at fragment granularity. That provides an intermediate granularity for parameters between the static definition at track level (in 'track' box) and the full dynamicity at sample level using sample grouping mechanism as described previously.

When this TrackGroupBox ('trgr') is defined in a TrackFragmentBox ('traf'), its content (i.e. the set of TrackGroupTypeBox) supersedes all indications of group of tracks associated with this track in a previous fragment or in the corresponding TrackBox ('trak') (i.e. with same track_ID). If the container TrackGroupBox ('trgr') is empty in a TrackFragmentBox ('traf'), then the fragment does not belong to any group. If the TrackGroupBox ('trgr') is not defined in a TrackFragmentBox ('traf'), track group definitions associated with this track in the 'moov' box (i.e. 'trgr' box defined in associated 'trak' box), if any, apply.

The TrackGroupBox ('trgr') defined in a TrackFragmentBox ('traf') enables indication of groups of tracks valid for a given track fragment. The scope of the Track Group Box ('trgr') in a track fragment is limited to the encompassing track fragment to cope with trick modes and seek management where some fragments may be lost or not requested by the client. In such a case, the client must not rely on information that is only defined in a previous fragment.

This enables a track to be added or removed from a track group along time.

As an alternative, the TrackGroupBox ('trgr') as defined above can also be combined with the ComponentToGroupBox to vary along time the set of groups which the track belongs to and possibly the parameters associated with the track within a given group. Similarly, when a TrackGroupBoxbox ('trgr') is defined in a TrackFragmentBox ('traf'), its content (i.e. the set of TrackGroupTypeBox) supersedes all information defined by any ComponentInComponentToGroupBox having a component_id equal to the track fragment's track_ID.

According to a fourth embodiment, one or more additional flags can be defined in any one of the previously defined boxes EntityToGroupBox, TrackGroupTypeBox, ComponentToGroupBox , and ComponentInComponentToGroupBox to signal whether the number of members of a group and/or the properties shared by the group and/or the properties associated with an entity or a component within a group are static or dynamic along time. This one or more flags allow the client parser to be signaled in the case where it should be prepared to parse further sub-level track group boxes in track fragments or sample grouping boxes (for instance CompactSampleToGroupBox or SampleToGroup, and SampleGroupDescriptionBox) to determine all existing groups and associated properties along time or if all group-related information is statically defined at the beginning of the presentation.

For instance, a <NUM>-bit flag (e.g. static_grouping_flag) in EntityToGroupBox or ComponentToGroupBox signals when set to <NUM> that the group of components or entities is static for the whole multimedia presentation duration and when set to <NUM> that the group of components or entities may change at track fragment level (e.g. using track grouping mechanism or sample grouping at fragment level).

In another example, a <NUM>-bit flag (e.g. static_properties_flag) in TrackGroupTypeBox or ComponentInComponentToGroupBox signals when set to <NUM> that the properties defined in the embedding box (respectively TrackGroupTypeBox or ComponentInComponentToGroupBox) remain the same for the whole duration of the presentation (if defined in a file-level boxes, e.g. 'moov', 'meta' boxes) or for the duration of the fragment (if defined in a fragment-level box, e.g. 'moof', 'meta' or `traf' boxes), and when set to <NUM> that the properties may change using either track grouping or sample grouping mechanisms at fragment level.

In an alternative of the previous examples, a <NUM>-bit flag (e.g. dynamic_properties_flag) may be used to precise whether the dynamicity is implemented using either track grouping or sample grouping mechanisms at fragment level, or possibly both:.

The use of above flags simultaneously may allow defining a hierachy of dynamicity as follows:.

In an alternative embodiment, it is possible to use multiple instances of above <NUM>-bit or <NUM>-bit flags to signal the dynamicity of following information separately:.

In an alternative, optionally, another parameter (e.g. static_group_id) can be defined in any one of the previously defined boxes EntityToGroupBox, TrackGroupTypeBox, ComponentToGroupBox, and Component InComponentToGroupBox to signal whether the content of the box has really changed compared to a previous instance of the same box with the same grouping_type or track_group_type and group_id or track_group_id. For each of above boxes, this static_group_id is increased each time a new instance of above boxes is defined with same group_id or track_group_id and grouping_type or track_grouping_type pair respectively, and the box defines different properties or list of components compared to a previous instance. This parameter static_group_id represents a kind of version number. The box can reuse a previous static_group_id parameter value when the box content (i.e. set of grouped components and/or properties) is exactly the same as previously defined. This parameter allows the client parser to know if something has changed before parsing the whole box.

<FIG> is a schematic block diagram of a computing device <NUM> for implementation of one or more embodiments of the invention. The computing device <NUM> may be a device such as a micro-computer, a workstation or a light portable device. The computing device <NUM> comprises a communication bus connected to:.

The executable code may be stored either in read only memory <NUM>, on the hard disk <NUM> or on a removable digital medium such as for example a disk. According to a variant, the executable code of the programs can be received by means of a communication network, via the network interface <NUM>, in order to be stored in one of the storage means of the communication device <NUM>, such as the hard disk <NUM>, before being executed.

The central processing unit <NUM> is adapted to control and direct the execution of the instructions or portions of software code of the program or programs according to embodiments of the invention, which instructions are stored in one of the aforementioned storage means. After powering on, the CPU <NUM> is capable of executing instructions from main RAM memory <NUM> relating to a software application after those instructions have been loaded from the program ROM <NUM> or the hard-disc (HD) <NUM> for example. Such a software application, when executed by the CPU <NUM>, causes the steps of the flowcharts shown in the previous figures to be performed.

In this embodiment, the apparatus is a programmable apparatus which uses software to implement the invention. However, alternatively, the present invention may be implemented in hardware (for example, in the form of an Application Specific Integrated Circuit or ASIC).

For example, the present invention may be embedded in a device like a camera, a smartphone, a head-mounted display or a tablet that acts as a remote controller for a TV or for multimedia display, for example to zoom in onto a particular region of interest. It can also be used from the same devices to have personalized browsing experience of a multimedia presentation by selecting specific areas of interest. Another usage from these devices and methods by a user is to share with other connected devices some selected sub-parts of his preferred videos. It can also be used with a smartphone or tablet to monitor what happens in a specific area of a building put under surveillance provided that the surveillance camera supports the method for providing data according to the invention.

Claim 1:
A method for generating one or more media files (<NUM>), according to ISO Base Media File Format, based on timed media data, the timed media data comprising samples (<NUM>, <NUM>, <NUM>, <NUM>), the method comprising:
obtaining the timed media data comprising the samples (<NUM>, <NUM>, <NUM>, <NUM>);
creating tracks (<NUM>) each comprising at least a part of at least one (<NUM>) of the obtained samples (<NUM>, <NUM>, <NUM>, <NUM>);
generating descriptive metadata associated with the created tracks; and
generating the one or more media files (<NUM>) based on the tracks and the descriptive metadata,
wherein the descriptive metadata comprises at least one TrackGroupTypeBox (<NUM>) including at least one track group identifier and describing at least one group of the created tracks by using the at least one track group identifier,
wherein the descriptive metadata further comprises a plurality of sets of properties for the created tracks and a SampleToGroupBox (<NUM>) characterised by including a grouping type parameter, and the descriptive metadata further comprises a set of properties (<NUM>) which is among the plurality of sets of properties and is associated with one or more samples of one of the created tracks,
wherein a value of the grouping type parameter is equal to the at least one track group identifier, and
wherein the associated set of properties is applied to the one or more samples within a track group identified by the grouping type parameter.