Patent Description:
The Internet is being evolved from a human-centered connection network over which people generate and consume information to the Internet of things (IoT) in which distributed elements such as things exchange and process information. The Internet of everything (IoE) is an example of convergence between big data processing and the IoT via a connection to a cloud server or the like.

To implement the IoT, technology elements such as sensing technology, wired and wireless communication, network infrastructure, service interfacing, and security are required. Recently, techniques such as sensor networks, machine-to-machine (M2M) communication, and machine type communication (MTC) are under study for connectivity between things.

In the IoT environment, an intelligent Internet technology (IT) service of creating new values to human living by collecting and analyzing data generated from connected things may be provided. The IoT may find its applications in the fields of smart homes, smart buildings, smart cities, smart cars or connected cars, smart grids, health care, smart appliances, and up-to-date medical services, through convergence between existing IT technologies and various industries. For IoT implementation, content has also been evolving. Along with the on-going evolution of black and white content to color content, high definition (HD) content, ultra high definition (UHD) content, and recent high dynamic range (HDR) content, driven by content distribution and standardization, virtual reality (VR) content reproducible on VR devices such as Oculus and Samsung's Gear VR is under development. A VR system refers to a system making interactions possible between a user and a content display device or a processing unit by monitoring the user, allowing the user to provide a feedback input to the content display device or processing unit by some kind of controller, and enabling the device or unit to process the input and adjust content accordingly.

A VR device may reproduce a three-dimensional (3D) image such as a spherical or cylindrical shape. The 3D image may be referred to as an omnidirectional image. The VR device is capable of displaying a specific area of the 3D image in consideration of a user's gaze direction or the like.

<CIT> discloses an image processing device and an image processing method with which it is possible to generate a high-quality texture image of a prescribed visual point using an omnidirectional image. A plurality of real cameras photograph a plurality of reference images at a plurality of viewpoint positions and in a plurality of sight directions. A virtual viewpoint image is generated by weighted averaging, with large weight added to a reference image close to a virtual visual point in a sight direction, the virtual visual point comprising a virtual viewpoint position and sight direction that are specified in accordance with the motion of a head-mounted display, and the virtual viewpoint image is displayed on the head-mounted display. The present disclosure can be applied, for example, to a home server or the like that generates a display image of a prescribed visual point from an omnidirectional image.

<CIT> discloses a display control apparatus. The display control apparatus includes: a creating unit that creates, from an image, a partial image of the image, and including in a center thereof an arbitrary viewpoint on the image; a display controller that displays, on a display, a display image including the image and a viewpoint listing image of a list of thumbnails of plural of the partial images; and an accepting unit that accepts input indicating a change of the image in the display image, according to operation on the image or the viewpoint listing image. The display controller changes the image in the display image according to the input, and changes a sequential order of the thumbnails in the viewpoint listing image such that a thumbnail of the partial image corresponding to a region of the image to be newly displayed according to the input is displayed at a predetermined position in the viewpoint listing image.

<CIT> proposes a method for transmitting <NUM> video. The method for transmitting <NUM> video according to the present invention may comprise the steps of: receiving <NUM> video data captured by at least one camera; projecting a 2D image obtained by processing the <NUM> video data; generating signaling information associated with the <NUM> video data; encoding the 2D image; and processing the encoded 2D image and the signaling information for transmission thereof, and transmitting the same through a broadcasting network.

<CIT> discloses an image capturing and processing system creates 3D image-based rendering (IBR) for real estate. The system provides image-based rendering of real property, the computer system including a user interface for visually presenting an image-based rendering of a real property to a user; and a processor to obtain two or more photorealistic viewpoints from ground truth image data capture locations; combine and process two or more instances of ground truth image data to create a plurality of synthesized viewpoints; and visually present a viewpoint in a virtual model of the real property on the user interface, the virtual model including photorealistic viewpoints and synthesized viewpoints.

3D content may include a plurality of viewpoints to provide a user with experience at various positions. Each of the plurality of viewpoints may correspond to a 3D image from the viewpoint. The VR device may display a 3D image viewed from a selected one of the plurality of viewpoints.

When switching occurs between the plurality of viewpoints, how to set a switched viewport may be an issue, and computations for viewport switching may be a load on the processor of the VR device.

The present disclosure is intended to provide a format of metadata for three-dimensional (3D) content to support easy switching between a plurality of viewpoints.

According to embodiments of the present disclosure, at least the following effects are achieved.

According to the present disclosure, a plurality of viewpoints included in three-dimensional (3D) content may be grouped and managed accordingly by means of provided metadata.

Further, according to the present disclosure, switching between a plurality of viewpoints included in 3D content may be supported by means of provided metadata.

That is, the present disclosure may provide a method of detecting defects in a remote radio head (RRH) which does not rely on thresholds and expert knowledge.

Advantages and features of the present disclosure, and a method of achieving them will be apparent with reference to embodiments described below in detail together with the accompanying drawings. However, the present disclosure is not limited to the embodiments disclosed below, but may be implemented in various different forms. The embodiments are provided only to make the present disclosure comprehensive, and indicate the full scope of the disclosure to those skilled in the art. The present disclosure is only defined by the scope of the appended claims.

<FIG> is a block diagram illustrating a system according to an embodiment of the present disclosure. The system according to the embodiment of the present disclosure may include a content provider <NUM>, a server <NUM>, and an electronic device <NUM>.

The content provider <NUM> may generate three-dimensional (3D) content. The 3D content includes a plurality of viewpoints. The content provider <NUM> generates the 3D content with a plurality of 3D images captured by a plurality of cameras <NUM>. The locations of the plurality of cameras <NUM> corresponds to the plurality of viewpoints, respectively. In some embodiments, the locations of the plurality of viewpoints may be set arbitrarily. In some embodiments, the plurality of 3D images included in the 3D content may be generated virtually, not based on images captured by the plurality of cameras <NUM>. The content provider <NUM> may transmit data of the 3D content to the server <NUM>. In some embodiments, the content provider <NUM> may be configured as a server independent of the server <NUM>. In some embodiments, the content provider <NUM> may be incorporated into the server <NUM>.

The server <NUM> may transmit, to the electronic device <NUM>, media data for the 3D content and metadata for the 3D content based on the data of the 3D content received from the content provider <NUM>. The media data and the metadata may be transmitted independently. According to some embodiments, the media data and the metadata may be transmitted from individual servers.

The media data may include data of the plurality of 3D images included in the 3D content. The server <NUM> may generate two-dimensional (2D) images by projecting each of the 3D images (e.g., by equi-rectangular projection (ERP)). The server <NUM> may transmit, to the electronic device <NUM>, data of the generated 2D images as the media data for the 3D content in conformance to a legacy protocol for transmitting data of a 2D image, such as MPEG. According to some embodiments, the media data may be generated by the content provider <NUM>, and the server <NUM> may forward the media data generated by the content provider <NUM> to the electronic device <NUM>.

The metadata may include information required to process the media data of the 3D content and reproduce the 3D content. According to embodiments of the present disclosure, the metadata may include information for grouping the plurality of viewpoints included in the 3D content, information for aligning the plurality of viewpoints, or information for supporting switching between the plurality of viewpoints. According to some embodiments, the content provider <NUM> may generate the metadata and provide the metadata to the server <NUM>, and the server <NUM> may forward the metadata to the electronic device <NUM>.

The electronic device <NUM> may reproduce the 3D content based on the received media data and metadata. According to some embodiments, the electronic device <NUM> may provide a signal for 3D content reproduction to another device capable of reproducing the 3D content, without directly reproducing the 3D content. The electronic device <NUM> may be a VR device or any device equipped with a display, which is capable of reproducing 3D content. According to some embodiments, the electronic device <NUM> may be a device capable of providing a signal for 3D content reproduction wiredly or wirelessly to a VR device or a device equipped with a display, which is capable of reproducing 3D content. For example, the electronic device <NUM> may be a smartphone, a television, a personal computer (PC), a laptop computer, a set-top box, or the like. The electronic device <NUM> may restore the 3D images from the data of the 2D images included in the media data to reproduce the 3D content or generate a signal for 3D content reproduction.

<FIG> is a flowchart illustrating an operation of a server according to an embodiment of the present disclosure. The server described with reference to <FIG> may be substantially identical to the server <NUM> illustrated in <FIG>.

The server identifies metadata for 3D content including a plurality of viewpoints in operation <NUM>. The identified metadata may be received from the content provider <NUM> or generated by the server.

The server transmits the identified metadata in operation <NUM>. The server may transmit the metadata to a device capable of processing data of 3D content (e.g., the electronic device <NUM>).

According to the present disclosure, the metadata may include information about at least one grouped viewpoint among the plurality of viewpoints included in the 3D content. In an embodiment, the information about the at least one grouped viewpoint may be provided in the form of a viewpoint alignment box, ViewpointAlignmentBox. The viewpoint alignment box ViewpointAlignmentBox may be included in a projection omnidirectional video box, ProjectedOmniVideoBox. The viewpoint alignment box, ViewpointAlignmentBox provides information about alignment of various viewpoints associated with content which has been configured collectively with multiple viewpoints (i.e. <NUM> videos) included in multiple tracks of a file (i.e., media data). The viewpoints are grouped into at least one viewpoint alignment group, each including adjacent viewpoints in the same content scene (e.g., the same physical space (a room, a playground, or a concert hall)). The viewpoint grouping may allow a more flexible viewpoint switching mechanism during switching between viewpoints, aside from an initial viewing orientation mechanism. A viewpoint may be represented by content included in a single track (in the case of single-track delivery) or may include tile-based tracks and may be represented by content represented by all tile tracks (in the case of multi-track delivery). For the syntax of the viewpoint alignment box, ViewpointAlignmentBox, it is assumed that there is a method of identifying various viewpoints in files (e.g., tracks having the same track group identifier (ID) identifying a track group type representing content from the same viewpoint). An exemplary syntax of the viewpoint alignment box, ViewpointAlignmentBox is given as follows.

aligned(<NUM>) class ViewpointAlignmentBox extends FullBox('vwpa', <NUM>, <NUM>) {
ViewpointAlignmentStruct()
}
aligned(<NUM>) class ViewpointAlignmentStruct() {
unsigned int(<NUM>) num_alignment_groups;
for (i = <NUM>; i < num_alignment_groups; i++) {
bit(<NUM>) reserved = <NUM>;
unsigned int(<NUM>) default_viewpoint_id[i];
unsigned int(<NUM>) num_aligned_viewpoints[i];
for (j = <NUM>; j < num_aligned_viewpoints; j++) {
unsigned int(<NUM>) viewpoint_id[j];
ViewpointAlignmentOffset() ;
}
}
}
aligned(<NUM>) class ViewpointAlignmentOffset() {
RotationStruct();
aligned(<NUM>) class RotationStruct() {
signed int(<NUM>) rotation_yaw;
signed int(<NUM>) rotation_pitch;
signed int(<NUM>) rotation_roll;
}.

The semantics of each field in the above syntax is described below.

► The global coordinates of each view point may vary according to camera configurations and settings during capturing content. It is assumed that the global coordinates are not pre-aligned with an arbitrary absolute reference of a real world, such as the global positioning system (GPS) global coordinates. However, the global coordinates are identifiable with respect to the absolute reference of the real world.

► The definition of the global coordinates is based on omnidirectional media format (OMAF) v1.

Now, a detailed description will be given of a method of grouping a plurality of viewpoints included in 3D content, with reference to <FIG> illustrates an exemplary method of grouping a plurality of viewpoints. 3D content may include a plurality of viewpoints V1, V2. Among the plurality of viewpoints, adjacent viewpoints may be grouped into the same group. For example, adjacent viewpoints V1, V2, V3, and V4 may be grouped into a first group G1, and other adjacent viewpoints V5, V6, and V7 may be grouped into a second group G2. According to some embodiments, at least a part of the plurality of viewpoints included in the 3D content may not be grouped. For example, the viewpoint V8 is excluded from grouping and thus does not belong to any group.

With reference to <FIG>, a method of aligning a plurality of viewpoints included in a group will be described below. <FIG> is a diagram illustrating an exemplary method of aligning a plurality of viewpoints. In <FIG>, V1 and V2 are reference numerals used for the convenience of description, not necessarily identical to V1 and V2 described before with reference to the foregoing drawing. The same thing applies to the other drawings. Aligning viewpoints with each other may mean aligning the coordinate axes of the viewpoints with each other. Aligning the coordinate axes of viewpoints may mean adjusting the coordinate axes of each of the viewpoints to be parallel to corresponding coordinate axes of the other viewpoints. Viewpoint alignment may take place within a single group. Aligning the coordinate axes of viewpoints is interpreted as sharing a common reference coordinate system between the viewpoints. In <FIG>, V1 and V2 denote viewpoints included in the same viewpoint group. The viewpoint V1 is a default viewpoint in the group. The coordinate axes of the remaining viewpoints except for the viewpoint V1 in the viewpoint group are aligned with the coordinate axes of the default viewpoint V1. For example, the X axis X<NUM> of the viewpoint V2 may be aligned to be parallel to the X axis X<NUM> of the viewpoint V1. An offset indicating a rotation value for converting the X axis, X<NUM> to the X axis X<NUM> may be included in metadata and provided from the server <NUM> to the electronic device <NUM>. While the description of <FIG> focuses only on X-axis alignment, Y and Z axes may also be aligned in the same manner. The axes of a coordinate system using the yaw, pitch, and roll axes may also be aligned in the same manner as in the coordinate system using the X, Y and Z axes. Offsets for aligning the yaw, pitch, and roll axes have been described before as rotation_yaw, rotation_pitch, and rotation roll in the viewpoint alignment box, ViewpointAlignmentBox.

As described above, the server <NUM> may transmit, to the electronic device <NUM>, metadata including information about grouped viewpoints, such as the viewpoint alignment box ViewpointAlignmentBox. The electronic device <NUM> may identify the grouped viewpoints based on the received metadata, and align the coordinates of the viewpoints of a viewpoint group with respect to the coordinate axes of a default viewpoint in the viewpoint group.

The metadata may further include information about viewpoint switching. In an embodiment, the information about viewpoint switching may be provided in the form of a viewpoint switching mode box ViewpointSwitchingModeBox. The viewpoint switching mode box, ViewpointSwitchingModeBox may be included in the projected omnidirectional video box ProjectedOmniVideoBox. The viewpoint switching mode box, ViewpointSwitchingModeBox provides information about switching modes for various viewpoints associated with content collectively configured with multiple viewpoints (i.e., <NUM> videos) included in multiple tracks of a file (i.e., media data). A viewpoint may be represented by content included in a single track (in the case of single-track delivery) or may include tile-based tracks and may be represented by content represented by all tile tracks (in the case of multi-track delivery). For the syntax in the viewpoint switching mode box ViewpointSwitchingModeBox, it is assumed that there is a method of identifying various viewpoints in files (e.g., tracks having the same track group ID identifying a track group type representing content from the same viewpoint). An exemplary syntax of the viewpoint switching mode box ViewpointSwitchingModeBox is given as follows.

aligned(<NUM>) class ViewpointSwitchingModeBox extends FullBox('vwps', <NUM>, <NUM>) {
ViewpointModeStruct()
}
aligned(<NUM>) class ViewpointModeStruct() {
unsigned int(<NUM>) num_viewpoints;
for (i = <NUM>; i < num_viewpoints; i++) {
unsigned int(<NUM>) viewpoint_id[i] ;
unsigned int(<NUM>) los_flag[i] ;
if (los_flag == <NUM>) {
bit(<NUM>) reserved = <NUM>;
unsigned int(<NUM>) los_mode;
}
}
}.

Each field of the above syntax has the following semantics.

In some embodiments, information about viewpoint switching such as the viewpoint switching mode box, ViewpointSwitchingModeBox may be used to provide information about switching between viewpoints in a single viewpoint group. Because the coordinate axes of the viewpoints in the single viewpoint group may be aligned, LoS-mode viewpoint switching may be performed easily in the single viewpoint group.

The LoS mode may refer to a viewpoint switching mode in which the direction of a viewport from a pre-switching viewpoint is set to be identical to the direction of the viewport from a post-switching viewpoint. In some embodiments, when a straight line connecting between viewpoints for switching is parallel to the direction of a viewport from a pre-switching viewpoint, the LoS mode may be set for the viewpoint to be switched. The LoS mode may include both the forward viewport switching mode and the reverse viewport switching mode.

<FIG> illustrates an example of the forward viewport switching mode of the LoS mode. In the example of <FIG>, a viewpoint V1 may be switched to a viewpoint V2. In the forward viewport switching mode, the direction of a viewport VP1 from the viewpoint V1 may be identical to that of a viewport VP2 from the viewpoint V2. That is, the difference between the direction of the viewport VP1 and the direction of the viewport VP2 may be <NUM>°.

<FIG> illustrates an example of the reverse viewport switching mode of the LoS mode. In the example of <FIG>, a viewpoint V1 may be switched to a viewpoint V2. In the forward viewport switching mode, the direction of a viewport VP1 from the viewpoint V1 may be opposite to that of a viewport VP2 from the viewpoint V2. That is, the difference between the direction of the viewport VP1 and the direction of the viewport VP2 may be <NUM>°.

<FIG> illustrates LoS in a coordinate system. The coordinates of the viewport VP1 from the viewpoint V1 may be expressed as (ϕ<NUM>, θ<NUM>) where ϕ<NUM> represents the azimuth of the viewport VP1 and θ<NUM> represents the elevation of the viewport VP1. When viewport switching occurs from the viewpoint V1 to the viewpoint V2 in the LoS mode, the viewport VP2 of the viewpoint V2 in the forward viewport switching mode may be represented as (ϕ<NUM>, θ<NUM>), whereas a viewport VP2' of the viewpoint V2 in the reverse viewport switching mode may be represented as (ϕ<NUM> - <NUM>, - θ<NUM>).

The electronic device <NUM> may identify for each viewpoint whether the viewpoint switching mode is the LoS mode based on the viewpoint switching mode box, ViewpointSwitchingModeBox included in the received metadata. When the viewpoint switching mode is the LoS mode, the electronic device <NUM> may identify whether the forward viewport switching mode or the reverse viewport switching mode is applied to the viewpoint. Therefore, the electronic device <NUM> may perform LoS switching to a specific viewpoint according to the received metadata.

One of the viewpoint switching modes, the non-LoS mode may include a central estimation-based viewpoint switching mode. The central estimation scheme refers to a process of setting a post-switching viewport to be directed toward a target point in the direction of a pre-switching viewport. The target point may be set such that the distance from the pre-switching viewpoint to the target point is equal to the distance from the post-switching viewpoint to the target point. With reference to <FIG> and <FIG>, the central estimation scheme will be described below in greater detail. <FIG> and <FIG> illustrate exemplary central estimation-based viewpoint switching.

<FIG> illustrates an exemplary central estimation scheme for the case in which the Z-axis coordinate Zv1 of a viewpoint V1 is identical to the Z-axis coordinate Zv2 of a viewpoint V2. A target point O may be located on a viewport VP1 from the viewpoint V1. The distance d1 from the viewpoint V1 to the target point O may be equal to the distance from the viewpoint V2 to the target point O. When the viewpoint V1 is switched to the viewpoint V2, the viewport VP2 may be set to be directed toward the target point O. In this case, the elevation θ<NUM> of the viewport VP1 from the viewpoint V1 may be equal to the elevation θ<NUM> of the viewport VP2 from the viewpoint V2. The azimuth of the viewport VP1 from the viewpoint V1 may be different from the azimuth of the viewport VP2 from the viewpoint V2. The azimuth of the viewport VP2 from the viewpoint V2 may be calculated from the coordinates of the viewpoint V1, the coordinates of the viewpoint V2, and the azimuth of the viewport VP1 from the viewpoint V1.

<FIG> illustrates an exemplary central estimation scheme for the case in which the Z-axis coordinate Zv1 of the viewpoint V1 is smaller than the Z-axis coordinate Zv2 of the viewpoint V2. The target point O and the viewport VP2 from the switched viewpoint V2 may be set in the same manner as described with reference to <FIG>. When viewport switching occurs from the viewpoint V1 to the viewpoint V2, the elevation θ<NUM> of the viewport VP1 from the viewpoint V1 may be larger than the elevation θ<NUM> of the viewport VP2 from the viewpoint V2. The azimuth of the viewport VP2 from the viewpoint V2 may be calculated from the coordinates of the viewpoint V1, the coordinates of the viewpoint V2, and the azimuth of the viewport VP1 from the viewpoint V1.

<FIG> illustrates an exemplary central estimation scheme for the case I which the Z-axis coordinate Zv1 of the viewpoint V1 is larger than the Z-axis coordinate Zv2 of the viewpoint V2. The target point O and the viewport VP2 from the switched viewpoint V2 may be set in the same manner as described with reference to <FIG>. When viewport switching occurs from the viewpoint V1 to the viewpoint V2, the elevation θ<NUM> of the viewport VP1 from the viewpoint V1 may be smaller than the elevation θ<NUM> of the viewport VP2 from the viewpoint V2. The azimuth of the viewport VP2 from the viewpoint V2 may be calculated from the coordinates of the viewpoint V1, the coordinates of the viewpoint V2, and the azimuth of the viewport VP1 from the viewpoint V1.

Among the viewpoint switching modes, the non-LoS mode may include a viewpoint switching mode based on depth tracking (content depth-enhanced non-LoS viewpoint switching). In the depth tracking scheme, a point spaced from a pre-switching viewpoint on a pre-switching viewport may be set as a target point, and a post-switching viewport may be set to be directed toward the target point. With reference to <FIG>, <FIG>, depth tracking-based viewpoint switching will be described below. <FIG>, <FIG> illustrate exemplary depth tracking-based viewpoint switching.

<FIG> illustrates an exemplary depth tracking scheme for the case in which the Z-axis coordinate Zv1 of the viewpoint V1 is identical to the Z-axis coordinate Zv2 of the viewpoint V2. <FIG> illustrates an exemplary depth tracking scheme for the case in which the Z-axis coordinate Zv1 of the viewpoint V1 is smaller than the Z-axis coordinate Zv2 of the viewpoint V2. <FIG> illustrates an exemplary depth tracking scheme for the case in which the Z-axis coordinate Zv1 of the viewpoint V1 is larger than the Z-axis coordinate Zv2 of the viewpoint V2. In the illustrated cases of <FIG>, <FIG>, the target point O may be determined according to a predetermined distance d1 from the viewpoint V1 on the viewport VP2 from the viewpoint V1. The distance d1 from the viewpoint V1 to the target point O may correspond to the depth of a 3D image from the viewpoint V1. When the viewpoint V1 is switched to the viewpoint V2, the viewport VP2 from the viewpoint V2 may be set to be directed toward the target point O. The elevation θ<NUM> of the viewport VP2 from the viewpoint V2 and the distance d2 from the target point V2 to the target point O may be calculated from the distance d1 from the viewpoint V1 to the target point O, the distance dv1v2 between the viewpoints V1 and V2, and the elevation θ<NUM> of the viewport VP1 from the viewpoint V1 by triangulation. The azimuth of the viewport VP1 from the viewpoint V1 and the azimuth of the viewport VP2 from the viewpoint V2 may be different. The azimuth of the viewport VP2 from the viewpoint V2 may be calculated from the coordinates of the viewpoint V1, the coordinates of the viewpoint V2, and the azimuth of the viewport VP1 from the viewpoint V1.

In the depth tracking-based LoS mode, metadata may include additional information indicating a content depth. An exemplary syntax and semantics of the additional information are given as follows.

In some embodiments, metadata may include a box containing both of information about grouped viewpoints and information for viewpoint alignment. In some embodiments, the viewpoint alignment box, ViewpointAlignmentBox may include both of the information about grouped viewpoints and the information for viewpoint alignment. An exemplary syntax of the viewpoint alignment box, ViewpointAlignmentBox is given as follows. aligned(<NUM>) class ViewpointAlignmentBox extends FullBox('vwpa', <NUM>, <NUM>) {
ViewpointAlignmentStruct()
}
aligned(<NUM>) class ViewpointAlignmentStruct() {
unsigned int(<NUM>) num_alignment_groups;
for (i = <NUM>; i < num_alignment_groups; i++) {
unsigned int(<NUM>) alignment_group_id[j];
unsigned int(<NUM>) default_viewpoint_id[i];
unsigned int(<NUM>) num_aligned_viewpoints[i];
unsigned int(<NUM>) alignment_group_content_coverage_flag;
unsigned int(<NUM>) alignment_group_initial_orientation_flag;
if (alignment_group_initial_orientation_flag == <NUM>) {
signed int(<NUM>) center_x;
signed int(<NUM>) center_y;
signed int(<NUM>) center_z;
}
for (j = <NUM>; j < num_aligned_viewpoints; j++) {
unsigned int(<NUM>) viewpoint_id[j];
ViewpointAlignmentOffset() ;
SphereRegionStruct(<NUM>) ;
unsigned int(<NUM>) Depth();
OR
signed int(<NUM>) center_x;
signed int(<NUM>) center_y;
signed int(<NUM>) center_z;
unsigned int(<NUM>) viewpoint _switch_mode;
if (alignment_group_content_coverage_flag ==<NUM>) {
SphereRegionStruct(l);
}
}
}
}
aligned(<NUM>) class ViewpointAlignmentOffset() {
RotationStruct();
aligned(<NUM>) class RotationStruct() {
signed int(<NUM>) rotation_yaw;
signed int(<NUM>) rotation_pitch;
signed int(<NUM>) rotation_roll;
}.

Among the fields of the above syntax, the fields which have not been described before have the following semantics.

In viewpoint_switch mode set to <NUM>, the OMAF player is expected to change the viewport orientation such that it corresponds to the global coordinates as specified by the group initial orientation of the viewpoint.

The values of <NUM> to <NUM> for viewpoint_switch mode are reserved.

In some embodiments, the metadata may further include a viewpoint list box, OMAFViewpointListBox for a list of viewpoint IDs and a hint as to where to find corresponding information metadata for viewpoints. The viewpoint list box OMAFViewpointListBox may be defined in a meta box 'meta' included in a movie box 'moov'. An exemplary syntax of the viewpoint list box, OMAFViewpointListBox is given as follows.

aligned(<NUM>) class OMAFViewpointListBox extends Box('vpli') {
unsigned int(<NUM>) num_viewpoints;
for (i = <NUM>; i < num_viewpoints; i++) {
bit(<NUM>) reserved = <NUM>;
unsigned int(<NUM>) viewpoint_id;
unsigned int(<NUM>) initial_viewpoint_flag;
unsigned int(<NUM>) viewpoint_delivery_type_flag;
if (viewpoint_delivery_type_flag == <NUM>)
unsigned int(<NUM>) track_ID;
}
if (viewpoint_delivery_type_flag == <NUM>)
unsigned int(<NUM>) track_group_ID;
}
}.

A viewpoint may be represented by one track only or a collective group of tracks, each including a part of <NUM> video viewpoints. A viewpoint ID is assigned to each viewpoint. The viewpoint ID may be independent of track IDs. However, there should be a form of association for referencing and linking the ID of a viewpoint and a track carrying content. The simplest method may be to map track IDs to the IDs of single track-delivered viewpoints. A track group ID may be used for a multi-track-delivered viewpoint. Depending on whether a viewpoint is delivered in a single track or multiple tracks, the viewpoint ID of the viewpoint may correspond to either <NUM>) a track ID or <NUM>) a track group ID.

The semantics of the fields used in the above syntax are given as follows.

A new box "vpin" (OMAFViewpointInformationBox) may be defined in a 'meta' box contained in a track box 'trak', for transmission of single-track viewpoint information metadata. If any viewpoint is specified as having viewpoint_delivery_type_flag equal to <NUM> in a viewpoint list box 'vpli', the viewpoint information box, OMAFViewpointInformationBox should exist in the meta box included in the track box for a track corresponding to the viewpoint. An exemplary syntax of the viewpoint information box, OMAFViewpointInformationBox is given as follows. aligned(<NUM>) class OMAFViewpointInfoBox extends Box('vpin') {
unsigned int(<NUM>) viewpoint_id;
ViewpointInfoStruct()
}
aligned(<NUM>) ViewpointlnfoStruct() {
ViewpointPosStruct() ;
ViewpointGlobalCoordinateSysRotationStruct() ;
unsigned int(<NUM>) viewpoint_switch_mode;
string viewpoint_description;
}
aligned(<NUM>) ViewpointPosStruct() {
signed int(<NUM>) viewpoint_pos_x;
signed int(<NUM>) viewpoint_pos y;
signed int(<NUM>) viewpoint_pos_z;
}
aligned(<NUM>) class ViewpointGlobalCoordinateSysRotationStruct() {
signed int(<NUM>) viewpoint_gcs_yaw;
signed int(<NUM>) viewpoint_gcs_pitch;
signed int(<NUM>) viewpoint_gcs_roll;.

The semantics of the above syntax is given as follows.

When a user switches between two different viewpoints, switching between the orientations of viewports before and after the viewpoint switching depends on the locations and content of related viewpoints. Therefore, there may be multiple viewpoint switching modes for a viewport, including a plurality of switching modes such as forward LoS, reverse LoS, center (-estimated) non-LoS, and content depth-enhanced non-LoS.

In some embodiments, metadata for content including a plurality of viewpoints may further include the following fields.

In some embodiments, a track group type "vipo" may be defined. Tracks containing the same value as track_group_id in TrackGroupTypeBox having the same track_group_type as "vipo" are a collective group of tracks belonging to the same viewpoint. When any viewpoint is specified as having viewpoint_delivery_type_flag equal to <NUM> in ViewpointListBox "vpli", there should be TrackGroupTypeBox having the same value as track_group_type identical to "vipo" and track_group_id in all tracks belonging to the same viewpoint. Bit <NUM> (the least significant bit (LSB)) of each of the flags of TrackGroupTypeBox is used to indicate the uniqueness of track_group_id. A related exemplary syntax is given as follows. aligned(<NUM>) class ViewpointBox extends TrackGroupTypeBox('vipo') {
// track_group_id is inherited from TrackGroupTypeBox;
unsigned int(<NUM>) viewpoint_id;
ViewpointInfoStruct()
}
aligned(<NUM>) ViewpointlnfoStruct() {
ViewpointPosStruct() ;
ViewpointGlobalCoordinateSysRotationStruct() ;
unsigned int(<NUM>) viewpoint_switch_mode;
string viewpoint_description;
}
aligned(<NUM>) ViewpointPosStruct() {
signed int(<NUM>) viewpoint_pos_x;
signed int(<NUM>) viewpoint_pos_y;
signed int(<NUM>) viewpoint_pos_z;
}
aligned(<NUM>) class ViewpointGlobalCoordinateSysRotationStruct() {
signed int(<NUM>) viewpoint_gcs_yaw;
signed int(<NUM>) viewpoint_gcs_pitch;
signed int(<NUM>) viewpoint_gcs_roll;
}.

In some embodiments, a new track group type "algr" may be defined. Tracks including the value of track_group_id in TrackGroupTypeBox having the same track_group_type as "algr" are a collective group of tracks belonging to the same alignment group. Bit <NUM> (bit <NUM> is the LSB) of each of the flags of TrackGroupTypeBox is used to indicate the uniqueness of track_group_id. An exemplary related syntax is given as follows. aligned(<NUM>) class AlignmentGroupBox extends TrackGroupTypeBox('algr') {
// track_group_id is inherited from TrackGroupTypeBox;
AlignmentGroupInfoStruct()
}
aligned(<NUM>) AlignmentGroupInfoStruct () {
bit(<NUM>) reserved = <NUM>;
unsigned int(<NUM>) alignment_group_id;
unsigned int(<NUM>) group common_reference flag;
unsigned int(<NUM>) alignment_group_initial_orientation_flag;
if (alignment_group_initial_orientation_flag == <NUM>) {
signed int(<NUM>) centre_x;
signed int(<NUM>) centre_y;
signed int(<NUM>) centre_z;
}
string group_description; }.

Content may have multiple viewpoints, some of which may be captured in different scenes and locations. If all viewpoints are aligned in the same reference coordinate system, certain viewpoint positions may be irrational for representation in viewpoint position metadata. Viewpoint position metadata is mainly used to have coordinated viewport switching, when switching occurs between two viewpoints. In some situations, it may be desirable to group the viewpoints such that viewpoints are aligned with respect to other viewpoints only within the same group, and viewpoints that do not belong to the same arrangement group are not aligned with each other. group_common reference flag is used to indicate whether the viewpoints of the alignment group are globally or locally aligned with respect to the alignment group.

Even though any viewpoint is selected for switching in a group, initial orientation may be set for alignment groups as a whole by defining alignment groups containing a group of viewpoints, such that a client device may display a viewpoint corresponding to the center X, Y, Z point defined in space in the reference coordinate system. The alignment grouping mechanism described above may use a track grouping design without the need for explicitly specifying any viewpoint ID. Further, it is possible to simply list a set of viewpoints that use viewpoint IDs to define an alignment group by using the track grouping mechanism. It is possible to directly identify whether a track belongs to the same alignment group by its track_group_id (because track_group_id is unique) without the need for identifying its viewpoint_id first of all. The viewpoint_id of the track may be individually known through one of a track group type box having the same group type as 'vipo', an OMAF viewpoint information box, or an OMAR viewpoint list box.

Various exemplary information, boxes, fields, and parameters that may be included in metadata have been described above. However, the names of boxes, fields, and parameters are only examples, and those skilled in the art are will apparently understand that the names may be freely changed while maintaining the essential properties of the fields and parameters. Further, it will be apparent to those skilled in the art that mapping of values and attributes to fields and parameters may be changed according to selection. The metadata may be configured to include all or at least one selected one of the exemplary boxes described above. Each of the boxes described above may be configured to include all or at least one selected one of the fields described in relation to the corresponding box.

<FIG> is a flowchart illustrating an operation of an electronic device according to an embodiment of the present disclosure. The electronic device <NUM> may receive metadata for 3D content including a plurality of viewpoints in operation <NUM>. The metadata received at the electronic device <NUM> may be transmitted from the afore-described server <NUM>.

The electronic device <NUM> may process media data for the 3D content based on the received metadata in operation <NUM>. The media data for the 3D content may be transmitted along with or separately from the metadata from the server <NUM>. In some embodiments, the electronic device <NUM> may receive the media data from a server different from the server <NUM> which transmits the metadata. According to some embodiments, the electronic device <NUM> may receive the media data from a server of the content provider <NUM>. According to some embodiments, the electronic device <NUM> may obtain the media data from a storage device such as a compact disk read-only memory (CD-ROM), a digital versatile disk read-only memory (DVD-ROM), a hard disk, a floppy disk, or a universal serial bus (USB) storage device. The electronic device <NUM> may play back the 3D content on a display included in the electronic device <NUM> by processing the media data for the 3D content based on the received metadata. In some embodiments, the electronic device <NUM> may transmit a signal for reproducing the 3D content (a video signal and an audio signal) on a display outside of the electronic device <NUM> to another electronic device having the display.

<FIG> is a block diagram illustrating a server according to an embodiment of the present disclosure. A server <NUM> may be identical to the server <NUM> illustrated in <FIG>. The server <NUM> may include a controller <NUM>, a transceiver <NUM>, and a memory <NUM>.

The controller <NUM> may perform computations and functions required for operations of the server <NUM>. The controller <NUM> may be connected to elements of the server <NUM>, including the transceiver <NUM> and the memory <NUM> and control operations of the elements. Therefore, the operations of the server <NUM> may be interpreted as performed substantially by the controller <NUM>. The controller <NUM> may be configured with at least one processor.

The server <NUM> may communicate with other entities through the transceiver <NUM>. The transceiver <NUM> may include wired or wireless communication interfaces. The transceiver <NUM> may conduct communication by known wired or wireless communication protocols such as wireless fidelity (Wi-Fi), long term evolution (LTE), code division multiple access (CDMA), worldwide interoperability for microwave access (Wi-MAX), wireless broadband (Wi-Bro), and USB.

The memory <NUM> may include information required for operations of the server <NUM> and the controller <NUM>. For example, the memory <NUM> may store temporary or non-temporary data required for computations of the controller <NUM>. The memory <NUM> may store instructions executable by the controller <NUM>. The memory <NUM> may be configured to include at least one of a transitory memory, a non-transitory memory, a rewritable memory, or a non-rewritable memory.

<FIG> is a block diagram illustrating an electronic device according to an embodiment of the present disclosure. An electronic device <NUM> may be substantially identical to the electronic device <NUM> illustrated in <FIG>. The electronic device <NUM> may include a controller <NUM>, a transceiver <NUM>, a memory <NUM>, and a display <NUM>.

The description of the controller <NUM>, the transceiver <NUM>, and the memory <NUM> is substantially identical to that of the controller <NUM>, the transceiver <NUM>, and the memory <NUM> of the server <NUM>, and thus will not be provided herein.

The display <NUM> may reproduce 3D content under the control of the controller <NUM>. According to some embodiments, the electronic device <NUM> may reproduce the 3D content on a display of another electronic device, instead of the display <NUM>. According to some embodiments, the electronic device <NUM> may not include the display <NUM>.

Claim 1:
A method for transmitting metadata for omnidirectional content including a plurality of viewpoints, the method comprising:
identifying the metadata for the omnidirectional content including the plurality of viewpoints; and
transmitting the identified metadata,
wherein the metadata includes information about an identifier, ID, of a viewpoint group including at least two viewpoints of the plurality of viewpoints, each viewpoint in the viewpoint group being adjacent to each other in a same content scene, and each viewpoint in the viewpoint group being aligned with respect to coordinate axes of a default viewpoint in the viewpoint group.
wherein the coordinate axes of the default viewpoint are shared as a common reference coordinate system between the at least two viewpoints in the viewpoint group,
wherein the each viewpoint in the viewpoint group is aligned with respect to coordinate axes of the default viewpoint in the viewpoint group based on yaw, pitch, and roll rotation angles, respectively, of X, Y, and Z axes of a global coordinate system of the each viewpoint relative to the common reference coordinate system; and
wherein each viewpoint corresponds to a location from which a corresponding omnidirectional image is captured by a corresponding camera.