Patent ID: 12231611

EXAMPLE NETWORKS FOR IMPLEMENTATION OF THE EMBODIMENTS

FIG.1Ais a diagram illustrating an example communications system100in which one or more disclosed embodiments may be implemented. The communications system100may be a multiple access system that provides content, such as voice, data, video, messaging, broadcast, etc., to multiple wireless users. The communications system100may enable multiple wireless users to access such content through the sharing of system resources, including wireless bandwidth. For example, the communications systems100may employ one or more channel access methods, such as code division multiple access (CDMA), time division multiple access (TDMA), frequency division multiple access (FDMA), orthogonal FDMA (OFDMA), single-carrier FDMA (SC-FDMA), zero-tail unique-word DFT-Spread OFDM (ZT UW DTS-s OFDM), unique word OFDM (UW-OFDM), resource block-filtered OFDM, filter bank multicarrier (FBMC), and the like.

As shown inFIG.1A, the communications system100may include wireless transmit/receive units (WTRUs)102a,102b,102c,102d, a RAN104/113, a CN106/115, a public switched telephone network (PSTN)108, the Internet110, and other networks112, though it will be appreciated that the disclosed embodiments contemplate any number of WTRUs, base stations, networks, and/or network elements. Each of the WTRUs102a,102b,102c,102dmay be any type of device configured to operate and/or communicate in a wireless environment. By way of example, the WTRUs102a,102b,102c,102d, any of which may be referred to as a “station” and/or a “STA”, may be configured to transmit and/or receive wireless signals and may include a user equipment (UE), a mobile station, a fixed or mobile subscriber unit, a subscription-based unit, a pager, a cellular telephone, a personal digital assistant (PDA), a smartphone, a laptop, a netbook, a personal computer, a wireless sensor, a hotspot or Mi-Fi device, an Internet of Things (IoT) device, a watch or other wearable, a head-mounted display (HMD), a vehicle, a drone, a medical device and applications (e.g., remote surgery), an industrial device and applications (e.g., a robot and/or other wireless devices operating in an industrial and/or an automated processing chain contexts), a consumer electronics device, a device operating on commercial and/or industrial wireless networks, and the like. Any of the WTRUs102a,102b,102cand102dmay be interchangeably referred to as a UE.

The communications systems100may also include a base station114aand/or a base station114b. Each of the base stations114a,114bmay be any type of device configured to wirelessly interface with at least one of the WTRUs102a,102b,102c,102dto facilitate access to one or more communication networks, such as the CN106/115, the Internet110, and/or the other networks112. By way of example, the base stations114a,114bmay be a base transceiver station (BTS), a Node-B, an eNode B, a Home Node B, a Home eNode B, a gNB, a NR NodeB, a site controller, an access point (AP), a wireless router, and the like. While the base stations114a,114bare each depicted as a single element, it will be appreciated that the base stations114a,114bmay include any number of interconnected base stations and/or network elements.

The base station114amay be part of the RAN104/113, which may also include other base stations and/or network elements (not shown), such as a base station controller (BSC), a radio network controller (RNC), relay nodes, etc. The base station114aand/or the base station114bmay be configured to transmit and/or receive wireless signals on one or more carrier frequencies, which may be referred to as a cell (not shown). These frequencies may be in licensed spectrum, unlicensed spectrum, or a combination of licensed and unlicensed spectrum. A cell may provide coverage for a wireless service to a specific geographical area that may be relatively fixed or that may change over time. The cell may further be divided into cell sectors. For example, the cell associated with the base station114amay be divided into three sectors. Thus, in one embodiment, the base station114amay include three transceivers, i.e., one for each sector of the cell. In an embodiment, the base station114amay employ multiple-input multiple output (MIMO) technology and may utilize multiple transceivers for each sector of the cell. For example, beamforming may be used to transmit and/or receive signals in desired spatial directions.

The base stations114a,114bmay communicate with one or more of the WTRUs102a,102b,102c,102dover an air interface116, which may be any suitable wireless communication link (e.g., radio frequency (RF), microwave, centimeter wave, micrometer wave, infrared (IR), ultraviolet (UV), visible light, etc.). The air interface116may be established using any suitable radio access technology (RAT).

More specifically, as noted above, the communications system100may be a multiple access system and may employ one or more channel access schemes, such as CDMA, TDMA, FDMA, OFDMA, SC-FDMA, and the like. For example, the base station114ain the RAN104/113and the WTRUs102a,102b,102cmay implement a radio technology such as Universal Mobile Telecommunications System (UMTS) Terrestrial Radio Access (UTRA), which may establish the air interface115/116/117using wideband CDMA (WCDMA). WCDMA may include communication protocols such as High-Speed Packet Access (HSPA) and/or Evolved HSPA (HSPA+). HSPA may include High-Speed Downlink (DL) Packet Access (HSDPA) and/or High-Speed UL Packet Access (HSUPA).

In an embodiment, the base station114aand the WTRUs102a,102b,102cmay implement a radio technology such as Evolved UMTS Terrestrial Radio Access (E-UTRA), which may establish the air interface116using Long Term Evolution (LTE) and/or LTE-Advanced (LTE-A) and/or LTE-Advanced Pro (LTE-A Pro).

In an embodiment, the base station114aand the WTRUs102a,102b,102cmay implement a radio technology such as NR Radio Access, which may establish the air interface116using New Radio (NR).

In an embodiment, the base station114aand the WTRUs102a,102b,102cmay implement multiple radio access technologies. For example, the base station114aand the WTRUs102a,102b,102cmay implement LTE radio access and NR radio access together, for instance using dual connectivity (DC) principles. Thus, the air interface utilized by WTRUs102a,102b,102cmay be characterized by multiple types of radio access technologies and/or transmissions sent to/from multiple types of base stations (e.g., a eNB and a gNB).

In other embodiments, the base station114aand the WTRUs102a,102b,102cmay implement radio technologies such as IEEE 802.11 (i.e., Wireless Fidelity (WiFi), IEEE 802.16 (i.e., Worldwide Interoperability for Microwave Access (WiMAX)), CDMA2000, CDMA2000 1×, CDMA2000 EV-DO, Interim Standard 2000 (IS-2000), Interim Standard 95 (IS-95), Interim Standard 856 (IS-856), Global System for Mobile communications (GSM), Enhanced Data rates for GSM Evolution (EDGE), GSM EDGE (GERAN), and the like.

The base station114binFIG.1Amay be a wireless router, Home Node B, Home eNode B, or access point, for example, and may utilize any suitable RAT for facilitating wireless connectivity in a localized area, such as a place of business, a home, a vehicle, a campus, an industrial facility, an air corridor (e.g., for use by drones), a roadway, and the like. In one embodiment, the base station114band the WTRUs102c,102dmay implement a radio technology such as IEEE 802.11 to establish a wireless local area network (WLAN). In an embodiment, the base station114band the WTRUs102c,102dmay implement a radio technology such as IEEE 802.15 to establish a wireless personal area network (WPAN). In yet another embodiment, the base station114band the WTRUs102c,102dmay utilize a cellular-based RAT (e.g., WCDMA, CDMA2000, GSM, LTE, LTE-A, LTE-A Pro, NR etc.) to establish a picocell or femtocell. As shown inFIG.1A, the base station114bmay have a direct connection to the Internet110. Thus, the base station114bmay not be required to access the Internet110via the CN106/115.

The RAN104/113may be in communication with the CN106/115, which may be any type of network configured to provide voice, data, applications, and/or voice over internet protocol (VoIP) services to one or more of the WTRUs102a,102b,102c,102d. The data may have varying quality of service (QoS) requirements, such as differing throughput requirements, latency requirements, error tolerance requirements, reliability requirements, data throughput requirements, mobility requirements, and the like. The CN106/115may provide call control, billing services, mobile location-based services, pre-paid calling, Internet connectivity, video distribution, etc., and/or perform high-level security functions, such as user authentication. Although not shown inFIG.1A, it will be appreciated that the RAN104/113and/or the CN106/115may be in direct or indirect communication with other RANs that employ the same RAT as the RAN104/113or a different RAT. For example, in addition to being connected to the RAN104/113, which may be utilizing a NR radio technology, the CN106/115may also be in communication with another RAN (not shown) employing a GSM, UMTS, CDMA 2000, WiMAX, E-UTRA, or WiFi radio technology.

The CN106/115may also serve as a gateway for the WTRUs102a,102b,102c,102dto access the PSTN108, the Internet110, and/or the other networks112. The PSTN108may include circuit-switched telephone networks that provide plain old telephone service (POTS). The Internet110may include a global system of interconnected computer networks and devices that use common communication protocols, such as the transmission control protocol (TCP), user datagram protocol (UDP) and/or the internet protocol (IP) in the TCP/IP internet protocol suite. The networks112may include wired and/or wireless communications networks owned and/or operated by other service providers. For example, the networks112may include another CN connected to one or more RANs, which may employ the same RAT as the RAN104/113or a different RAT.

Some or all of the WTRUs102a,102b,102c,102din the communications system100may include multi-mode capabilities (e.g., the WTRUs102a,102b,102c,102dmay include multiple transceivers for communicating with different wireless networks over different wireless links). For example, the WTRU102cshown inFIG.1Amay be configured to communicate with the base station114a, which may employ a cellular-based radio technology, and with the base station114b, which may employ an IEEE 802 radio technology.

FIG.1Bis a system diagram illustrating an example WTRU102. As shown inFIG.1B, the WTRU102may include a processor118, a transceiver120, a transmit/receive element122, a speaker/microphone124, a keypad126, a display/touchpad128, non-removable memory130, removable memory132, a power source134, a global positioning system (GPS) chipset136, and/or other peripherals138, among others. It will be appreciated that the WTRU102may include any sub-combination of the foregoing elements while remaining consistent with an embodiment.

The processor118may be a general purpose processor, a special purpose processor, a conventional processor, a digital signal processor (DSP), a plurality of microprocessors, one or more microprocessors in association with a DSP core, a controller, a microcontroller, Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) circuits, any other type of integrated circuit (IC), a state machine, and the like. The processor118may perform signal coding, data processing, power control, input/output processing, and/or any other functionality that enables the WTRU102to operate in a wireless environment. The processor118may be coupled to the transceiver120, which may be coupled to the transmit/receive element122. WhileFIG.1Bdepicts the processor118and the transceiver120as separate components, it will be appreciated that the processor118and the transceiver120may be integrated together in an electronic package or chip.

The transmit/receive element122may be configured to transmit signals to, or receive signals from, a base station (e.g., the base station114a) over the air interface116. For example, in one embodiment, the transmit/receive element122may be an antenna configured to transmit and/or receive RF signals. In an embodiment, the transmit/receive element122may be an emitter/detector configured to transmit and/or receive IR, UV, or visible light signals, for example. In yet another embodiment, the transmit/receive element122may be configured to transmit and/or receive both RF and light signals. It will be appreciated that the transmit/receive element122may be configured to transmit and/or receive any combination of wireless signals.

Although the transmit/receive element122is depicted inFIG.1Bas a single element, the WTRU102may include any number of transmit/receive elements122. More specifically, the WTRU102may employ MIMO technology. Thus, in one embodiment, the WTRU102may include two or more transmit/receive elements122(e.g., multiple antennas) for transmitting and receiving wireless signals over the air interface116.

The transceiver120may be configured to modulate the signals that are to be transmitted by the transmit/receive element122and to demodulate the signals that are received by the transmit/receive element122. As noted above, the WTRU102may have multi-mode capabilities. Thus, the transceiver120may include multiple transceivers for enabling the WTRU102to communicate via multiple RATs, such as NR and IEEE 802.11, for example.

The processor118of the WTRU102may be coupled to, and may receive user input data from, the speaker/microphone124, the keypad126, and/or the display/touchpad128(e.g., a liquid crystal display (LCD) display unit or organic light-emitting diode (OLED) display unit). The processor118may also output user data to the speaker/microphone124, the keypad126, and/or the display/touchpad128. In addition, the processor118may access information from, and store data in, any type of suitable memory, such as the non-removable memory130and/or the removable memory132. The non-removable memory130may include random-access memory (RAM), read-only memory (ROM), a hard disk, or any other type of memory storage device. The removable memory132may include a subscriber identity module (SIM) card, a memory stick, a secure digital (SD) memory card, and the like. In other embodiments, the processor118may access information from, and store data in, memory that is not physically located on the WTRU102, such as on a server or a home computer (not shown).

The processor118may receive power from the power source134, and may be configured to distribute and/or control the power to the other components in the WTRU102. The power source134may be any suitable device for powering the WTRU102. For example, the power source134may include one or more dry cell batteries (e.g., nickel-cadmium (NiCd), nickel-zinc (NiZn), nickel metal hydride (NiMH), lithium-ion (Li-ion), etc.), solar cells, fuel cells, and the like.

The processor118may also be coupled to the GPS chipset136, which may be configured to provide location information (e.g., longitude and latitude) regarding the current location of the WTRU102. In addition to, or in lieu of, the information from the GPS chipset136, the WTRU102may receive location information over the air interface116from a base station (e.g., base stations114a,114b) and/or determine its location based on the timing of the signals being received from two or more nearby base stations. It will be appreciated that the WTRU102may acquire location information by way of any suitable location-determination method while remaining consistent with an embodiment.

The processor118may further be coupled to other peripherals138, which may include one or more software and/or hardware modules that provide additional features, functionality and/or wired or wireless connectivity. For example, the peripherals138may include an accelerometer, an e-compass, a satellite transceiver, a digital camera (for photographs and/or video), a universal serial bus (USB) port, a vibration device, a television transceiver, a hands free headset, a Bluetooth® module, a frequency modulated (FM) radio unit, a digital music player, a media player, a video game player module, an Internet browser, a Virtual Reality and/or Augmented Reality (VR/AR) device, an activity tracker, and the like. The peripherals138may include one or more sensors, the sensors may be one or more of a gyroscope, an accelerometer, a hall effect sensor, a magnetometer, an orientation sensor, a proximity sensor, a temperature sensor, a time sensor; a geolocation sensor; an altimeter, a light sensor, a touch sensor, a magnetometer, a barometer, a gesture sensor, a biometric sensor, and/or a humidity sensor.

The WTRU102may include a full duplex radio for which transmission and reception of some or all of the signals (e.g., associated with particular subframes for both the UL (e.g., for transmission) and downlink (e.g., for reception) may be concurrent and/or simultaneous. The full duplex radio may include an interference management unit to reduce and or substantially eliminate self-interference via either hardware (e.g., a choke) or signal processing via a processor (e.g., a separate processor (not shown) or via processor118). In an embodiment, the WRTU102may include a half-duplex radio for which transmission and reception of some or all of the signals (e.g., associated with particular subframes for either the UL (e.g., for transmission) or the downlink (e.g., for reception)).

Although the WTRU is described inFIGS.1A-1Bas a wireless terminal, it is contemplated that in certain representative embodiments that such a terminal may use (e.g., temporarily or permanently) wired communication interfaces with the communication network.

In representative embodiments, the other network112may be a WLAN.

A WLAN in Infrastructure Basic Service Set (BSS) mode may have an Access Point (AP) for the BSS and one or more stations (STAs) associated with the AP. The AP may have an access or an interface to a Distribution System (DS) or another type of wired/wireless network that carries traffic in to and/or out of the BSS. Traffic to STAs that originates from outside the BSS may arrive through the AP and may be delivered to the STAs. Traffic originating from STAs to destinations outside the BSS may be sent to the AP to be delivered to respective destinations. Traffic between STAs within the BSS may be sent through the AP, for example, where the source STA may send traffic to the AP and the AP may deliver the traffic to the destination STA. The traffic between STAs within a BSS may be considered and/or referred to as peer-to-peer traffic. The peer-to-peer traffic may be sent between (e.g., directly between) the source and destination STAs with a direct link setup (DLS). In certain representative embodiments, the DLS may use an 802.11e DLS or an 802.11z tunneled DLS (TDLS). A WLAN using an Independent BSS (IBSS) mode may not have an AP, and the STAs (e.g., all of the STAs) within or using the IBSS may communicate directly with each other. The IBSS mode of communication may sometimes be referred to herein as an “ad-hoc” mode of communication.

When using the 802.11ac infrastructure mode of operation or a similar mode of operations, the AP may transmit a beacon on a fixed channel, such as a primary channel. The primary channel may be a fixed width (e.g., 20 MHz wide bandwidth) or a dynamically set width via signaling. The primary channel may be the operating channel of the BSS and may be used by the STAs to establish a connection with the AP. In certain representative embodiments, Carrier Sense Multiple Access with Collision Avoidance (CSMA/CA) may be implemented, for example in in 802.11 systems. For CSMA/CA, the STAs (e.g., every STA), including the AP, may sense the primary channel. If the primary channel is sensed/detected and/or determined to be busy by a particular STA, the particular STA may back off. One STA (e.g., only one station) may transmit at any given time in a given BSS.

High Throughput (HT) STAs may use a 40 MHz wide channel for communication, for example, via a combination of the primary 20 MHz channel with an adjacent or nonadjacent 20 MHz channel to form a 40 MHz wide channel.

Very High Throughput (VHT) STAs may support 20 MHz, 40 MHz, 80 MHz, and/or 160 MHz wide channels. The 40 MHz, and/or 80 MHz, channels may be formed by combining contiguous 20 MHz channels. A 160 MHz channel may be formed by combining 8 contiguous 20 MHz channels, or by combining two non-contiguous 80 MHz channels, which may be referred to as an 80+80 configuration. For the 80+80 configuration, the data, after channel encoding, may be passed through a segment parser that may divide the data into two streams. Inverse Fast Fourier Transform (IFFT) processing, and time domain processing, may be done on each stream separately. The streams may be mapped on to the two 80 MHz channels, and the data may be transmitted by a transmitting STA. At the receiver of the receiving STA, the above described operation for the 80+80 configuration may be reversed, and the combined data may be sent to the Medium Access Control (MAC).

Sub 1 GHz modes of operation are supported by 802.11af and 802.11ah. The channel operating bandwidths, and carriers, are reduced in 802.11af and 802.11ah relative to those used in 802.11n, and 802.11ac. 802.11af supports 5 MHz, 10 MHz and 20 MHz bandwidths in the TV White Space (TVWS) spectrum, and 802.11ah supports 1 MHz, 2 MHz, 4 MHz, 8 MHz, and 16 MHz bandwidths using non-TVWS spectrum. According to a representative embodiment, 802.11ah may support Meter Type Control/Machine-Type Communications, such as MTC devices in a macro coverage area. MTC devices may have certain capabilities, for example, limited capabilities including support for (e.g., only support for) certain and/or limited bandwidths. The MTC devices may include a battery with a battery life above a threshold (e.g., to maintain a very long battery life).

WLAN systems, which may support multiple channels, and channel bandwidths, such as 802.11n, 802.11ac, 802.11af, and 802.11ah, include a channel which may be designated as the primary channel. The primary channel may have a bandwidth equal to the largest common operating bandwidth supported by all STAs in the BSS. The bandwidth of the primary channel may be set and/or limited by a STA, from among all STAs in operating in a BSS, which supports the smallest bandwidth operating mode. In the example of 802.11ah, the primary channel may be 1 MHz wide for STAs (e.g., MTC type devices) that support (e.g., only support) a 1 MHz mode, even if the AP, and other STAs in the BSS support 2 MHz, 4 MHz, 8 MHz, 16 MHz, and/or other channel bandwidth operating modes. Carrier sensing and/or Network Allocation Vector (NAV) settings may depend on the status of the primary channel. If the primary channel is busy, for example, due to a STA (which supports only a 1 MHz operating mode), transmitting to the AP, the entire available frequency bands may be considered busy even though a majority of the frequency bands remains idle and may be available.

In the United States, the available frequency bands, which may be used by 802.11ah, are from 902 MHz to 928 MHz. In Korea, the available frequency bands are from 917.5 MHz to 923.5 MHz. In Japan, the available frequency bands are from 916.5 MHz to 927.5 MHz. The total bandwidth available for 802.11ah is 6 MHz to 26 MHz depending on the country code.

In view ofFIGS.1A-1B, and the corresponding description ofFIGS.1A-1B, one or more, or all, of the functions described herein with regard to one or more of: WTRU102a-d, Base Station114a-b, eNode-B160a-c, MME162, SGW164, PGW166, gNB180a-c, AMF182a-b, UPF184a-b, SMF183a-b, DN185a-b, and/or any other device(s) described herein, may be performed by one or more emulation devices (not shown). The emulation devices may be one or more devices configured to emulate one or more, or all, of the functions described herein. For example, the emulation devices may be used to test other devices and/or to simulate network and/or WTRU functions.

The emulation devices may be designed to implement one or more tests of other devices in a lab environment and/or in an operator network environment. For example, the one or more emulation devices may perform the one or more, or all, functions while being fully or partially implemented and/or deployed as part of a wired and/or wireless communication network in order to test other devices within the communication network. The one or more emulation devices may perform the one or more, or all, functions while being temporarily implemented/deployed as part of a wired and/or wireless communication network. The emulation device may be directly coupled to another device for purposes of testing and/or may performing testing using over-the-air wireless communications.

The one or more emulation devices may perform the one or more, including all, functions while not being implemented/deployed as part of a wired and/or wireless communication network. For example, the emulation devices may be utilized in a testing scenario in a testing laboratory and/or a non-deployed (e.g., testing) wired and/or wireless communication network in order to implement testing of one or more components. The one or more emulation devices may be test equipment. Direct RF coupling and/or wireless communications via RF circuitry (e.g., which may include one or more antennas) may be used by the emulation devices to transmit and/or receive data.

DETAILED DESCRIPTION

Issues Addressed in Some Embodiments.

An omnidirectional video provides a 360-degree experience that enables the viewer to watch the video in all directions around a central viewing position. However, the viewer has generally been limited to a single viewpoint and is not able to navigate the scene by changing their viewpoint. For large-scale events such as the Olympic Games opening ceremony, NFL or NBA tournaments, carnival parades, etc., a single 360° video camera is not enough to capture the entire scene. A more enhanced experience can be provided by capturing the scene from multiple viewpoints and providing the user with the ability to switch between different viewpoints while watching the video.FIG.3shows a user interface that may be presented to the user in some embodiments to indicate available viewpoints. In this example, the user interface displays an overhead view of a venue and provides indications of locations of available viewpoints. In this case, viewpoint302is the active viewpoint (the viewpoint from which the user is currently experiencing the presentation) and is displayed in a highlighted fashion. Other viewpoints, such as viewpoints304,306,308,310,312,314,316, may be displayed to indicate their availability, but they are not currently selected by the user.

During playback, a user interface such as that illustrated inFIG.3may be overlaid over the rendered frame at one of the four corners for example, and the user can select a different viewpoint using a user input device such as a touch screen or an HMD controller. A viewpoint switch is then triggered and the user's view is transitioned so that frames from the target viewpoint are rendered on the display. In some embodiments, a transition effect (e.g., fade-out/fade-in) accompanies the transition between viewpoints.

FIG.4illustrates another user design example in which the location of available viewpoints is indicated using icons displayed as overlays on content400displayed on a head-mounted display. The position of each viewpoint icon in the user's view corresponds to the spatial position of an available viewpoint. In the example ofFIG.4, icons406,414may be displayed to correspond to the viewpoints306,314, respectively, ofFIG.3. The viewpoint icons may be rendered with the correct depth effect to allow the user to perceive each viewpoint position in three dimensional space within the scene. For example, an icon416(corresponding to viewpoint position316) may be displayed at a larger size than icons406,414to indicate that the viewpoint corresponding to icon416is closer to the current viewpoint. The user can select a viewpoint icon in order to switch the user's view of the rendered scene to the associated viewpoint.

In example embodiments, to enable support for multiple viewpoints, information about the available viewpoints is signaled to the player (which may, for example, be an omnidirectional media player equipped with a DASH client running on a user device such as an HMD). This information may include aspects such as the number of available viewpoints, the position and range of each viewpoint, and when video data are available for a viewpoint. Moreover, since most omnidirectional media presentations are experienced through a head-mounted display, a sudden change in viewpoint may feel unnatural to a viewer who is immersed in a virtual environment. It is therefore preferable to support viewpoint transition effects that would provide a smooth transition when the user changes their viewpoint. These transitions can also be used by content producers to guide the user's experience.

Grouping Viewpoint Media Components.

In some embodiments, media samples for omnidirectional media content with multiple viewpoints are stored in a number of tracks within the container file. A video player playing or streaming the content operates to identify which tracks belong to which viewpoint. To enable this, a mapping is provided between the media tracks within the file and the viewpoint to which they belong. In some embodiments, this mapping is signaled at the media container (file format) level. In some embodiments, this mapping is signaled at the transport protocol-level (DASH).

Media Container-Level Signaling (File Format).

In ISO/IEC 14496-12 (ISO BMFF), a TrackGroupBox is defined to enable grouping a number of tracks in the container file that share a certain characteristic or have a particular relationship. The track group box contains zero or more boxes, and the particular characteristic or the relationship is indicated by the box type of the contained boxes. The contained boxes include an identifier, which can be used to conclude the tracks belonging to the same track group. The tracks that contain the same type of a contained box within the TrackGroupBox and have the same identifier value within these contained boxes belong to the same track group.

aligned(8) class TrackGroupBox extends Box(‘trgr’) {}

A track group type is defined extending a TrackGroupTypeBox which contains a track_group_id identifier and a track_group_type which stores a four-character code identifying the group type. The pair of track_group_id and track_group_type identifies a track group within the file.

To group a number of media tracks that belong a single viewpoint together, in some embodiments, a new group type (ViewpointGroupTypeBox) is defined as follows:

aligned(8) class ViewpointGroupTypeBox extendsTrackGroupTypeBox(‘vpgr’) {// additional viewpoint data can be defined here}

In some embodiments, media have a ViewpointGroupTypeBox within the TrackGroupBox, and tracks that belong to the same viewpoint have the same value of track_group_id in the respective ViewpointGroupTypeBox. A 3DoF+ omnidirectional media player can therefore identify available viewpoints by parsing each track in the container and checking the number of unique track_group_id values within the ViewpointGroupTypeBox for each track.

Transport Protocol-Level Signaling (DASH).

The OMAF standard defines delivery-related interfaces for DASH. In some embodiments, information related to the different viewpoints is signaled in the media presentation descriptor. In DASH, each media component is represented by an AdaptationSet element in the MPD. In some embodiments, AdaptationSet elements belonging to the same viewpoint are grouped by either defining an additional attribute to the AdaptationSet element or adding a descriptor to the AdaptationSet where a viewpoint identifier is provided.

A number of descriptors are defined in the MPEG-DASH standard. These include a SupplementalProperty descriptor which can be used by the media presentation author to express that the descriptor contains supplemental information that may be used by the DASH client for optimized processing. The semantics of the signaled information are specific to the scheme employed, which is identified by the @schemeIdUri attribute. In the present disclosure, a number of new XML elements and attributes are described for signaling viewpoint-related information. The new elements can either be defined in the same namespace as the one defined in the latest version of the OMAF standard (um:mpeg:mpegL:omaf:2017) or in a separate new namespace (um:mpeg:mpegL:omaf:2019) to distinguish between OMAF v1 features and OMAF v2 features. For the sake of explanation, the namespace (urn:mpeg:mpegL:omaf:2017) is used in the remainder of this document.

To identify and describe the viewpoint to which a media component belongs, embodiments are described in which a SupplementalProperty element is added with a @schemeIdUri attribute equal to “urn:mpeg:mpegL:omaf:2017:ovp”. Such a descriptor is referred to herein as an OMAF viewpoint (OVP) descriptor. In some embodiments, at most one OVP descriptor may be present at the adaptation set level. The OVP descriptor may have a @viewpoint_id attribute with a value representing a unique viewpoint identifier. Examples of semantics for @viewpoint_id are given in Table 1. AdaptationSet elements with the same @viewpoint_id value may be recognized by the player as belonging to the same viewpoint.

TABLE 1Semantics of omaf:@viewpoint_id attributeAttribute for OVP descriptorUseData typeDescriptionomaf:@viewpoint_idMxs:stringSpecifies a unique viewpoint within the scene.For ISO Base Media File Format Segments,the value of @viewpoint_id shall beequal to track_group_id in theViewpointGroupTypeBox in sampleentries of the Initialization Segment.
Signaling Viewpoint Information.

In order for a player to identify the attributes belonging to different viewpoints (e.g., spatial relationship between viewpoints, availability of the viewpoint, etc.), in some methods described herein, additional metadata describing the viewpoint is signaled in the container file (and in the MPD file in the case of streaming). Examples of viewpoint attributes that are signaled in some embodiments include viewpoint position, viewpoint effective range, viewpoint type, and viewpoint availability. Viewpoint position specifies the position of the viewpoint within the 3D space of the captured scene. A viewpoint's effective range is the distance from the viewpoint within which objects can be rendered with a certain level of quality. The certain level of quality may be, for example, a minimum level of quality, a quality level exceeding a known quality threshold, a guaranteed level of quality, or a level of quality approved by or deemed acceptable to the provider of the omnidirectional media content. For example, an object that is within the effective range would be of sufficient size in the rendered image to provide a resolution that provides good quality and ensures an acceptable viewing experience for a user. The effective range of a viewpoint depends on the characteristics of the capturing device (e.g., camera sensor resolution, field-of-view, etc.). The effective range may be determined at least in part by the camera lens density, representing the number of lenses integrated into a 360-degree video camera.

FIG.5shows an example of the effective range of different cameras. In this example, viewpoints502,504,506,508,510,512,514, and516are illustrated along with dotted circles503,505,507,509,511,513,515, and517indicating effective ranges of the respective viewpoints. The omnidirectional cameras located at viewpoints502and510may contain more lenses to cover a broad area, and so the effective ranges of viewpoints502and510may be able to cover the penalty areas520,522as illustrated inFIG.5. The cameras along the side of the field in this example may have fewer lenses so the effective range of these viewpoints (504,506,508,512,514,516) may be smaller than that of viewpoints502and510. In general, an omnidirectional camera which has more lenses, more component cameras, or which has higher quality component cameras (e.g. component cameras with high quality optics, high resolution, and/or high frame rate) may be associated with a higher effective range.

In another embodiment, the viewpoint effective range may be determined at least in part by camera lens parameters such as focal length, aperture, depth of field and focus distance, etc. The effective range may define a minimum range and maximum range, with the effective range being between the minimum range and maximum range without stitching error.

A viewpoint may be categorized as a real viewpoint or as a virtual viewpoint. A real viewpoint is a viewpoint where an actual capturing device was placed to capture the scene from that position of the viewpoint. A virtual viewpoint refers to a viewpoint where the rendering of viewports at that position calls for performing additional processing, such as view synthesis, which may make use of auxiliary information and/or video data from one or more other (e.g. real) viewpoints.

Viewpoint availability specifies at what time(s) during the presentation are media data available for the viewpoint.

The user's interaction with a viewport scene such as zoom-in or zoom-out may be supported within the effective range. A virtual viewpoint may only be identified within the effective range of one or multiple cameras. The effective range may also be used as reference to generate a transition path. For example, a transition from viewpoint A to viewpoint B may involve multiple transition viewpoints if the effective range of these viewpoints covers the transition path.

Media Container-Level Signaling of Viewpoint Information.

In ISO BMFF, viewpoint-related information for static viewpoints can be signaled in the ‘meta’ box (MetaBox) at the file level. The ‘meta’ box holds static metadata and contains only one mandatory ‘hdlr’ box (HandlerBox) which declares the structure or format of the MetaBox. In some embodiments, for OMAF v2 metadata, the four-character code ‘omv2’ is used for the handler_type value in the ‘hdlr’ box. To identify the available viewpoints in the file, some embodiments use a box called OMAFViewpointListBox which contains a list of OMAFViewpointInfoBox instances. Each OMAFViewpointInfoBox holds information about a certain viewpoint. An example of syntax of the OMAFViewpointListBox is as follows.

Box Type:‘ovpl’Container:MetaBoxMandatory:NoQuantity:Zero or onealigned(8) class OMAFViewpointListBox extends Box(‘ovpl’) {unsigned int(16) num_viewpoints;OMAFViewpointInfoBox viewpoints[ ];}

An example of semantics for OMAFViewpointListBox is as follows:num_viewpoints indicates the number of viewpoints in the media file.viewpoints is a list of OMAFViewpointInfoBox instances.

An example of a syntax of OMAFViewpointInfoBox is given below.

Box Type:‘ovpi’Container:OMAFViewpointListBoxMandatory:NoQuantity:Zero or morealigned(8) class OMAFViewpointInfoBox extends Box(‘ovpi’) {unsigned int(16) viewpoint_id;bit(1) effective_range_flag;bit(1) virtual_viewpoint_flag;bit(1) dynamic_position_flag;bit(5) reserved;if (effective_range_flag == 1) {unsigned int(32) effective_range;}unsigned int(32) num_availability_intervals;OMAFViewpointPositionGlobalBox( );// optionalOMAFViewpointPositionCartesianBox( );// optionalOMAFViewpointAvailabilityIntervalBox availability_intervals[ ];Box other_boxes[ ];}

An example of semantics of OMAFViewpointInfoBox is as follows:viewpoint_id is a unique identifier for the viewpoint.virtual viewpoint flag indicates whether the viewpoint is a viewpoint is a virtual viewpoint (with no capturing device placed at the position of the viewpoint) or a captured viewpoint. Information needed to generate the virtual viewpoint is signaled in the OMAFVirtualViewpointConfigBox.dynamic_position_flag indicates whether the position is static or dynamic. If this flag is set, the position of the viewpoint is provided using a timed-metadata track. Otherwise, the position is indicated by an OMAFViewpointPositionGlobalBox and/or an OMAFViewpointPositionCartesianBox in this OMAFViewpointInfoBox.effective range is the radius defining a volumetric sphere centered at the viewpoint for which the viewpoint provides rendering at a certain quality (e.g. a minimum level of quality, a quality level exceeding a known quality threshold, a guaranteed level of quality, or a level of quality approved by or deemed acceptable to the provider of the omnidirectional media content).num_availability_intervals indicates the number of time intervals during which this viewpoint is available.availability_intervals is a list of OMAFViewpointAvailabilityIntervalBox instances.

In some embodiments, when the viewpoint position in space changes over time, position information is signaled using a timed-metadata track. Timed metadata tracks are tracks within the media container (ISO BMFF) file where the samples represent dynamic metadata information. For dynamic viewpoint position information, some embodiments use a timed-metadata track with the sample entry type ‘vpps’. The sample entry for this track may be as follows.

aligned(8) class OMAFDynamicViewpointSampleEntry extendsMetadataSampleEntry(‘vpps’) {unsigned int(16) viewpoint_id;unsigned int(3) coordinate_system_type;bit(5) reserved;}

An example of semantics for the OMAFDynamicViewpointSampleEntry is as follows.viewpoint_id is the identifier of the viewpoint for which the samples of this timed-metadata track define the position.coordinate_system_type indicates the coordinate system used for defining the position of the viewpoint.

In some embodiments, samples for the viewpoint position metadata track have the following structure.

aligned(8) class OMAFViewpointPositionSample {if (coordinate_system_type == 1) {ViewpointPositionGlobalStruct( );} else if (coordinate_system_type == 2) {ViewpointPositionCartesianStruct( );}}

The sample format may be dependent on the coordinate system type defined in the sample entry of the timed-metadata track. ViewpointPositionGlobalStruct and ViewpointPositionCartesianStruct are described in further detail below.

Transport Protocol-Level Signaling of Viewpoint Information.

To identify and describe the set of viewpoints that are available in a media presentation, some embodiments include a SupplementaryProperty descriptor at the Period level. This descriptor may have a @schemeIdUri equal to “urn:mpeg:mpegL:omaf:2017:ovl” and is referred to herein as an OMAF viewpoint list (OVL) descriptor. In some embodiments, at most one OVL descriptor may be present at the Period level. The OVL descriptor may contain at least one ovp element. An ovp element has an @id attribute with a value representing a unique viewpoint identifier and may contain sub-elements with information about the viewpoint.

Table 2 lists examples of elements and attributes used for signaling viewpoint information in the MPD file for DASH clients. More details are given below.

TABLE 2Semantics of elements and attributes of example OVL descriptor.Elements and attributes forOVL descriptorUseData typeDescriptionovp1 . . . NContainer element whose attributes andelements specify the characteristics of aviewpoint.ovp@idMxs:stringA unique identifier for the viewpoint.ovp@effectiveRangeOxs:unsignedIntIndicates the range from within theviewpoint (in metres) for which theviewpoint is able to provide a certainquality of rendering (e.g. a minimum levelof quality, a quality level exceeding aknown quality threshold, a guaranteedlevel of quality, or a level of qualityapproved by or deemed acceptable to theprovider of the omnidirectional mediacontent).ovp@virtualODxs:booleanA flag indicating whether the viewpoint isa real viewpoint with a capturing deviceat the viewpoint position (value ‘false’) ora virtual viewpoint (value ‘true’) whichrequires other reference viewpointsand/or additional auxiliary information torender the scene for the viewpointposition. If not present, the default valueis ‘false’.ovp@synthesisMethodCMxs:unsignedByteIndicates which indicates the methodused to generate the virtual viewpoint.This attribute shall only be present when@virtual has the value ‘true’.ovp@refViewpointIdsCMxs:stringA comma separated list of viewpointidentifiers. The viewpoints in this list areused as references by the synthesisprocess that generates this (virtual)viewpoint. This attribute shall be presentonly when @virtual has the value ‘true’.ovp@dynamicPositionODxs:booleanA flag indicating whether the position ofthe viewpoint changes over time or isstatic. If not present, the default value is‘false’.ovp:positionCMomaf:viewpointPositionTypeA container element whose elementsidentify the position of the viewpointwithin the world of the presentation. Thiselement is present only when@dynamicPosition = ‘false’ovp:position:globalOomaf:An element whose attributes identify theviewpointGlobalPositionTypeposition of the viewpoint in terms ofgeolocation.ovp:position:Mxs:doubleLongitude value of the viewpointsglobal@longitudeposition measured in degrees.ovp:position:Mxs:doubleLatitude value of the viewpoint's positionglobal@latitudemeasured in degrees.ovp:position:Oxs:doubleAltitude value of the viewpoint's positionglobal@altitudemeasured in degrees.ovp:position:CartesianOomaf:An element whose attributes identify theviewpointCartesianPositionTypeposition of the viewpoint in terms ofCartesian coordinates.ovp:position:Mxs:intThe x-axis coordinate.cartesian@xovp:position:Mxs:intThe y-axis coordinate.cartesian@yovp:position:ODxs:intThe z-axis coordinate. If not present, thecartesian@zdefault value is zero.ovp:availability1 . . . Nomaf:viewpointAvailabilityTypeAn element whose attributes define atime interval during which the viewpoint isavailable in the presentation.ovp:availability@startMxs:unsignedLongPresentation time which defines the startof a time interval during which theviewpoint is available. The viewpoint maynot be available before this time.The value of the presentation time inseconds is the division of the value of thisattribute and the value of the @timescaleattribute.ovp:availability@endOxs:unsignedLongPresentation time which defines the endof a time interval during which theviewpoint is available. The viewpoint mayno longer be available after this time.The value of the presentation time inseconds is the division of the value of thisattribute and the value of the @timescaleattribute.If not present, the viewpoint continues tobe available until the end of thepresentation.

In Table 2 and other tables in the present disclosure, elements are bold; attributes are non-bold and preceded with an A. “M” indicates that, in the particular embodiment shown in the table, the attribute is mandatory, “0” indicates that, in the particular embodiment shown in the table, the attribute is optional, “OD” indicates that, in the particular embodiment shown in the table, the attribute is optional with a default value, “CM” indicates that, in the particular embodiment shown in the table, the attribute is conditionally mandatory.

<cminOccurs> . . . <maxOccurs> (N=unbounded).

The data types for various elements and attributes are as defined in the XML schema. An XML schema for ovp is provided in the section “XML Schema for DASH signaling,” below.

Viewpoint Position.

“Real” viewpoints correspond to 360° video cameras that are placed at different positions to capture the scene from different vantage points. In some embodiments, viewpoints may represent views from virtual positions. The virtual positions may represent points which are not associated with the location of a physical camera. The virtual positions may represent points from which synthetic content may be rendered, or points from which content captured by one or more cameras at other (real) viewpoints may be transformed, processed or combined in order to synthesize a virtual view. To provide the player with useful information on the camera setup used to capture the scene and their layout, the spatial relationship between the viewpoints in some embodiments is signaled by providing the position of each viewpoint. Position information may be represented in different ways in different embodiments. In some embodiments, global geolocation coordinates similar to the ones used by GPS systems may be used to identify to location of the camera/viewpoint. Alternatively, the Cartesian coordinate system may be used for positioning.

Media Container-Level Signaling of Viewpoint Position.

Described herein are two examples of boxes that may be used to identify the position of the viewpoint when present in the OMAFViewpointInfoBox, namely OMAFViewpointPositionGlobalBox and OMAFViewpointPositionCartesianBox. In some embodiments, these boxes are optional. An example syntax of the proposed position boxes is given below. Additional boxes may also be introduced to provide position information based on other coordinate systems.

Box Type:‘vpgl’Container:OMAFViewpointInfoBoxMandatory:NoQuantity:Zero or onealigned(8) class ViewpointPositionGlobalStruct( ) {signed int(32) longitude;signed int(32) latitude;signed int(32) altitude;}aligned(8) class OMAFViewpointPositionGlobalBox extends Box(‘vpgl’) {ViewpointPositionGlobalStruct( );}

In some embodiments, double-precision or floating point types are used for the longitude, latitude, and/or altitude values.

Box Type: ‘vpcr’Container: OMAFViewpointInfoBoxMandatory:NoQuantity: Zero or onealigned(8) class ViewpointPositionCartesianStruct( ) {signed int(32) x;signed int(32) y;signed int(32) z;}aligned(8) class OMAFViewpointPositionCartesianBox extendsBox(‘vpcr’) {ViewpointPositionCartesianStruct( );}
Transport Protocol-Level Signaling of Viewpoint Position.

To signal the position of a viewpoint, in some embodiments, an ovp:position element may be added to the ovp element. This element may include an ovp:position:global element and/or an ovp:position:cartesian element. In some embodiments, at most one of each of these elements is present within an ovp:position element. Attributes of the ovp:position:global element provide the position of the viewpoint in terms of global geolocation coordinates in units of degrees. In some embodiments, the ovp:position:global element has three attributes: @longitude, @latitude, and @altitude. In some embodiments, the @altitude attribute is optional and may not be present. Attributes of the ovp:position:catersian attribute provide the position of the viewpoint in terms of Cartesian coordinates. In some embodiments, three attributes are defined for the ovp:position:cartesian element: @x, @y, and @z, where only @z is optional.

Viewpoint Availability.

In some cases, a viewpoint may not be available for the entire duration of the media presentation. Therefore, in some embodiments, the availability of a viewpoint is signaled before the media samples for that viewpoint are processed. This enables a player to only process the samples for the tracks belonging to a specific viewpoint when the viewpoint is available.

Changes in viewpoint availability over time are illustrated inFIG.6. At time t1, only viewpoints601,602,603, and604are available. Later during the presentation at time t2, a penalty shot is given to one of the teams and most of the players are close to the goal on the right side. At that point in time, two additional viewpoints605and606are made available to the user until time t3. The time interval between t2and t3is an availability interval for viewpoints605and606. Using viewpoint availability information (e.g. received from a server), a player or streaming client operates to indicate to the user the availability of additional viewpoints at time t2during playback, e.g., using the UI shown inFIG.3orFIG.4. At the start of an availability interval, the player may present options to the user to switch to any or all of the viewpoints which are available (e.g. are newly available) during the availability interval. As shown inFIG.6, the user may be given the options to switch to viewpoints605or606starting from time t2. At the end of an availability interval, the player may remove options to switch to viewpoints which are no longer available after the availability interval ends. In some embodiments, if the user is still at one of these viewpoints when the availability interval ends (e.g. at time t3as illustrated inFIG.6), the user may be returned to the viewpoint at which the user was prior to switching to the viewpoint which is no longer available (e.g. viewpoint605or6066as illustrated inFIG.6).

In some embodiments, viewpoint availability intervals may also be signaled for virtual viewpoints. However, the availability of these viewpoints is dependent on the availability of other reference viewpoints as well as any auxiliary information used to support the rendering of the virtual viewpoint.

Media Container-Level Signaling of Viewpoint Availability.

In some embodiments, a box (OMAFViewpointAvailaibilityIntervalBox) is introduced to signal availability intervals. Zero or more instances of this box may be present in an OMAFViewpointInfoBox. When no OMAFViewpointAvailaibilityIntervalBox instances are present for a viewpoint, this indicates that the viewpoint is available for the entire duration of the presentation.

Box Type: ‘vpai’Container: OMAFViewpointInfoBoxMandatory:NoQuantity: Zero or morealigned(8) class OMAFViewpointAvailabilityIntervalBoxextends Box(‘vpai’) {bit(1) open_interval_flag;bit(7) reserved;unsigned int(64) start_time; // mandatoryunsigned int(64) end_time;}

An example of semantics for OMAFViewpointAvailabilityIntervalBox is as follows:open_inverval_flag a flag indicating the availability interval is an open interval (value 1) where the viewpoint is available from start_time until the end of the presentation, or a closed interval (value 0). If the flag is set (value 1), no end_time field is present in this box.start_time the presentation time at which the viewpoint is available (corresponds to the composition time for the first sample in the interval).end_time the presentation time after which the viewpoint is no longer available (corresponds to the composition time of the last sample in the interval).
Transport Protocol-Level Signaling of Viewpoint Availability.

In some embodiments, to signal the availability of a viewpoint in the MPD file, one or more ovp:availability elements may be added to an instance of the ovp element. This element signifies an availability period and has two attributes, @start and @end, indicating the presentation time at which the viewpoint is available and the presentation time of the last sample of the availability interval, respectively.

Virtual Viewpoints.

In some embodiments, virtual viewpoints are generated using an omnidirectional virtual view synthesis process. In some embodiments, this process makes use of one or more input (reference) viewpoints and their associated depth maps and additional metadata describing the translation vectors between the input viewpoints positions and the virtual viewpoint position. In some such embodiments, each pixel of the input omnidirectional viewpoints is mapped to a position in the virtual viewpoint sphere by mapping the pixels of the equirectangular frames of the reference viewpoints to points in 3D space and then projecting them back at the target virtual viewpoint. One such view synthesis process is described in greater detail in “Extended VSRS for 360-degree video”, MPEG121, Gwangju, Korea, January 2018, m41990, and is illustrated inFIG.8. In the example ofFIG.8, a point802has a position described by angular coordinates (φ,θ) and depth z with respect to an input viewpoint804. In the generation of a virtual viewpoint806, which is displaced from input viewpoint804by a vector (Tx, Ty, Tz), angular coordinates (φ′,θ′) are found for the point802with respect to the virtual viewpoint806. The displacement vector (Tx, Ty, Tz) may be determined based on viewpoint positions signaled in a container file, a manifest, a timed-metadata track, or otherwise.

Various techniques may be used to generate virtual viewpoint in different embodiments. Virtual viewpoint frames synthesized from different reference viewpoints may then be merged together using a blending process to generate the final equirectangular frame at the virtual viewpoint. Holes appearing the final frame due to occlusions at the reference viewpoint may be processed using an inpainting and hole filling step.

A virtual viewpoint is a non-captured viewpoint. Viewports can be rendered at a virtual viewpoint using video data from other viewpoints and/or other supplementary information. In some embodiments, the information used to render the scene from a virtual viewpoint is signaled in an OMAFVirtualViewpointConfigBox that is present in the OMAFViewpointInfoBox when the virtual_viewpoint flag is set. In some embodiments, the OMAFirtualViewpointConfigBox may be defined as follows.

Box Type: ‘vvpc’Container: OMAFViewpointInfoBoxMandatory:NoQuantity: Zero or morealigned(8) class OMAFVirtualViewpointConfigBox extends Box(‘vvpc’) {unsigned int(5) synthesis_method;unsigned int(3) num_reference_viewpoints;unsigned int(16) reference_viewpoints[ ];// optional boxes but no fields}

Examples of semantics for the OMAFVirtualViewpointConfigBox fields are given below.synthesis_method indicates which synthesis method is used to generate the virtual viewpoint. The value of synthesis_method may be an index to a listed table of view synthesis methods. For example: depth-image-based rendering, image-warping-based synthesis, etc.num_reference_viewpoints indicates the number of viewpoints that will be used as references in the synthesis of the virtual viewpoint.reference_viewpoints is a list of the viewpoint ids which are used as references when synthesizing a viewport for this viewpoint.

In another embodiment, the identifiers of the tracks containing information needed for the synthesis process are directly signalled in a virtual viewpoint configuration box, which may be implemented as follows.

aligned(8) class OMAFVirtualViewpointConfigBox extends Box(‘vvpc’) {unsigned int(5) synthesis_method;unsigned int(3) num_reference_tracks;unsigned int(16) reference_track_ids[ ];// optional boxes but no fields}

An example of semantics of the OMAFVirtualViewpointConfigBox fields for this embodiment is as follows.synthesis_method indicates which synthesis method is used to generate the virtual viewpoint. The value of synthesis_method is the index to a listed table of view synthesis methods. For example: depth-image-based rendering, image-warping-based synthesis, etc.num_reference_tracks indicates the number of tracks within the container file that will be used as references in the synthesis of the virtual viewpoint.reference_track_ids is a list of track identifiers for the tracks used in the synthesis of viewports for this viewpoint.
Signaling Viewpoint Groups.

In large-scale events such as the FIFA World Cup, a number of events may be running in parallel at different venues or locations. For example, a number of games may take place in different stadiums, possibly in different cities. In some embodiments, viewpoints can be grouped based on the geolocation of the event/venue. In some embodiments, a ViewpointGroupStruct structure is used to store information about a group of viewpoints within the media container file. An example of syntax of this structure is as follows.

aligned(8) class ViewpointGroupStruct( ) {unsigned int(8) viewpoint_group_id;signed int(32) longitude;signed int(32) latitude;unsigned int(8) num_viewpoints;unsigned int(16) viewpoint_ids[ ];string viewpoint_group_name;}

An example of semantics of the fields of ViewpointGroupStruct is as follows.viewpoint_group_id is a unique id that identifies the viewpoint group.longitude is the longitude coordinate of the geolocation of the event/venue where the viewpoints are located.latitude is the latitude coordinate of the geolocation of the event/venue where the viewpoints are located.num_viewpoints is the number of viewpoints within the viewpoint group.viewpoint_ids is an array with the ids of the viewpoints that are part of the viewpoint group.viewpoint_group name is a string with a name describing the group.

To signal the available viewpoint groups within the media container file, an OMAFViewpointGroupsBox may be added to the MetaBox in the ISO BMFF container file. An example of the syntax of an OMAFViewpointGroupsBox is given below.

Box Type: ‘ovpg’Container: MetaBoxMandatory:NoQuantity: Zero or onealigned(8) class OMAFViewpointGroupsBox extends Box(‘ovpg’) {unsigned int(8) num_viewpoint_groups;ViewpointGroupStruct viewpoint_groups[ ];}

An example of semantics for fields of this box is follows:num_viewpoint_groups is the number of viewpoint groups.viewpoint_groups is an array of ViewpointGroupStruct instances, providing information about each viewpoint group.

For transport-protocol-level signaling (e.g. DASH), to signal the viewpoint groups available in a media presentation, an ovg element may be defined and signaled in the OVL descriptor described above. The OVL descriptor may contain one or more ovg elements. An ovg element has an @id attribute with a value representing a unique viewpoint group identifier and other attributes that describe the group. Table 3 lists attributes of an example of an ovg element.

TABLE 3Semantics of the attributes of an example omaf:ovg element.Elements and attributesfor OVL descriptorUseData typeDescription@idMxs:stringContainer element whose attributes and elementsspecify the characteristics of a viewpoint.@nameOxs:stringA name for the group.@longitudeMxs:doubleThe longitude of the geolocation of the event/venue atwhich the viewpoint group is located, measured indegrees.@latitudeMxs:doubleThe latitude of the geolocation of the event/venue atwhich the viewpoint group is located, measured indegrees.@viewpointIdsMxs:stringA comma-separated list of the identifiers of theviewpoints that belong to the group.
Signaling Viewpoint Transition Effects.

Disclosed herein are the following examples of types of transitions: basic transitions, viewpoint path transitions, and auxiliary information transitions. Basic transitions are predefined transitions that can be used when switching from one viewpoint to another. An example of such a transition is the fade-to-black transition, where the rendered view gradually fades out to black then fades in to a frame from the new viewpoint. A viewpoint path transition enables content producers to specify a path that the player may follow across other viewpoints when switching to the target viewpoint. Auxiliary information transitions are transitions which rely on auxiliary information that the content producer provides in separate tracks. For example, an auxiliary track may contain depth information which can be used to render intermediate virtual views as the viewport moves from the first viewpoint to the target viewpoint.

In some embodiments, transitions may be based on the rendering of intermediate virtual views. This can be done using a view synthesis process such as depth-image-based rendering (DIBR), described for example in C. Fehn, “Depth-image-based rendering (DIBR), compression, and transmission for a new approach on 3D-TV,” in SPIE Stereoscopic Displays and Virtual Reality Systems XI, vol. 5291, May 2004, pp. 93-104. DIBR uses depth information to project the pixels in a 2D plane to their position in 3D space and re-project them back to another plane. Since no capturing devices (e.g., no 360-degree cameras) are present at these intermediate viewpoints, they are referred to herein as virtual viewpoints. The number of intermediate virtual viewpoints rendered between the source and destination viewpoints determines the smoothness of the transition and depends on the capabilities of the player/device and the availability of auxiliary information for these intermediate viewpoints.

FIG.7illustrates an embodiment using virtual viewpoints. In the example ofFIG.7, only viewpoints702,704,706,708,710,712,714,716are viewpoints with capturing devices and the remaining intermediate viewpoints (703,705,707,709,711,713,715,717) are virtual viewpoints. Other types of auxiliary information include: point cloud streams, additional reference frames from nearby viewpoints (to enhance the quality of virtual views), and occlusion information (to support the hole-filling step in the view synthesis process and improve the quality of the resulting virtual view at intermediate viewpoints). In some embodiments, point cloud streams are used to enable rendering virtual views at arbitrary viewpoint positions between the source and destination viewpoints. In some embodiments, point clouds are rendered using techniques described in Paul Rosenthal, Lars Linsen, “Image-space point cloud rendering”, in Proceedings of Computer Graphics International, pp. 136-143, 2008.

Media Container-Level Signaling of Viewpoint Transition Effects.

Some embodiments operate to signal transition effects between pairs of viewpoints in the container file as a list of boxes in a new OMAFViewpointTransitionEffectListBox which can be placed in the MetaBox at the file level. In some embodiments, at most one instance of this box is present in the MetaBox. Boxes in OMAFViewpointTransitionEffectListBox are instances of OMAFViewpointTransitionBox. An example of syntax of the two boxes is given below.

Box Type: ‘vptl’Container: MetaBoxMandatory:NoQuantity: Zero or onealigned(8) class OMAFViewpointTransitionEffectListBox extendsBox(‘vptl’) {OMAFViewpointTransitionBox transitions[ ];}Box Type: ‘vpte’Container: OMAFViewpointTransitionEffectListBoxMandatory:NoQuantity: One or morealigned(8) class OMAFViewpointTransitionEffectBox extendsBox(‘vpte’) {unsigned int(16) src_viewpoint_id; // mandatoryunsigned int(16) dst_viewpoint_id; // mandatoryunsigned int(8) transition_type; // mandatory// additional box to specify the parameters of the transition}

An example of semantics for the fields of OMAFViewpointTransitionBox is as follows:src_viewpoint_id is the id of the source viewpoint.dst_viewpoint_id is the id of the destination viewpoint.transition type is an integer identifying the type of transition. Value 0 indicates a basic transition. Value 1 indicates a viewpoint path transition. Value 2 indicates an auxiliary information transition. Remaining values are reserved for future transitions.

In some embodiments, additional boxes related to the specific type of transition and providing additional information may be present in the OMAFViewpointTransitionBox. An additional box may be defined for each of the previously defined transitions types. An OMAFBasicViewpointTransitionBox is present if the transition_type field of an OMAFViewpointTransitionBox is equal to 0. This box contains only one field, basic_transition_type, whose value indicates a specific transition from a set of pre-defined basic transitions. A OMAFPathViewpointTransitionBox is present when the transition type field of OMAFViewpointTransitionBox is equal to 1. This box contains a list of viewpoint identifiers which the player can follow when the user requests a transition to the target viewpoint. In some embodiments, a field may also be provided to indicate the speed of the transition along the path. A OMAFAuxiliaryInfoViewpointTransitionBox is present when the transition_type field of OMAFViewpointTransitionBox is equal to 2. This box contains two fields: a type field specifying the nature of the transition (e.g., generating virtual viewpoints), and an aux_track_id providing a reference to one of the tracks in the file which includes timed auxiliary information used to perform the transition effect. Examples of the syntax of the three aforementioned boxes are given below.

aligned(8) class OMAFBasicViewpointTransitionBox extendsBox(‘vptb’) {unsigned int(8) basic_transition_type;}aligned(8) class OMAFPathViewpointTransitionBox extendsBox(‘vptp’) {unsigned int(16) intermediate_viewpoints[ ];}aligned(8) class OMAFAuxiliaryInfoViewpointTransitionBox extendsBox(‘vpta’) {unsigned int(8) type;unsigned int(32) aux_track_id;}
Transport Protocol-Level Signaling (e.g. DASH) of Viewpoint Transition Effects.

Viewpoint transition effect information signaled at the container-level may also be signaled at the transport protocol level in the manifest file. If the container file contains viewpoint transition effect information, this information preferably matches the information signaled in the manifest file. In some embodiments, viewpoint transition effects are signaled within an OVL descriptor such as that described above. A transition effect between a viewpoint pair may be signaled by an ovp:transition element. In one example, this element has three attributes: @src, @dst, and @type. These attributes designate the id of the source viewpoint, the id of the destination viewpoint, and the type of the transition effect, respectively. For certain types of transition effects, the ovp:transition element may contain child elements providing additional information used by the client to render these transitions.

Table 4 lists examples of elements and attributes that may be used for signaling viewpoint transition effects in the MPD file.

TABLE 4Semantics of elements and attributes of an example ovp:transition element.Elements and attributes forovp:transitionUseData typeDescriptionovp:transition@srcMxs:stringThe identifier for the source viewpoint.ovp:transition@dstMxs:stringThe identifier for the destination viewpoint.ovp:transition@typeMxs:unsignedByteIndicates the type of the transition. Value 0 indicatesa basic transition. Value 1 indicates a path transition.Value 2 indicates an auxiliary information transition.Other values are reserved.ovp:transition:basicCMxs:vpBasicTransitionTypeThis element shall only be present ifovp:transition@type attribute is set to 0.ovp:transition:basicMxs:unsignedByteIndicates the type of the basic transition effect.@typeovp:transition:pathCMxs:vpPathTransitionTypeProvides additional information about the pathtransition effect.This element shall only be present ifovp:transition@type attribute is set to 1.ovp:transition:pathMxs:stringA comma separated list of the identifiers of@viewpointsviewpoints along the path from the source viewpointto the destination viewpoint.ovp:transition:auxCMxs:vpAuxTransitionTypeProvides additional information about the auxiliaryinformation transition effect.This element shall only be present ifovp:transition@type attribute is set to 2.ovp:transition:auxMxs:stringA comma-separated list of the ids of auxiliary@auxIdListinformation AdaptationSet elements.
Signaling Recommended Projection Format for FoV.

Different projection formats may advantageous within different FoV ranges. For example, a rectilinear projection format may work well at a field of view of 90°, but an undesirable stretching effect may be visible using rectilinear projection at larger fields of view, such as 130°. Conversely, projection formats such as a “little planet” stereographic projection or a fisheye projection format may not work well at a FoV of 90° but may present a reasonable rendering experience at a higher FoV degree.

In some embodiments, to signal the recommended projection format for a range of device field of view (FoV) values, a OMAFRecommendedProjectionListBox is provided as additional metadata information in the ‘meta’ box. This box contains one or more OMAFRecommendedProjectionBox instances. An OMAFRecommendedProjectionBox defines horizontal and vertical FoV ranges and provides a recommended projection type for the specified FOV ranges. A player or streaming client which receives this signaling may determine the size of the field of view of the device on which the player or streaming client is running (e.g. it may look up the device's FOV capabilities from a local database, or it may obtain this property through an API call to the operating system of the HMD). The player or streaming client may compare this determined field of view size to the FOV ranges defined in the OMAFRecommendedProjectionBoxes in order to determine which of the recommended projection types corresponds to the field of view of the device. The player or streaming client may then request content in the determined recommended projection format. Examples of the syntax for these boxes are provided below.

Box Type: ‘orpl’Container: MetaBoxMandatory:NoQuantity: Zero or onealigned(8) class OMAFRecommendedProjectionListBox extendsBox(‘orpl’) {OMAFRecommendedProjectionBox recommendations[ ];}Box Type: ‘orpr’Container: OMAFRecommendedProjectionListBoxMandatory:NoQuantity: One or morealigned(8) class OMAFRecommendedProjectionBox extends Box(‘orpr’) {bit(3) reserved = 0;unsigned int(5) projection_type;unsigned int(32) min_hor_fov;unsigned int(32) min_ver_fov;unsigned int(32) max_hor_fov;unsigned int(32) max_ver_fov;}

Examples of semantics of the fields OMAFRecommendedProjectionBox are as follows:projection_type indicates the type of the mapping of the projected picture onto the spherical coordinate system as specified by the OMAF standard. The value of projection_type may be the index of a list of rendering projection methods including rectilinear projection, little planet projection, equidistant projection, fisheye projection, etc.min_hor_fov and min_ver_fov provide a minimum horizontal and vertical display field of view, in units of 2−16degrees. min_hor_fov may be in the range of 0 to 360*216, inclusive. min_ver_fov may be in the range of 0 to 180*216, inclusive.max_hor_fov and max_ver_fov provide a maximum horizontal and vertical display field of view, in units of 2−16degrees. max_hor_fov may be in the range of 0 to 360*216, inclusive. max_ver_fov may be in the range of 0 to 180*216, inclusive.

In a case in which the projection format is recommended for a specific FoV, min_hor_fov is equal to max_hor_fov and min_ver_fov is equal to max_ver_fov.

In another embodiment, the content author or content provider may provide information identifying a recommended viewport for the devices with different FoV configurations with the suitable projection recommendation. Different devices with different FoVs may follow the recommended viewport and use the recommended projection format to render the 360 video content.

OMAF describes a recommended viewport information box (RcvpInfoBox) as follows.

class RcvpInfoBox extends FullBox(‘rvif’, 0, 0) {unsigned int(8) viewport_type;string viewport_description;}

The viewport_type specifies the type of the recommended viewport as listed in Table 5.

TABLE 5Recommended viewport type.ValueDescription0A recommended viewport suggested according to thecreative intent of the content author or content provider(e.g. the “director's cut)1A recommended viewport selected based onmeasurements of viewing statistics2 . . . 239Reserved (for use by future extensions of ISO/IEC23090-2)240 . . . 255Unspecified (for use by applications or externalspecifications)

In some embodiments, an additional type of recommended viewport (which may be assigned, e.g., type 2) is used based on the FOV of the rendering devices. In some embodiments, the viewport_description of RcvpInfoBox may be used to indicate the recommended rendering projection method and the corresponding rendering FOV range. In some embodiments, an optional box is added in a RcvpInfoBox based on the viewport_type to indicate the additional parameters used for the corresponding recommended type. For example, OMAFRecommendedProjectionBox may be signaled when the viewport type is associated with the FOV.

class RcvpInfoBox extends FullBox(‘rvif’, 0, 0) {unsigned int(8) viewport_type;string viewport_description;Box[ ] other_boxes; // optional}

In another embodiment, a recommended viewport may accommodate multiple recommended types, or sub-types to offer user flexible selection. For example, the viewing statistics may be further divided into the statistics by the measuring period (e.g. weekly, monthly), the geography (countries, cities) or ages (youth, adult). Table 6 illustrates a hierarchical recommendation structure that may be used in some embodiments.

TABLE 6Director's Cut2D RenderingGeneralPG-13RNC-17VR RenderingGeneralPG-13RNC-17StatisticsTime PeriodTodayWeekMonthYearGeographicNorth AmericaSouth AmericaAsiaEurope

A recursive RcvpInfoBox structure is used in some embodiments to support a hierarchical recommendation structure. The other_boxes field proposed in a RcvpInfoBox structure may include RcvpInfoBox to specify the sub type as follows.

class RcvpInfoBox extends FullBox(‘rvif’, 0, 0) {unsigned int(8) viewport_type;string viewport_description;RcvpinfoBox( ); // optional;}

A single director's cut recommended viewport may offer multiple tracks, and each may support one or more recommended rendering projection methods for a FOV range. A RcvpInfoBox example structure is illustrated below. The value of viewport_type of the primary RcvpInfoBox is 0 indicating such recommended viewport is per director's cut, and the value of viewport_type (e.g. 1) in the secondary RcvpInfoBox may indicate the track associated with this director's cut recommended viewport is recommended for the device with particular rendering FOV. One or more instances of OMAFRecommendedProjectionBox may be signaled to provide recommended projection method(s) for the corresponding FOV range.

RcvpInfoBox{viewport_type = 0; // recommended director's cutRcvpInfoBox {Viewport_type = 1; // recommeded for device FOVOMAFRecommendedProjectionBox( ); // projection method 1OMAFRecommendedProjectionBox( ); // projection method 2viewport_description;}viewport_description;}

In a DASH MPD, the SupplementalProperty and/or EssentialProperty descriptors with @schemeldUri equal to “urn:mpeg:dash:crd” may be used to provide a content recommendation description (CRD). The @value of the SupplementalProperty or EssentialProperty elements using the CRD scheme may be implemented as a comma separated list of values for CRD parameters as shown in Table 7.

TABLE 7EssentialProperty@value and/or SupplementalProperty@value attributes for an example CRD scheme.EssentialProperty@value orSupplementalProperty@valueparameterUseDescriptionrecommendation_typeMNon-negative integer in decimal representation providing the primaryrecommendation type such as director's cut or statistic measurement.sub_typeOA comma separated list of non-negative integer in decimal representationexpressing the hierarchical sub-type of the recommendations. Fordirector's cut recommendation type, the sub-type can be the film ratingtype, or rendering device capabilities (e.g. 2D display, VR display, orstereoscopic display). Multiple sub-types can be listed in a commaseparated list to indicate hierarchical sub-type structure.When not present, the Representation associated to this descriptor doesnot have a sub-type.content_desciptionOString representation providing additional recommendation information.For example, the recommended FOV range or preferred projection methodmay be included here for the director's cut recommendation for VR displayso that the end user can identify the appropriate Object associated to thisdescriptor.
XML Schema for DASH Signaling.

An example of an XML schema for DASH signaling that may be used in some embodiments is the following:

<?xml version=“1.0” encoding=“UTF-8”?><xs:schema xmlns:xs=“http://www.w3.org/2001/XMLSchema”targetNamespace=“urn:mpeg:mpegI:omaf:2017”xmlns:omaf=“urn:mpeg:mpegI:omaf:2017”elementFormDefault=“qualified”?<xs:element name=“ovp” type=“omaf:viewpointType”/><xs:element name=“ovg” type=“omaf:viewpointGroupType”/><xs:complexType name=“viewpointType”?<xs:attribute name=“id” type=“xs:string” use=“required” /><xs:attribute name=“effective_range” type=“xs:unsignedInt” use=“optional” /><xs:attribute name=“virtual” type=“xs:boolean” use=“optional” de-fault=“-false” /><xs:attribute name=“synthesisMethod” type=“xs:unsignedByte” use=“optional” /><xs:attribute name=“refViewpointIds” type=“xs:boolean” use=“optional” /><xs:attribute name=“dynamicPosition” type=“xs:boolean” use=“optional” default=“false” /><xs:element name=“position” type=“omaf:viewpointPositionType” minOccurs=“0”maxOccurs=“1”/><xs:element name=“availability” type=“omaf:viewpointAvailabilityType”maxOccurs=“unbounded” /><xs:element name=“transition” type=“omaf:vpTransitionType” minOccurs=“0”maxOccurs=“unbounded” /></xs:complexType><xs:complexType name=“viewpointPositionType”><xs:element name=“global” type=“omaf:viewpointGlobalPositionType” maxOccurs=“1” /><xs:element name=“cartesian” type=“omaf:viewpointCartesianPositionType” maxOccurs=“1” /></xs:complexType><xs:complexType name=“viewpointGlobalPositionType” use=“optional” maxOccurs=“1”><xs:attribute name=“longitude” type=“xs:double” use=“required” /><xs:attribute name=“latitude” type=“xs:double” use=“required” /><xs:attribute name=“altitude” type=“xs:double” use=“optional” default=“0” /></xs:complexType><xs:complexType name=“viewpointCartesianPositionType” use=“optional” maxOccurs=“1”><xs:attribute name=“x” type=“xs:int” use=“required” /><xs:attribute name=“y” type=“xs:int” use=“required” /><xs:attribute name=“z” type=“xs:int” use=“optional” default=“0” /></xs:complexType><xs:complexType name=“viewpointAvailabilityType” use=“optional” maxOccurs=“unbounded”><xs:attribute name=“start” type=“xs:unsignedLong” use=“required” /><xs:attribute name=“end” type=“xs:unsignedLong” use=“optional” /></xs:complexType><xs:complexType name=“vpTransitionType” use=“optional” maxOccurs=“unbounded”><xs:attribute name=“src” type=“xs:string” use=“required” /><xs:attribute name=“dst” type=“xs:string” use=“required” /><xs:attribute name=“type” type=“xs:unsignedByte” use=“required” /><xs:element name=“omaf:vpBasicTransitionType” use=“optional” maxOccurs=“1” /><xs:element name=“omaf:vpPathTransitionType” use=“optional” maxOccurs=“1” /><xs:element name=“omaf:vpAuxTransitionType” use=“optional” maxOccurs=“1” /></xs:complexType><xs:complexType name=“vpBasicTransitionType”><xs:attribute name=“type” type=“unsignedByte” use=“required” /></xs:complexType><xs:complextType name=“vpPathTransitionType”><xs:attribute name=“viewpoints” type=“xs:string” use=“required” /></xs:complextType><xs:complexType name=“vpAuxTransitionType”><xs:attribute name=“auxIdList” type=“xs:string” use=“required” /></xs:complexType><xs:complexType name=“viewpointGroupType”><xs:attribute name=“id” type=“xs:string” use=“required” /><xs:attribute name=“name” type=“xs:string” use=“optional” /><xs:attribute name=“longitude” type=“xs:double” use=“required” /><xs:attribute name=“latitude” type=“xs:double” use=“required” /><xs:attribute name=“viewpointIds” type=“xs:string” use=“required” /></xs:complexType></xs:schema>
Additional Embodiments.

In some embodiments, a method includes: receiving at least first 360-degree video data representing a view from a first viewpoint and second 360-degree video data representing a view from a second viewpoint; and generating a container file (e.g. an ISO Base Media File Format file) for at least the first video data and the second video data. In the container file: the first video data is organized into a first set of tracks and the second video data is organized in a second set of tracks; each of the tracks in the first set of tracks includes a first track-group identifier associated with the first viewpoint; and each of the tracks in the second set of tracks includes a second track-group identifier associated with the second viewpoint.

In some such embodiments, each of the tracks in the first set of tracks includes a respective instance of a viewpoint-group-type box that contains the first track-group identifier; and each of the tracks in the second set of tracks includes a respective instance of a viewpoint-group-type box that contains the second track-group identifier.

In some embodiments where the container file is organized in a hierarchical box structure, and the container file includes a viewpoint-list box that identifies at least a first viewpoint-information box and a second viewpoint-information box, the first viewpoint-information box includes at least (i) the first track-group identifier and (ii) an indication of time intervals for which video from the first viewpoint is available; and the second viewpoint-information box includes at least (i) the second track-group identifier and (ii) an indication of time intervals for which video from the second viewpoint is available. The indications of time intervals may be lists of instances of respective viewpoint availability interval boxes.

In some embodiments, where the container file is organized in a hierarchical box structure, and where the container file includes a viewpoint-list box identifying at least a first viewpoint-information box and a second viewpoint-information box: the first viewpoint-information box includes at least (i) the first track-group identifier and (ii) an indication of a position of the first viewpoint; and the second viewpoint-information box includes at least (i) the second track-group identifier and (ii) an indication of a position of the second viewpoint. The indication of position may include Cartesian coordinates or latitude and longitude coordinates.

In some embodiments where the container file is organized in a hierarchical box structure, and where the container file includes a viewpoint-list box identifying at least a first viewpoint-information box and a second viewpoint-information box: the first viewpoint-information box includes at least (i) the first track-group identifier and (ii) an indication of an effective range of the first viewpoint; and the second viewpoint-information box includes at least (i) the second track-group identifier and (ii) an indication of an effective range of the second viewpoint.

In some embodiments where the container file is organized in a hierarchical box structure, and the container file includes a transition-effect-list box identifying at least one transition-effect box, each transition-effect box includes: an identifier of a source viewpoint; an identifier of a destination viewpoint; and an identifier of a transition type. The identifier of the transition type may identify a basic transition or a viewpoint path transition. Where the identifier of the transition type identifies a path-viewpoint-transition box, the path-viewpoint-transition box may include a list of viewpoint identifiers. Where the identifier of the transition type identifies an auxiliary-information-viewpoint-transition box, the auxiliary-information-viewpoint-transition box may include a track identifier.

In some embodiments, where the container file is organized in a hierarchical box structure including a meta box: the meta box identifies at least one recommended-projection-list box; and each recommended-projection-list box includes information identifying (i) a projection type and (ii) a corresponding field-of-view range. The information identifying the corresponding field-of-view range may include: a minimum horizontal field of view angle; a maximum horizontal field of view angle; a minimum vertical field of view angle; and a maximum vertical field of view angle.

Some embodiments include anon-transitory computer storage medium storing a container file generated according to any of the methods described herein.

In some embodiments a method includes: receiving at least first 360-degree video data representing a view from a first viewpoint and second 360-degree video data representing a view from a second viewpoint; and generating a manifest, such as an MPEG-DASH MPD. In the manifest: at least one stream in a first set of streams is identified, each stream in the first set representing at least a portion of the first video data; at least one stream in a second set of streams is identified, each stream in the second set representing at least a portion of the second video data; each of the streams in the first set is associated in the manifest with a first viewpoint identifier and; each of the streams in the second set is associated in the manifest with a second viewpoint identifier.

In some such embodiments, each of the streams in the first set is associated in the manifest with a respective adaptation set that has the first viewpoint identifier as an attribute; and each of the streams in the second set is associated in the manifest with a respective adaptation set that has the second viewpoint identifier as an attribute. The attribute may be a @viewpoint_id attribute.

In some embodiments, each of the streams in the first set is associated in the manifest with a respective adaptation set that has the first viewpoint identifier in a first descriptor; and each of the streams in the second set is associated in the manifest with a respective adaptation set that has the second viewpoint identifier in a second descriptor. The first and second descriptors may be SupplementalProperty descriptors.

In some embodiments, the manifest includes an attribute indicating an effective range for each of the viewpoints. In some embodiments, the manifest includes an attribute indicating a position for each of the viewpoints. The attribute indicating position may include cartesian coordinates or latitude and longitude coordinates. In some embodiments, the manifest includes, for each viewpoint, information indicating at least one time period during which video for the respective viewpoint is available.

In some embodiments, the first video data and second video data are received in a container file (such as an ISO Base Media File Format file) in which: the first video data is organized into a first set of tracks and the second video data is organized in a second set of tracks; each of the tracks in the first set of tracks includes a first track-group identifier associated with the first viewpoint; and each of the tracks in the second set of tracks includes a second track-group identifier associated with the second viewpoint. The viewpoint identifiers used in the manifest are equal to the respective track-group identifiers in the container file.

In some embodiments, a method includes: receiving a manifest identifying a plurality of 360-degree video streams, the manifest including, for each identified stream, information identifying a viewpoint location of the respective stream; retrieving and displaying a first video stream identified in the manifest; and overlaying on the display of the first video stream a user interface element indicating the viewpoint location of a second video stream identified in the manifest. In some embodiments, the method includes, in response to selection of the user interface element, retrieving and displaying the second video stream.

In some embodiments, where the manifest further includes information identifying an effective range of at least one of the identified streams, the method further includes displaying an indication of the effective range. In some embodiments, where the manifest further includes information identifying a period of availability of the second video stream, the user interface element is displayed only during the period of availability.

In some embodiments, the manifest includes information identifying a transition type for a transition from the first video stream to the second video stream. In response to selection of the user interface element, the method includes: presenting a transition having the identified transition type; and retrieving and displaying the second video stream, the second video stream being displayed after the presentation of the transition.

In some embodiments, where the manifest further includes information identifying a location of at least one virtual viewpoint, the method further includes, in response to selection of the virtual viewpoint, synthesizing a view from the virtual viewpoint and displaying the synthesized view.

In some embodiments, a method includes: receiving a manifest (an MPEG-DASH MPD) identifying a plurality of 360-degree video streams, the manifest including information identifying a respective projection format of each of the video streams, the manifest further including information identifying a respective range of field-of-view sizes for each of the projection formats; determining a field-of-view size for display; selecting at least one of the video streams such that the determined field-of-view size is within the identified range of field-of-view sizes for the projection format of the selected video streams; and retrieving at least one of the selected video streams and displaying the retrieved video stream with the determined field-of-view size.

Further embodiments include a system comprising a processor and a non-transitory computer-readable medium storing instructions operative when executed on the processor to perform any of the methods described herein.

Note that various hardware elements of one or more of the described embodiments are referred to as “modules” that carry out (i.e., perform, execute, and the like) various functions that are described herein in connection with the respective modules. As used herein, a module includes hardware (e.g., one or more processors, one or more microprocessors, one or more microcontrollers, one or more microchips, one or more application-specific integrated circuits (ASICs), one or more field programmable gate arrays (FPGAs), one or more memory devices) deemed suitable by those of skill in the relevant art for a given implementation. Each described module may also include instructions executable for carrying out the one or more functions described as being carried out by the respective module, and it is noted that those instructions could take the form of or include hardware (i.e., hardwired) instructions, firmware instructions, software instructions, and/or the like, and may be stored in any suitable non-transitory computer-readable medium or media, such as commonly referred to as RAM, ROM, etc.

Although features and elements are described above in particular combinations, one of ordinary skill in the art will appreciate that each feature or element can be used alone or in any combination with the other features and elements. In addition, the methods described herein may be implemented in a computer program, software, or firmware incorporated in a computer-readable medium for execution by a computer or processor. Examples of computer-readable storage media include, but are not limited to, a read only memory (ROM), a random access memory (RAM), a register, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROM disks, and digital versatile disks (DVDs). A processor in association with software may be used to implement a radio frequency transceiver for use in a WTRU, UE, terminal, base station, RNC, or any host computer.