Patent Description:
A technique is proposed in which a wide angle of view video such as an entire celestial sphere video is distributed and displayed on a display device such as a head mounted display (HMD) (see Patent Document <NUM>).

<CIT> discloses a display device for displaying a first image on which a second image displayable as a moving image is superimposed. The display device includes determination means for determining whether the second image is browsed, and display switching means that switches whether to reproduce the second image as a moving image or as a still image in accordance with the determination result of the determination means.

<CIT> discloses an image processing apparatus that permits a user to readily recognize a current state of an angle of view. When an angle of view is switched from a wide-angle to a narrow-angle, range display is performed for a predetermined time prior to the switching. Range display includes a narrow-angle image frame formed with lines indicating a border of an image displayed for a predetermined time prior to the switching. After the predetermined time, a narrow-angle image is displayed and the range display is no longer displayed. The range display allows a user to readily recognize that the wide-angle image has been switched to the narrow-angle image by recognizing to what area in a pre-switching wide angle of view image the post-switching narrow-angle image corresponds.

<CIT> discloses a system that facilitates the automatic video recording of events which include a moving target. The system deploys at least two cameras, one (master camera) operating in wide-angle mode to cover a large field of view or the entire scene. At least one second camera operates in a pan/tilt/zoom-in mode relative to the master. A computer analyzes the video input from the first camera and "tracks" the target, using image recognition. The coordinates of the tracked target are transmitted to a second computer to control the second camera(s) to record the event at close-up angles.

The distribution of the wide angle of view videos, by the way, enables distribution of videos with realistic feelings. However, depending on a line condition in presentation or a live performance, there are scenes for switching between a wide angle of view video and a partial video (hereinafter, also referred to as a planar video) that is a part of the wide angle of view video.

In the technique according to Patent Document <NUM>, in switching from a wide angle of view video to a planar video, it is not explicitly displayed at which part in the wide angle of view video the planar video is switched to.

For this reason, in a case where the wide angle of view video is switched to the planar video, a user who is a viewer cannot recognize that the planar video displayed after switching derives from which part in the wide angle of view video that the user had been viewing until then. Hence, there has been a sense of incongruity in viewing, in some cases.

The present disclosure has been made in view of such circumstances, and in particular, has an object to enable viewing without a sense of incongruity in switching between a wide angle of view video and a planar video that is a part of the wide angle of view video.

According to a first aspect, the present invention provides a reproducing device in accordance with independent claim <NUM>. According to a second aspect, the present invention provides a reproducing method in accordance with independent claim <NUM>. According to a third aspect, the present invention provides a program in accordance with independent claim <NUM>. Further aspects are set forth in the dependent claims, the drawings and the following description.

It is to be noted that in the present specification and the drawings, components having substantially the same functional configuration are designated by the same reference numerals to omit duplicate description.

Hereinafter, modes for carrying out the present technology will be described. The description will be given in the following order.

The present disclosure intends to switch between a wide angle of view video and a planar video that is a part of the wide angle of view video without a sense of incongruity for viewing.

Therefore, first, the outline of the present disclosure will be described.

A user wears a head mounted display (HMD) on the head, and a wide angle of view video is displayed according to movements of the user's head. Consideration is given to a configuration in which, for example, as shown in the left part of <FIG>, a wide angle of view video P1 including a relayed video of football game is displayed on a display unit D of the head mounted display (HMD).

In such a configuration, the wide angle of view video P1 in the left part of <FIG> is, in some cases, switched to a planar video P2 as shown in the right part of <FIG> to enlarge and display an area F' where two players including a player keeping a ball and a player of the opponent team who is marking the player are present in the wide angle of view video P1.

It is to be noted that the wide angle of view video P1 is a video that has been captured by a first camera for capturing a wide angle of view video, and the planar video P2 is a video that has an angle of view within the wide angle of view video P1 and that has been captured by a second camera, which is different from the first camera and which captures a video having a narrower angle of view than that of the video P1.

However, in a case where the wide angle of view video P1 as shown in the left part of <FIG> is suddenly switched to the planar video P2 as shown in the right part of <FIG> at a predetermined timing, the user who views the video displayed on the display unit D can recognize that the video that the user is viewing has been switched.

However, the user may not be able to recognize what positional relationship the planar video P2 to be displayed after switching has with respect to an immediately previous the wide angle of view video P1, and may feel a sense of incongruity.

Therefore, in the present disclosure, as shown in the central part of <FIG>, an area corresponding to an area F', which becomes the planar video P2 to be switched to next, is displayed as a frame F in the wide angle of view video P1.

Then, in the present disclosure, the wide angle of view video P1, to which the frame F has been added as shown in the central part of <FIG> to the wide angle of view video P1 in the left part of <FIG>, is displayed, and is then switched to the planar video P2 shown in the right part of <FIG>.

In this manner, at the timing immediately before the wide angle of view video P1 is switched to the planar video P2 to be displayed, a position corresponding to the planar video P2 is displayed as the frame F in the wide angle of view video P1. This configuration makes the user recognize that the planar video P2 to be switched to at the next timing corresponds to a video at which position in the wide angle of view video P1 immediately before switching, and then switches the video.

Consequently, the user viewing the display unit D recognizes the frame F displayed in the wide angle of view video P1, so as to be able to view the planar video P2 after switching while recognizing the positional relationship between the planar video P2 to be switched to next and the wide angle of view video P1 immediately before the switching.

As a result, the user can continue viewing without a sense of incongruity when the wide angle of view video P1 is switched to the planar video P2.

Next, a configuration example of a reproducing system in a first embodiment of the present disclosure will be described with reference to a block diagram of <FIG>.

In switching between a wide angle of view video and a planar video that is a partial video in the wide angle of view video, the reproducing system according to the present disclosure displays the position of the planar video to be switched to next as a frame in the wide angle of view video, and then switches to the planar video, so as to switch between the videos without a sense of incongruity.

A reproducing system <NUM> of <FIG> includes an entire celestial sphere camera <NUM>, a planar camera <NUM>, a personal computer (PC) <NUM>, a server <NUM>, a reproducing device <NUM>, a head mounted display (HMD) <NUM>, and encoding devices <NUM> and <NUM>.

The reproducing system <NUM> of <FIG> is a system that relays and distributes contents, for example, sports, plays, and the like from a venue, and the entire celestial sphere camera <NUM>, the planar camera <NUM>, and the personal computer (PC) <NUM> are configured as venue facilities of a sport facility, a theater, or the like.

In addition, the reproducing device <NUM> and the HMD <NUM> are configured as facilities on a user's side that receives content distribution.

Then, the server <NUM> acquires a video as a content from the venue facilities via a network, and distributes the video to the facilities on the user's side.

The entire celestial sphere camera <NUM> is a camera installed in a fixed state, and captures an entire celestial sphere video including a wide angle of view video having a range of <NUM> degrees, a range of <NUM> degrees, or the like with respect to the horizontal direction and the vertical direction, and outputs the video to the encoding device <NUM>. The encoding device <NUM> encodes the entire celestial sphere video, and outputs the entire celestial sphere video that has been encoded to the server <NUM>. It is to be noted that the entire celestial sphere camera <NUM> may have a function of the encoding device <NUM> so as to encode the captured entire celestial sphere video, and may output the entire celestial sphere video that has been encoded to the server <NUM>. Further, the entire celestial sphere video that has been captured by the entire celestial sphere camera <NUM> is encoded by the encoding device <NUM>, and is then output to the server <NUM>. However, in the following description, the process of encoding, by the encoding device <NUM>, the entire celestial sphere video is omitted as necessary, and, for example, will be described as "the entire celestial sphere camera <NUM> outputs the entire celestial sphere video that has been captured to the server <NUM>". However, only the expression is omitted, and in reality, the entire celestial sphere video that has been captured by the entire celestial sphere camera <NUM> is encoded by the encoding device <NUM>, and is then output to the server <NUM>.

The planar camera <NUM> is a camera provided on a camera platform 12a, and captures a partial range of a range captured by the entire celestial sphere camera <NUM> as a planar video, for example, with a horizontal-vertical ratio of <NUM>: <NUM>, while changing a video-capturing position and pan-tilt using the camera platform 12a, and outputs the planar video to the encoding device <NUM>. The encoding device <NUM> encodes the entire celestial sphere video, and outputs the entire celestial sphere video that has been encoded video to the server <NUM>. It is to be noted that the planar camera <NUM> may have the function of the encoding device <NUM> to encode the planar video that has been captured, and may output the planar video that has been encoded to the server <NUM>. Further, the planar video that has been captured by the planar camera <NUM> is encoded by the encoding device <NUM>, and is then output to the server <NUM>. However, in the following description, the process of encoding, by the encoding device <NUM>, the planar video is omitted as necessary, and will be simply described as "the planar camera <NUM> outputs the planar video that has been captured to the server <NUM>". However, only the expression is omitted, and in reality, the planar video that has been captured by the planar camera <NUM> is encoded by the encoding device <NUM>, and is then output to the server <NUM>.

Further, in the present disclosure, the description will be given with regard to an example in which an entire celestial sphere video and a planar video including a partial video of the entire celestial sphere video are respectively captured by the entire celestial sphere camera <NUM> and the planar camera <NUM>. However, a configuration other than the entire celestial sphere camera <NUM> and the planar camera <NUM> may be used, as long as such a configuration is capable of capturing a wide angle of view video and a planar video including a partial video that is a part of the entire celestial sphere video.

The camera platform 12a outputs information indicating the video-capturing position and a pan-tilt direction of the planar camera <NUM>, as pan-tilt information, to the PC <NUM>.

The PC <NUM> acquires the pan-tilt information supplied from the camera platform 12a of the planar camera <NUM>, generates metadata on the basis of the pan-tilt information, and outputs the metadata to the server <NUM>.

More specifically, the PC <NUM> includes a controller <NUM>, a data controller <NUM>, and a delay processing unit <NUM>.

The controller <NUM> includes a processor, a memory, and the like, and controls the entire operation of the PC <NUM>, and controls the data controller <NUM> and the delay processing unit <NUM>.

The data controller <NUM> generates metadata including the pan-tilt information on the basis of the pan-tilt information supplied from the camera platform 12a, and outputs the metadata to the delay processing unit <NUM>.

The delay processing unit <NUM> temporarily stores the metadata supplied from the data controller <NUM>, delays by a predetermined period so as to adjust the timing to match the timing when the planar video that has been captured by the planar camera <NUM> is output to the server <NUM>, and outputs the metadata to the server <NUM>.

That is, the planar video output from the planar camera <NUM> to the server <NUM> and the metadata including the pan-tilt information of the planar camera <NUM>, which is capturing the planar video, are supplied to the server <NUM> at the same timing.

The server <NUM> receives and stores the entire celestial sphere video supplied from the entire celestial sphere camera <NUM>, the planar video supplied from the planar camera <NUM>, and the metadata supplied from the PC <NUM>, and in addition, outputs the entire celestial sphere video, the planar video, and the metadata to the reproducing device <NUM>.

More specifically, the server <NUM> includes a controller <NUM>, a metadata storage unit <NUM>, a planar video storage unit <NUM>, and an entire celestial sphere video storage unit <NUM>.

The controller <NUM> includes a processor, a memory, and the like, and controls the entire operation of the server <NUM>.

The controller <NUM> controls the operations of the metadata storage unit <NUM>, the planar video storage unit <NUM>, and the entire celestial sphere video storage unit <NUM>.

When the metadata is supplied from the PC <NUM>, the controller <NUM> acquires the metadata, and causes the metadata storage unit <NUM> to store the metadata. Further, the controller <NUM> reads the metadata stored in the metadata storage unit <NUM>, and transmits the metadata to the reproducing device <NUM>.

When the planar video is supplied from the planar camera <NUM>, the controller <NUM> acquires the planar video, and causes the planar video storage unit <NUM> to store the planar video. Further, the controller <NUM> reads the planar video stored in the planar video storage unit <NUM>, and transmits the planar video to the reproducing device <NUM>.

When the entire celestial sphere video is supplied from the entire celestial sphere camera <NUM>, the controller <NUM> acquires the entire celestial sphere video, and causes the entire celestial sphere video storage unit <NUM> to store the entire celestial sphere video. Further, the controller <NUM> reads the entire celestial sphere video stored in the entire celestial sphere video storage unit <NUM>, and transmits the entire celestial sphere video to the reproducing device <NUM>.

The reproducing device <NUM> acquires the entire celestial sphere video, the planar video, and the metadata supplied from the server <NUM>, performs rendering on the basis of head tracking information from the HMD <NUM> worn by the user, and outputs the videos to the HMD <NUM> for display.

The reproducing device <NUM> causes the HMD <NUM> to display the entire celestial sphere video in a case where the line quality is good, and causes the HMD <NUM> to display the planar video in a case where the line quality is degraded, depending on the line quality of the communication line with the server <NUM>.

In such a situation, in switching from the entire celestial sphere video to the planar video, the reproducing device <NUM> displays the range of the planar video to be switched to next as a frame in the entire celestial sphere video, so as to display that the planar video to be switched to next is at which position in the entire celestial sphere video, and then switches to the planar video and displays the planar video.

By switching from the entire celestial sphere video to the planar video in this manner, the user can view the video switched from the entire celestial sphere video to the planar video without a sense of incongruity.

It is to be noted that when the planar video is switched to the entire celestial sphere video, the frame indicating the range of the planar video is displayed in the entire celestial sphere video for a predetermined period at the timing of switching to the entire celestial sphere video.

By switching from the planar video to the entire celestial sphere video in this manner, the user can view the video switched from the planar video to the entire celestial sphere video without a sense of incongruity.

More specifically, the reproducing device <NUM> includes a connection controller <NUM>, a switching determination unit <NUM>, a decoding unit <NUM>, a meta parser unit <NUM>, a video controller <NUM>, a user interface processing unit <NUM>, and a rendering unit <NUM>.

The connection controller <NUM> acquires the entire celestial sphere video and the planar video from the server <NUM>, and outputs the videos to the decoding unit <NUM>.

In such a situation, the connection controller <NUM> outputs to the switching determination unit <NUM>, information indicating the line quality of the line for acquiring the entire celestial sphere video from the server <NUM>, for example, information regarding the line speed, information regarding a remaining amount of buffer data of the entire celestial sphere video and the planar video that are being received, and the like.

On the basis of the information indicating the line quality supplied from the connection controller <NUM>, the switching determination unit <NUM> supplies the video controller <NUM> with switching determination information for instructing displaying of the entire celestial sphere video on the HMD <NUM>, in a case where, for example, the line quality such as the line speed is faster than a predetermined line speed and is better than a predetermined quality. Further, in a case where the information indicating the line quality is the information regarding the remaining amount of the buffer data of the entire celestial sphere video and the planar video that are being received, when the remaining amount of the buffer data is equal to or more than a predetermined threshold, the switching determination unit <NUM> may determine that the line quality is better than the predetermined quality and may instruct displaying of the entire celestial sphere video on the HMD <NUM>.

In addition, in a case where the line quality is not better than the predetermined quality, the switching determination unit <NUM> supplies the video controller <NUM> with switching determination information for instructing displaying of the planar video on the HMD <NUM>.

The decoding unit <NUM> decodes the entire celestial sphere video and the planar video, and outputs the videos to the video controller <NUM>.

The meta parser unit <NUM> acquires the metadata supplied from the server <NUM>, and outputs the metadata to the video controller <NUM>.

The user interface processing unit <NUM> acquires the head tracking information supplied from the HMD <NUM>, and outputs the head tracking information to the video controller <NUM>.

The head tracking information is, for example, information regarding the position and the direction of the HMD <NUM> worn on the user's head, and is information indicating at which position and in which direction the user is facing.

The video controller <NUM> determines a video on the basis of the switching determination information supplied from the switching determination unit <NUM>, the entire celestial sphere video and the planar video supplied from the decoding unit <NUM>, the metadata supplied from the meta parser unit <NUM>, and the head tracking information supplied from the user interface processing unit <NUM>, and outputs the determined video to the rendering unit <NUM>.

The rendering unit <NUM> renders the video supplied from the video controller <NUM>, outputs the video that has been rendered to the HMD <NUM> for display.

The HMD <NUM> is a display device worn on the user's head, and displays the video supplied from the reproducing device <NUM>.

More specifically, the HMD <NUM> includes a controller <NUM>, a head tracking information acquisition unit <NUM>, and a display unit <NUM>.

The controller <NUM> includes a processor and a memory, and controls the entire operation of the HMD <NUM>.

The head tracking information acquisition unit <NUM> includes, for example, a motion sensor or the like, acquires information regarding the position, the direction of the user's head, and the like as the head tracking information, and transmits the head tracking information to the reproducing device <NUM>.

The display unit <NUM> includes a liquid crystal display (LCD) and an organic electro luminescence (EL), is provided in a viewing direction of the user wearing the HMD <NUM>, and displays the video supplied from the reproducing device <NUM>.

Next, an entire celestial sphere video capture process by the entire celestial sphere camera <NUM> will be described with reference to a flowchart of <FIG>.

In step S11, the entire celestial sphere camera <NUM> captures an entire celestial sphere video, and outputs the entire celestial sphere video to the encoding device <NUM>.

In step S12, the encoding device <NUM> encodes the entire celestial sphere video that has been supplied from the entire celestial sphere camera <NUM>, and transmits the entire celestial sphere video that has been encoded to the server <NUM>.

In step S13, the entire celestial sphere camera <NUM> determines whether or not the end of the process has been instructed. In a case where the end has not been instructed, the process returns to step S11, and the subsequent processes are repeated.

Then, in step S13, in a case where the end of the process has been instructed, the process ends.

By the above process, the entire celestial sphere video is captured by the entire celestial sphere camera <NUM>, is encoded by the encoding device <NUM>, and is continuously transmitted to the server <NUM>.

Next, a planar video capture process by the planar camera <NUM> will be described with reference to a flowchart of <FIG>.

In step S31, the planar camera <NUM> captures a planar video including a partial video of the entire celestial sphere video captured by the entire celestial sphere camera <NUM>, and outputs the planar video to the encoding device <NUM>.

In step S32, the camera platform 12a of the planar camera <NUM> acquires the pan-tilt information including the information regarding the video-capturing position and pan-tilt that is a video-capturing direction of the planar camera <NUM>.

In step S33, the encoding device <NUM> encodes the planar video that has been captured, and outputs the planar video that has been encoded to the server <NUM>.

In step S34, the camera platform 12a outputs the pan-tilt information to the PC <NUM>.

In step S35, the planar camera <NUM> determines whether or not the end of the process has been instructed. In a case were the end has not been instructed, the process returns to step S31, and the subsequent processes are repeated.

Then, in step S35, in a case where the end of the process has been instructed, the process ends.

By the above process, the planar video is captured by the planar camera <NUM>, is encoded by the encoding device <NUM>, and is transmitted to the server <NUM>, and in addition, the pan-tilt information including the video-capturing position and the information regarding the pan-tilt of the planar camera <NUM> at the timing when the planar video is captured is output to the PC13.

Next, the metadata generation process by the PC <NUM> in <FIG> will be described with reference to a flowchart of <FIG>.

In step S51, the data controller <NUM> acquires the pan-tilt information supplied from the camera platform 12a of the planar camera <NUM>.

In step S52, the data controller <NUM> generates metadata on the basis of the pan-tilt information that has been acquired, and outputs the metadata to the delay processing unit <NUM>.

In step S53, the delay processing unit <NUM> acquires the metadata supplied from the data controller <NUM>, stores the metadata until the timing when the corresponding planar video in the planar camera <NUM> is transmitted to the server <NUM>, and delays the output.

In step S54, the delay processing unit <NUM> outputs the metadata to the server <NUM> at the timing when the planar video corresponding to the pan-tilt information included in the metadata that is stored is transmitted to the server <NUM>.

In step S55, the controller <NUM> determines whether or not the end of the process has been instructed. In a case where the end has not been instructed, the process returns to step S51, and the subsequent processes are repeated.

Then, in step S55, in a case where the end of the process has been instructed, the process ends.

By the above process, the metadata based on the pan-tilt information including the video-capturing position and the information regarding the pan-tilt at the timing when the planar video is captured by the planar camera <NUM> is generated, and in addition, the metadata is output to the server <NUM> at the timing when the corresponding planar video is transmitted to the server <NUM>.

As a result, the planar video and the metadata including the corresponding pan-tilt information are adjusted at the same timing to be transmitted to the server <NUM>.

Next, a video transmission process by the server <NUM> of <FIG> will be described with reference to a flowchart of <FIG>.

In step S71, the controller <NUM> of the server <NUM> acquires the entire celestial sphere video that has been captured by the entire celestial sphere camera <NUM> and that has been encoded by the encoding device <NUM>, and causes the entire celestial sphere video storage unit <NUM> to store the entire celestial sphere video.

In step S72, the controller <NUM> reads the entire celestial sphere video that is stored in the entire celestial sphere video storage unit <NUM>, and outputs the entire celestial sphere video to the reproducing device <NUM>.

In step S73, the controller <NUM> acquires a planar video that has been captured by the planar camera <NUM> and that has been encoded by the encoding device <NUM>, and causes the planar video storage unit <NUM> to store the planar video.

In step S74, the controller <NUM> reads the planar video that is stored in the planar video storage unit <NUM>, and outputs the planar video to the reproducing device <NUM>.

In step S75, the controller <NUM> acquires the metadata supplied from the PC <NUM>, and causes the metadata storage unit <NUM> to store the metadata.

In step S76, the controller <NUM> reads the metadata that is stored in the metadata storage unit <NUM>, and outputs the metadata to the reproducing device <NUM>.

In step S77, the controller <NUM> determines whether or not the end of the process has been instructed. In a case where the end has not been instructed, the process returns to step S71, and the subsequent processes are repeated.

Then, in step S77, in a case where the end of the process has been instructed, the process ends.

By the above process, the entire celestial sphere video that has been captured by the entire celestial sphere camera <NUM> and that has been encoded by the encoding device <NUM>, the planar video that has been captured by the planar camera <NUM> and that has been encoded by the encoding device <NUM>, and the metadata that has been generated by the PC <NUM> are respectively transmitted to the reproducing device <NUM>.

In such a situation, regarding the entire celestial sphere video, the planar video, and the metadata, respectively corresponding pieces of information at the same timing are supplied to the reproducing device <NUM>.

Therefore, the processes of steps S71 and S72, the processes of steps S73 and S74, and the processes of steps S75 and S76 in the flowchart of <FIG> are substantially parallel processes.

Next, a reproduction process by the reproducing device <NUM> of <FIG> will be described with reference to the flowchart of <FIG>.

In step Sill, the connection controller <NUM> acquires the entire celestial sphere video and the planar video that are encoded from the server <NUM>, and outputs the videos to the decoding unit <NUM>.

In step S112, the decoding unit <NUM> decodes the entire celestial sphere video and the planar video that are encoded and that have been supplied from the connection controller <NUM>, and outputs the videos that have been decoded to the video controller <NUM>. That is, the video controller <NUM> acquires the entire celestial sphere video and the planar video that have been decoded.

In step S113, the meta parser unit <NUM> acquires the metadata supplied from the server <NUM>, and outputs the metadata to the video controller <NUM>. That is, the video controller <NUM> acquires the metadata.

In step S114, the video controller <NUM> acquires head tracking information from the HMD <NUM>.

In step S115, the connection controller <NUM> outputs to the switching determination unit <NUM>, information regarding, for example, the line speed and the remaining amount of the buffer data of the entire celestial sphere video and the planar video that are being received, as the information regarding the line quality of the entire celestial sphere video from the server <NUM>. That is, the switching determination unit <NUM> acquires the information regarding the line speed and the remaining amount of the buffer data of the videos, as the information regarding the line quality.

In step S116, the switching determination unit <NUM> determines whether or not the line speed is faster than a predetermined speed and whether or not the remaining amount of the buffer data is equal to or larger than a predetermined threshold on the basis of the information regarding the line quality including the line speed or the remaining amount of the buffer data of the entire celestial sphere video and the planar video that are being received, so as to determine whether or not the line quality is sufficient to be capable of reproducing the entire celestial sphere video, that is, whether or not to reproduce the entire celestial sphere video.

In a case where it is determined in step S116 that the line quality is sufficient to reproduce the entire celestial sphere video, the switching determination unit <NUM> outputs the switching determination information for instructing reproduction of the entire celestial sphere video to the video controller <NUM>. The process proceeds to step S117.

In step S117, the video controller <NUM> selects the entire celestial sphere video that has been decoded on the basis of the switching determination information for instructing the reproduction of the entire celestial sphere video and the head tracking information, and outputs the entire celestial sphere video that has been selected to the rendering unit <NUM>.

In step S118, the rendering unit <NUM> renders the entire celestial sphere video that has been supplied from the reproducing device <NUM>, and outputs the entire celestial sphere video that has been rendered to the HMD <NUM> for display.

In step S119, the video controller <NUM> determines whether or not the end of the process has been instructed. In a case where the end has not been instructed, the process returns to step Sill, and the subsequent processes are repeated.

Then, in step S119, in a case where it is determined that the end has been instructed, the process ends.

On the other hand, in a case where it is determined in step S116 that the line quality is not sufficient to reproduce the entire celestial sphere video, the switching determination unit <NUM> outputs the switching determination information for instructing the reproduction of the planar video to the video controller <NUM> and the process proceeds to step S120.

In step S120, the video controller <NUM> reads the pan-tilt information indicating the video-capturing position and the direction of the planar camera <NUM> included in the metadata on the basis of the switching determination information, identifies the position of the planar video in the entire celestial sphere video, and sets a display position and a size of a frame indicating the corresponding position in the entire celestial sphere video.

In step S121, the video controller <NUM> generates the entire celestial sphere video, to which a frame indicating the position and the size corresponding to the planar video has been added on the basis of the head tracking information, the entire celestial sphere video that has been decoded, and the information regarding the position and the size of the frame, and outputs the entire celestial sphere video with the frame to the rendering unit <NUM>.

In step S122, the rendering unit <NUM> renders the entire celestial sphere video with the frame indicating the position and the size corresponding to the planar video, outputs the entire celestial sphere video with the frame to the HMD <NUM> for display.

In step S123, the video controller <NUM> generates a transition video for transitioning from the entire celestial sphere video to the planar video.

The transition video for transitioning from the entire celestial sphere video to the planar video is, for example, a video in a process of gradually enlarging an area in the planar video displayed as the frame in the entire celestial sphere video and transitioning to the planar video. The entire celestial sphere video and the planar video are used for, for example, interpolation generation.

In step S124, the rendering unit <NUM> renders the transition video for transitioning from the entire celestial sphere video to the planar video, and outputs the transition video to the HMD <NUM> for display.

In step S125, the video controller <NUM> outputs the planar video that has been decoded to the rendering unit <NUM>.

In step S126, the rendering unit <NUM> renders the planar video, and outputs the planar video to the HMD <NUM> for display.

That is, by the processes of steps S120 to S126, in the HMD16, after the frame indicating the corresponding position of the planar video is added and the entire celestial sphere video is displayed, the transition video for transitioning to the planar video is displayed, and is then switched to displaying of the planar video.

In step S127, the connection controller <NUM> acquires the entire celestial sphere video and the planar video from the server <NUM>, and outputs the videos to the decoding unit <NUM>.

In step S128, the decoding unit <NUM> decodes the entire celestial sphere video and the planar video that have been supplied from the connection controller <NUM>, and outputs the entire celestial sphere video and the planar video that have been decoded to the video controller <NUM>. That is, the video controller <NUM> acquires the entire celestial sphere video and the planar video that have been decoded.

In step S129, the meta parser unit <NUM> acquires the metadata supplied from the server <NUM>, and outputs the metadata to the video controller <NUM>. That is, the video controller <NUM> acquires the metadata.

In step S130, the video controller <NUM> acquires the head tracking information from the HMD <NUM>.

In step S131, the connection controller <NUM> outputs to the switching determination unit <NUM>, the information regarding, for example, the line speed or the remaining amount of the buffer data of the entire celestial sphere video and the planar video that are being received, as the information regarding the line quality of the entire celestial sphere video from the server <NUM>. That is, the connection controller <NUM> acquires the information regarding the line speed or the remaining amount of the buffer data of the videos, as the information regarding the line quality.

In step S132, the switching determination unit <NUM> determines whether or not the line quality is sufficient to be capable of reproducing the entire celestial sphere video from the information regarding the line quality including the line speed and the remaining amount of the buffer data of the entire celestial sphere video and the planar video that are being received, that is, whether or not to reproduce the entire celestial sphere video.

In a case where it is determined in step S132 that the line quality is not sufficient to reproduce the entire celestial sphere video, the switching determination unit <NUM> outputs the switching determination information for instructing the reproduction of the planar video to the video controller <NUM>. The process proceeds to step S133.

In step S133, the video controller <NUM> determines whether or not the end of the process has been instructed. In a case where the end has not been instructed, the process returns to step S125, and the subsequent processes are repeated.

Then, in step S133, in a case where it is determined that the end has been instructed, the process ends.

That is, in a case where a state in which the sufficient line quality for reproducing the entire celestial sphere video cannot be obtained continues, the processes of steps S125 to S133 are repeated, and the planar video is continuously rendered and displayed on the HMD <NUM>.

Further, in step S132, in a case where it is determined that the line quality is sufficient to reproduce the entire celestial sphere video, the switching determination unit <NUM> outputs the switching determination information for instructing the reproduction of the entire celestial sphere video to the video controller <NUM>. The process proceeds to step S134.

In step S134, the video controller <NUM> generates a transition video for transitioning from the planar video to the entire celestial sphere video, and outputs the transition video that has been generated to the rendering unit <NUM>.

The transition video for transitioning from the planar video to the entire celestial sphere video is, for example, a video in a process of gradually shrinking the planar video to be embedded at the corresponding position and size in the entire celestial sphere video and transitioning to the entire celestial sphere video. The entire celestial sphere video and the planar video are used for, for example, interpolation generation.

In step S135, the rendering unit <NUM> outputs the transition video to the HMD <NUM> for display.

In step S136, the video controller <NUM> reads the pan-tilt information indicating the video-capturing position and the direction of the planar camera <NUM> included in the metadata, identifies the position of the planar video in the entire celestial sphere video, and sets the display position and the size of the frame indicating the corresponding position in the entire celestial sphere video.

In step S137, the video controller <NUM> generates the entire celestial sphere video, to which the frame indicating the position and the size corresponding to the planar video has been added on the basis of the head tracking information, the entire celestial sphere video that has been decoded, and the information regarding the position and the size of the frame, and outputs the entire celestial sphere video with the frame to the rendering unit <NUM>.

In step S138, the rendering unit <NUM> outputs the entire celestial sphere video to which the frame has been added to the HMD <NUM> for display. The process returns to step S117.

That is, by a reverse process of the case where the entire celestial sphere video transitions to the planar video, also when the planar video transitions to the entire celestial sphere video, the planar video transitions to the entire celestial sphere video, and then the frame corresponding to the planar video is temporarily displayed in the entire celestial sphere video so that the planar video at which position in the entire celestial sphere video having been displayed can be recognized.

It is to be noted that in the above, the description has been given with regard to an example in which the position and the size corresponding to the planar video to be switched to next in the entire celestial sphere video are represented by the display information including the frame F in <FIG>, for example. However, the display information other than the frame may be applicable as long as the position and the size corresponding to the planar video to be switched to next in the entire celestial sphere video can be represented. For example, the corresponding area may be colored translucently, or only the corresponding area may be displayed as a black-and-white video.

In addition, depending on the direction of the user wearing the HMD16, there is no area of the planar video to be switched to next in the entire celestial sphere video in the user's field of view. Hence, there is a possibility that the frame indicating the position and the size corresponding to the planar video to be switched to next cannot be displayed.

In such a case, for example, switching from the entire celestial sphere video to the planar video may be waited until the frame indicating the position and the size corresponding to the planar video to be switched to next can be displayed in the user's field of view, and then the switching may be conducted.

Alternatively, in switching from the entire celestial sphere video to the planar video, a video in which the entirety of the entire celestial sphere video, to which a frame has been added, can be viewed is temporarily generated and displayed as a transition video, for example. After forcibly making the user view the video to which the frame has been added, the switching may be conducted by transitioning to the planar video.

Furthermore, in the above, the description has been given with regard to an example in which the line speed, which is the line quality of the entire celestial sphere video, or the remaining amount of the buffer data of the entire celestial sphere video and the planar video that are being received are used as a reference for switching between displaying of the entire celestial sphere video and displaying of the planar video. However, the switching may be conducted with another reference, for example, the switching may be conducted in response to a user's instruction, or may be conducted when displaying a predetermined specific scene according to a presentation or the like of a content.

Next, a display process by the HMD <NUM> of <FIG> will be described with reference to a flowchart of <FIG>.

In step S151, the head tracking information acquisition unit <NUM> of the HMD <NUM> acquires head tracking information of the user.

In step S152, the head tracking information acquisition unit <NUM> outputs the head tracking information that has been acquired to the reproducing device <NUM>.

In step S153, the controller <NUM> acquires the entire celestial sphere video, the planar video, or the transition video supplied from the reproducing device <NUM>, and causes the display unit <NUM> to display the videos.

In step S154, the controller <NUM> determines whether or not the end of the process has been instructed. In a case where the end has not been instructed, the process returns to step S151, and the subsequent processes are repeated.

Then, in step S154, in a case where it is determined that the end has been instructed, the process ends.

By the above series of processes, the head tracking information is acquired, and is continuously output to the reproducing device <NUM>, and in addition, the entire celestial sphere video, the planar video, or the transition video supplied from the reproducing device <NUM> is acquired, and is displayed continuously on the display unit <NUM>.

Consequently, the entire celestial sphere video or the planar video is switched and displayed according to the line quality. At the time of being switched, the frame located at the position and having the size of the corresponding planar video is displayed in the entire celestial sphere video. Further, the transition video is displayed, and is then switched to the planar video.

Therefore, even when the entire celestial sphere video is switched to the planar video, the user can recognize that the planar video at which position in the entire celestial sphere video has been switched to.

In addition, when the planar video is switched to the entire celestial sphere video, the transition video is displayed and then the entire celestial sphere video to which a frame has been added is displayed. This configuration enables recognition that the entire celestial sphere video has been switched from the planar video at which position in the entire celestial sphere video that has been switched to.

As a result, switching between the wide angle of view video and the planar video that is a part of the wide angle of view video is enabled without a sense of incongruity for viewing.

In the above, the description has been given with regard to an example in which the delay processing unit <NUM> of the PC <NUM> causes the planar camera <NUM> to capture the metadata including the pan-tilt information, and the timing with the planar video that has been encoded and output by the encoding device <NUM> is adjusted. However, the server <NUM> may generate time information indicating the timing of the planar video, may include the information in the metadata, and may output the metadata to the reproducing device <NUM>.

With such a configuration, in the reproducing device <NUM>, in selecting a planar video, the corresponding planar video and the metadata can be used according to the time information included in the metadata.

<FIG> shows a configuration example of the reproducing system <NUM> in which the server <NUM> generates the time information indicating the timing of the planar video, includes the time information in the metadata, and outputs the metadata to the reproducing device <NUM>.

In the reproducing system <NUM> of <FIG>, the delay processing unit <NUM> is omitted from the PC <NUM>, and in addition, in the server <NUM>, a reception metadata storage unit <NUM>, a video monitoring unit <NUM>, a time information generation unit <NUM>, and a transmission metadata storage unit <NUM> are provided, instead of the metadata storage unit <NUM>.

That is, in the PC <NUM> of <FIG>, the metadata that has been generated is output to the server <NUM> without a delay adjustment. Therefore, in the PC <NUM>, the delay processing unit <NUM> or the like can be omitted.

Further, the reception metadata storage unit <NUM> is controlled by the controller <NUM>, and stores the metadata supplied from the PC <NUM>, as reception metadata.

The video monitoring unit <NUM> monitors a timing when the planar video is stored in the planar video storage unit <NUM>, and outputs a signal indicating the timing when the planar video is stored to the time information generation unit <NUM>.

The time information generation unit <NUM> generates time information based on a signal indicating the timing when the planar video supplied from the video monitoring unit <NUM> is stored in the planar video storage unit <NUM>, reads the reception metadata stored in the reception metadata storage unit <NUM>, adds the time information that has been generated, and causes the transmission metadata storage unit <NUM> to store as transmission metadata.

Then, the controller <NUM> reads the transmission metadata stored in the transmission metadata storage unit <NUM>, and outputs as metadata to the reproducing device <NUM>.

In the reproducing device <NUM>, in a case where a planar video is selected on the basis of the switching determination information, the pan-tilt information corresponding to the planar video can be used by selecting the metadata including the corresponding time information.

Next, a metadata generation process by the PC <NUM> of <FIG> will be described with reference to a flowchart of <FIG>.

It is to be noted that processes of steps S171 to S174 in the flowchart of <FIG> are similar to the processes of steps S51, S52, S54, and S55 of <FIG>. Therefore, the descriptions thereof will be omitted.

That is, in the metadata generation process of <FIG>, the delay process of the metadata by the delay processing unit <NUM> in step S53 in the metadata generation process of <FIG> is omitted.

As a result, the delay process by the delay processing unit <NUM> can be omitted.

In step S191, the controller <NUM> of the server <NUM> acquires the entire celestial sphere video that has been captured by the entire celestial sphere camera <NUM> and that has been encoded and supplied by the encoding device <NUM>, and causes the entire celestial sphere video storage unit <NUM> to store the entire celestial sphere video.

In step S192, the controller <NUM> reads the entire celestial sphere video stored in the entire celestial sphere video storage unit <NUM>, and outputs the entire celestial sphere video to the reproducing device <NUM>.

In step S193, the controller <NUM> acquires the metadata supplied from the PC <NUM>, and causes the reception metadata storage unit <NUM> to store as the reception metadata.

In step S194, the controller <NUM> acquires the planar video that has been captured by the planar camera <NUM> and that has been encoded and supplied by the encoding device <NUM>, and causes the planar video storage unit <NUM> to store the planar video.

In step S195, the video monitoring unit <NUM> outputs, to the time information generation unit <NUM>, a signal indicating a timing when the planar video that has been captured by the planar camera <NUM> and that has been encoded and supplied by the encoding device <NUM> is stored in the planar video storage unit <NUM>.

In step S196, the time information generation unit <NUM> generates time information at the timing of receiving a signal supplied from the video monitoring unit <NUM>.

In step S197, the time information generation unit <NUM> reads the reception metadata that has been stored in the reception metadata storage unit <NUM>, adds the time information that has been generated, and causes the transmission metadata storage unit <NUM> to store as transmission metadata.

In step S198, the controller <NUM> reads the planar video stored in the planar video storage unit <NUM>, and outputs the planar video to the reproducing device <NUM>.

In step S199, the controller <NUM> reads the transmission metadata stored in the transmission metadata storage unit <NUM>, and outputs the transmission metadata as metadata to the reproducing device <NUM>.

In step S200, the controller <NUM> determines whether or not the end of the process has been instructed. In a case where the end has not been instructed, the process returns to step S191, and the subsequent processes are repeated.

Then, in step S200, in a case where the end of the process has been instructed, the process ends.

By the above process, in addition to the entire celestial sphere video and the planar video, the metadata, to which the time information corresponding to the timing when the planar video is acquired has been added, is transmitted to the reproducing device <NUM>.

It is to be noted that in the reproducing device <NUM>, in processing the planar video, the pan-tilt information included in the metadata including the same time information is used to perform a similar process to the reproduction process of <FIG>. Therefore, the description of the reproduction process will be omitted.

In the above, the description has been given with regard to an example in which in a case where the entire celestial sphere video, the planar video, and the metadata are supplied to the reproducing device <NUM>, and in the reproducing device <NUM>, when the entire celestial sphere video is switched to the planar video according to the line quality, a frame is added and displayed at a position corresponding to the planar video to be switched to and displayed next in the entire celestial sphere video on the basis of the metadata.

However, in the PC <NUM>, a framed entire celestial sphere video, to which a frame indicating the position of the planar video has been added in the entire celestial sphere video, may be generated, and in addition to the entire celestial sphere video and the planar video, the framed entire celestial sphere video may be supplied to the reproducing device <NUM>, so that the reproducing device <NUM> may switch and display the videos.

<FIG> shows a configuration example of the reproducing system <NUM>, in which in the PC <NUM>, the framed entire celestial sphere video, to which the frame indicating the position of the planar video has been added in the entire celestial sphere video, is generated, and in addition to the entire celestial sphere video and the planar video, the framed entire celestial sphere video is supplied to the reproducing device <NUM>, so that the reproducing device <NUM> switches and displays the videos.

In the reproducing system <NUM> of <FIG>, instead of the data controller <NUM> and the delay processing unit <NUM> of the PC <NUM> of <FIG>, a delay processing unit <NUM>, a synthesis processing unit <NUM>, and a data controller <NUM> are provided. This configuration is different from the reproducing system <NUM> in <FIG>.

Further, in the reproducing system <NUM> of <FIG>, a framed entire celestial sphere video storage unit <NUM> is provided in place of the metadata storage unit <NUM> in the server <NUM> of <FIG>. Furthermore, the meta parser unit <NUM> is omitted in the reproducing device <NUM>. This configuration is different from the reproducing system <NUM> in <FIG>.

The delay processing unit <NUM> of the PC <NUM> temporarily stores the entire celestial sphere video that has been captured by the entire celestial sphere camera <NUM>, and outputs the entire celestial sphere video to the synthesis processing unit <NUM> to match the timing when the entire celestial sphere video is output from the entire celestial sphere camera <NUM>.

The data controller <NUM> obtains the position and the size of the planar video in the entire celestial sphere video on the basis of the pan-tilt information supplied from the camera platform 12a, generates frame data for identifying the corresponding range, and outputs the frame data to the synthesis processing unit <NUM>.

On the basis of the frame data supplied from the data controller <NUM>, the synthesis processing unit <NUM> adds a video of the frame corresponding to the planar video to the entire celestial sphere video to generate a framed entire celestial sphere video, and outputs the framed entire celestial sphere video to the server <NUM>.

The controller <NUM> of the server <NUM> acquires the framed entire celestial sphere video supplied from the PC <NUM>, and causes the framed entire celestial sphere video storage unit <NUM> to store the framed entire celestial sphere video.

Then, the controller <NUM> reads the framed entire celestial sphere video stored in the framed entire celestial sphere video storage unit <NUM>, and outputs the framed entire celestial sphere video to the reproducing device <NUM>.

Consequently, the connection controller <NUM> in the reproducing device <NUM> of <FIG> acquires the framed entire celestial sphere video in addition to the entire celestial sphere video and the planar video supplied from the server <NUM>, and outputs the videos to the decoding unit <NUM>.

Further, the decoding unit <NUM> decodes the framed entire celestial sphere video in addition to the entire celestial sphere video and the planar video, and outputs the videos to the video controller <NUM>.

The video controller <NUM> selects any of the entire celestial sphere video, the planar video, and the framed entire celestial sphere video that have been decoded on the basis of the switching determination information supplied from the switching determination unit <NUM>, and outputs the selected one to the rendering unit <NUM>.

Next, a framed entire celestial sphere video generation process in the PC <NUM> will be described with reference to a flowchart of <FIG>.

It is to be noted that the framed entire celestial sphere video generation process is a process performed on the PC <NUM>, instead of the metadata generation process that has been described with reference to the flowchart of <FIG>.

In step S221, the data controller <NUM> acquires the pan-tilt information supplied from the camera platform 12a of the planar camera <NUM>.

In step S222, the data controller <NUM> generates information regarding the position and the size of the frame corresponding to the planar video in the entire celestial sphere video as frame data on the basis of the pan-tilt information that has been acquired, and outputs the frame data to the synthesis processing unit <NUM>.

In step S223, the delay processing unit <NUM> acquires the entire celestial sphere video supplied from the entire celestial sphere camera <NUM>, temporarily stores the entire celestial sphere video, delays by a predetermined period, and then outputs the entire celestial sphere video to the synthesis processing unit <NUM>.

In step S224, the synthesis processing unit <NUM> sets the position and the size of the frame corresponding to the planar video on the entire celestial sphere video on the basis of the frame data.

In step S225, the synthesis processing unit <NUM> adds a frame of the position and the size that have been set onto the entire celestial sphere video, generates a framed entire celestial sphere video, encodes the framed entire celestial sphere video, and then outputs the framed entire celestial sphere video to the server <NUM>.

In step S226, the controller <NUM> determines whether or not the end of the process has been instructed. In a case where the end has not been instructed, the process returns to step S221, and the subsequent processes are repeated.

Then, in step S226, in a case where the end of the process has been instructed, the process ends.

By the above process, the frame data based on the pan-tilt information at the timing when the planar video is captured by the planar camera <NUM> is generated, and in addition, the framed entire celestial sphere video, to which the frame indicating the position and the size of the corresponding planar video has been added on the basis of the frame data, is generated, and is then output to the server <NUM>.

As a result, in addition to the entire celestial sphere video and the planar video, the framed entire celestial sphere video is transmitted to the server <NUM>.

It is to be noted that the processes of steps S241 to S244 and S247 in the flowchart of <FIG> are similar to the processes of steps S71 to S74 and S77 that have been described with reference to the flowchart of <FIG>. Therefore, the descriptions thereof will be omitted.

That is, in step S245, the controller <NUM> acquires the framed entire celestial sphere video supplied from the PC <NUM>, and causes the framed entire celestial sphere video storage unit <NUM> to store the framed entire celestial sphere video.

In step S246, the controller <NUM> reads the framed entire celestial sphere video stored in the framed entire celestial sphere video storage unit <NUM>, and outputs the framed entire celestial sphere video to the reproducing device <NUM>.

By the above process, the entire celestial sphere video, the planar video, and the framed entire celestial sphere video are respectively supplied from the entire celestial sphere camera <NUM>, the planar camera <NUM>, and the PC <NUM>, and are transmitted to the reproducing device <NUM>.

In such a situation, regarding the entire celestial sphere video, the planar video, and the framed entire celestial sphere video, respectively corresponding pieces of information at the same timing are supplied to the reproducing device <NUM>.

Therefore, the processes of steps S241 and S242, the processes of steps S243 and S244, and the processes of steps S245 and S246 in the flowchart of <FIG> are substantially parallel processes.

Next, a reproduction process by the reproducing device <NUM> of <FIG> will be described with reference to a flowchart of <FIG>.

In step S261, the connection controller <NUM> acquires the entire celestial sphere video, the planar video, and the framed entire celestial sphere video from the server <NUM>, and outputs the videos to the decoding unit <NUM>.

In step S262, the decoding unit <NUM> decodes the entire celestial sphere video, the planar video, and the framed entire celestial sphere video that have been supplied from the connection controller <NUM>, and outputs the videos to the video controller <NUM>. That is, the video controller <NUM> acquires the entire celestial sphere video, the planar video, and the framed entire celestial sphere video that have been decoded.

In step S263, the video controller <NUM> acquires the head tracking information from the HMD <NUM>.

In step S264, the connection controller <NUM> outputs to the switching determination unit <NUM>, information regarding, for example, the line speed or the remaining amount of the buffer data of the entire celestial sphere video and the planar video that are being received, as the information regarding the line quality of the entire celestial sphere video from the server <NUM>. That is, the switching determination unit <NUM> acquires the information regarding the line speed and the remaining amount of the buffer data of the videos, as the information regarding the line quality.

In step S265, the switching determination unit <NUM> determines whether or not the line speed is faster than a predetermined speed or whether or not the remaining amount of the buffer data is equal to or larger than a predetermined threshold from the information regarding the line quality including the line speed and the remaining amount of the buffer data of the entire celestial sphere video and the planar video that are being received, so as to determine whether or not the line quality is sufficient to be capable of reproducing the entire celestial sphere video, that is, whether or not to reproduce the entire celestial sphere video.

In a case where it is determined in step S265 that the line quality is sufficient to reproduce the entire celestial sphere video, the switching determination unit <NUM> outputs the switching determination information for instructing the reproduction of the entire celestial sphere video to the video controller <NUM>. The process proceeds to step S266.

In step S266, the video controller <NUM> outputs the entire celestial sphere video that has been decoded to the rendering unit <NUM> on the basis of the switching determination information.

In step S267, the rendering unit <NUM> renders the entire celestial sphere video, and outputs the entire celestial sphere video to the HMD16 for display.

In step S268, the video controller <NUM> determines whether or not the end of the process has been instructed. In a case where the end has not been instructed, the process returns to step S261, and the subsequent processes are repeated.

Then, in step S268, in a case where it is determined that the end has been instructed, the process ends.

On the other hand, in a case where it is determined in step S265 that the line quality is not sufficient to reproduce the entire celestial sphere video, the process proceeds to step S269.

In step S269, the video controller <NUM> outputs the framed entire celestial sphere video that has been decoded to the rendering unit <NUM>.

In step S270, the rendering unit <NUM> renders the framed entire celestial sphere video, and outputs the framed entire celestial sphere video to the HMD <NUM> for display.

In step S271, the video controller <NUM> generates a transition video for transitioning from the entire celestial sphere video to the planar video, and outputs the transition video to the rendering unit <NUM>.

In step S272, the rendering unit <NUM> renders the transition video for transitioning from the entire celestial sphere video to the planar video, and outputs the transition video to the HMD <NUM> for display.

In step S273, the video controller <NUM> outputs the planar video that has been decoded to the rendering unit <NUM>.

In step S274, the rendering unit <NUM> renders a planar video, and outputs the planar video to the HMD <NUM> for display.

That is, by the processes of steps S270 to S274, in the HMD16, after the framed entire celestial sphere video, to which the frame indicating the position corresponding to the planar video on the entire celestial sphere video has been added, is displayed, the transition video for transitioning to the planar video is displayed, and is switched to display the planar video.

In step S275, the connection controller <NUM> acquires the entire celestial sphere video, the planar video, and the framed entire celestial sphere video from the server <NUM>, and outputs the videos to the decoding unit <NUM>.

In step S276, the decoding unit <NUM> decodes the entire celestial sphere video, the planar video, and the framed entire celestial sphere video that have been supplied from the connection controller <NUM>, and outputs the videos to the video controller <NUM>. That is, the video controller <NUM> acquires the entire celestial sphere video, the planar video, and the framed entire celestial sphere video that have been decoded.

In step S277, the video controller <NUM> acquires the head tracking information from the HMD <NUM>.

In step S278, the connection controller <NUM> outputs to the switching determination unit <NUM>, for example, the information regarding the line speed or the remaining amount of the buffer data of the entire celestial sphere video and the planar video that are being received, as the information regarding the line quality of the entire celestial sphere video from the server <NUM>. That is, the switching determination unit <NUM> acquires the information regarding the line speed and the remaining amount of the buffer data of the videos, as the information regarding the line quality.

In step S279, the switching determination unit <NUM> determines whether or not the line speed is faster than a predetermined speed and whether or not the remaining amount of the buffer data is equal to or larger than a predetermined threshold from the information regarding the line quality including the line speed and the remaining amount of the buffer data of the entire celestial sphere video and the planar video that are being received, so as to determine whether or not the line quality is sufficient to be capable of reproducing the entire celestial sphere video, that is, whether or not to reproduce the entire celestial sphere video.

In a case where it is determined in step S279 that the line quality is not sufficient to reproduce the entire celestial sphere video, the switching determination unit <NUM> outputs the switching determination information for instructing the reproduction of the planar video to the video controller <NUM>. The process proceeds to step S280.

In step S280, the video controller <NUM> determines whether or not the end of the process has been instructed. In a case where the end has not been instructed, the process returns to step S273, and the subsequent processes are repeated.

Then, in step S280, in a case where it is determined that the end has been instructed, the process ends.

That is, in a case where a state in which the sufficient line quality for reproducing the entire celestial sphere video cannot be obtained continues, the processes of steps S273 to S280 are repeated, and the planar video is continuously rendered and displayed on the HMD <NUM>.

Further, in step S279, in a case where it is determined that the line quality is sufficient to reproduce the entire celestial sphere video, the switching determination unit <NUM> outputs the switching determination information for instructing the reproduction of the entire celestial sphere video to the video controller <NUM>. The process proceeds to step S281.

In step S281, the video controller <NUM> generates the transition video for transitioning from the planar video to the entire celestial sphere video, and outputs the transition video to the rendering unit <NUM>.

In step S282, the rendering unit <NUM> outputs the transition video to the HMD <NUM> for display.

In step S283, the video controller <NUM> outputs the framed entire celestial sphere video that has been decoded to the rendering unit <NUM>.

In step S284, the rendering unit <NUM> outputs the framed entire celestial sphere video to the HMD <NUM> for display. The process returns to step S266.

That is, by a reverse process of a case where the entire celestial sphere video transitions to the planar video, also in a case where the planar video transitions to the entire celestial sphere video, after transitioning to the entire celestial sphere video by the transition video, the framed entire celestial sphere video is displayed, and then the entire celestial sphere video is displayed.

In step S291, the head tracking information acquisition unit <NUM> of the HMD <NUM> acquires head tracking information of the user.

In step S292, the head tracking information acquisition unit <NUM> outputs the head tracking information that has been acquired to the reproducing device <NUM>.

In step S293, the controller <NUM> acquires the entire celestial sphere video, the planar video, the transition video, or the framed entire celestial sphere video supplied from the reproducing device <NUM>, and causes the display unit <NUM> to display.

In step S294, the controller <NUM> determines whether or not the end of the process has been instructed. In a case where the end has not been instructed, the process returns to step S291, and the subsequent processes are repeated.

Then, in step S294, in a case where it is determined that the end has been instructed, the process ends.

By the above series of processes, the head tracking information is acquired, and is continuously output to the reproducing device <NUM>, and in addition, the entire celestial sphere video, the planar video, or the framed entire celestial sphere video supplied from the reproducing device <NUM> is acquired, and is displayed continuously on the display unit <NUM>.

Consequently, the entire celestial sphere video or the planar video is switched according to the line quality. When switched, the framed entire celestial sphere video, to which a frame of the position and the size of the corresponding planar video has been added in the entire celestial sphere video, is displayed, so that the user can recognize that the planar video at which position has been switched to even when the entire celestial sphere video is switched to the planar video.

In addition, also when the planar video is switched to the entire celestial sphere video, the framed entire celestial sphere video is temporarily displayed after the transition, so as to enable recognition that the entire celestial sphere video has been switched from the planar video at which position in the entire celestial sphere video.

The above-described series of processes, by the way, can be performed by hardware, but can also be performed by software. In a case where a series of processes are performed by software, a program constituting the software is installed into a computer embedded in dedicated hardware, or various programs are installed into, for example, a general-purpose computer or the like capable of performing various functions from a recording medium.

<FIG> shows a configuration example of a general-purpose computer. Such a personal computer includes a central processing unit (CPU) <NUM> that is built in. An input and output interface <NUM> is connected with the CPU <NUM> via a bus <NUM>. A read only memory (ROM) <NUM> and a random access memory (RAM) <NUM> are connected with the bus <NUM>.

The input and output interface <NUM> is connected with an input unit <NUM> including input devices such as a keyboard and a mouse for a user to input operation commands, an output unit <NUM> for outputting a process operation screen and an image of the process result to a display device, a storage unit <NUM> including a hard disk drive or the like for storing programs and various data, and a communication unit <NUM> including a local area network (LAN) adapter or the like, and performing a communication process via a network represented by the Internet. Also connected is a drive <NUM> for reading data from and writing data into a magnetic disk (including a flexible disk), an optical disk (including a compact disc-read only memory (CD-ROM) and a digital versatile disc (DVD), a magneto-optical disk (including a mini disc (MD)), or a removable storage medium <NUM> such as a semiconductor memory.

The CPU <NUM> performs various processes according to the program stored in the ROM <NUM> or the program that has been read from a removable storage medium <NUM> such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, installed into the storage unit <NUM>, and loaded from the storage unit <NUM> into the RAM <NUM>. In the RAM <NUM>, for example, data necessary for the CPU <NUM> to perform various processes is also appropriately stored.

In the computer configured as described above, the CPU <NUM> loads the program stored in the storage unit <NUM> into the RAM <NUM> via the input and output interface <NUM> and the bus <NUM>, and performs the above-described series of processes.

The program performed by the computer (the CPU <NUM>) can be recorded into the removable storage medium <NUM> to be supplied as a package medium or the like, for example. In addition, programs can also be supplied via a wired or wireless transmission medium such as a local area network, the Internet, and digital satellite broadcasting.

On the computer, the program can be installed into the storage unit <NUM> via the input and output interface <NUM> by attaching the removable storage medium <NUM> onto the drive <NUM>. Further, the program can be received by the communication unit <NUM> and installed into the storage unit <NUM> via a wired or wireless transmission medium. In addition, the program can be installed beforehand into the ROM <NUM> or the storage unit <NUM>.

It is to be noted that the program performed by the computer may be a program that is processed in time series according to the order described in the present specification, or may be a program that is processed in parallel or at a necessary timing such as when a call is made.

It is to be noted that the CPU <NUM> in <FIG> realizes the functions of the controllers <NUM>, <NUM>, <NUM> and the video controller <NUM> in <FIG>.

Further, in the present specification, a system means a set of a plurality of components (devices, modules (parts), and the like), and whether or not all the components are included in the same housing does not a matter. Therefore, a plurality of devices accommodated in separate housings and connected via a network, and a device in which a plurality of modules is accommodated in one housing are both the systems.

It is to be noted that the embodiments of the present disclosure is not limited to the above-described embodiments, and various changes can be made without departing from the gist of the present disclosure.

For example, the present disclosure can be configured as cloud computing in which a plurality of devices shares one function via a network and process in cooperation.

Further, each step described in the above-mentioned flowchart can be performed by one device or can be shared and performed by a plurality of devices.

Claim 1:
A reproducing device (<NUM>) comprising:
a switching determination unit (<NUM>) configured to determine whether or not a switching condition for switching a display between a wide angle of view video (P1) and a partial video (P2) is satisfied, the partial video (P2) corresponding to a partial angle of view of the wide angle of view video (P1);
a video controller (<NUM>) configured to selectively display the wide angle of view video (P1) or the partial video (P2) on a basis of a determination result of the switching determination unit (<NUM>); and
a video acquisition unit (<NUM>) configured to acquire the wide angle of view video (P1) and the partial video (P2) from a server (<NUM>) that is externally provided;
wherein the video controller (<NUM>) is configured to, when switching the display between the wide angle of view video (P1) and the partial video (P2) on a basis of the determination result of the switching determination unit (<NUM>), display the wide angle of view video (P1), to which display information (F) indicating an angle of view of the partial video (P2) corresponding to the wide angle of view video (P1) has been added; and
wherein the switching determination unit (<NUM>) is configured to determine whether or not the switching condition for switching the display between the wide angle of view video (P1) and the partial video (P2) is satisfied on a basis of a line quality with the server (<NUM>) from which the video acquisition unit (<NUM>) acquires the wide angle of view video (P1).