An information processing apparatus that includes a receiving unit that receives a request including load information regarding a load and a sending unit that sends a data set in accordance with the request. The data set includes three-dimensional shape data, and left-eye texture data and right-eye texture data. The three-dimensional shape data has a vertex count corresponding to the load information. The left-eye texture data and the right-eye texture data correspond to the three-dimensional shape data.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a U.S. National Phase of International Patent Application No. PCT/JP2018/039717 filed on Oct. 25, 2018, which claims priority benefit of Japanese Patent Application No. JP 2018-010471 filed in the Japan Patent Office on Jan. 25, 2018. Each of the above-referenced applications is hereby incorporated herein by reference in its entirety.

TECHNICAL FIELD

The present disclosure relates to an information processing apparatus and an information processing method.

BACKGROUND ART

3D movies, 3D television, and the like each use a mechanism that has a left eye and a right eye view different images and uses the binocular parallax between the images to provide a user with a stereoscopic effect. Further, some techniques use HMDs (Head Mounted Displays) or the like mounted on the heads of users to reproduce motion parallax.

Images viewed from different viewpoints are required to reproduce motion parallax because it is necessary to move a viewpoint in accordance with the head position or the like of a user. Examples thereof include a technique for rendering (generating) an image viewed from each viewpoint by detecting/recording information regarding the three-dimensional shape of an object and using a three-dimensional model reconfigured on the basis of the information as in PTL 1 below.

CITATION LIST

Patent Literature

SUMMARY OF THE INVENTION

Problems to be Solved by the Invention

In the technique as described above, three-dimensional shape data used for rendering, however, has a great influence on the subjective image quality of an image (display image) viewed from each viewpoint. Depending on the accuracy or the like of the three-dimensional shape data, the subjective image quality of the display image may thus decrease.

Accordingly, the present disclosure proposes a mechanism that makes it possible to suppress a decrease in the subjective image quality of a display image generated on the basis of three-dimensional shape data.

Means for Solving the Problems

According to the present disclosure, there is provided an information processing apparatus including: a receiving unit that receives a request including load information regarding a load; and a sending unit that sends a data set in accordance with the request. The data set includes three-dimensional shape data, and left-eye texture data and right-eye texture data. The three-dimensional shape data has a vertex count corresponding to the load information. The left-eye texture data and the right-eye texture data correspond to the three-dimensional shape data.

In addition, according to the present disclosure, there is provided an information processing apparatus including: a sending unit that sends a request including load information regarding a load; a receiving unit that receives a data set including three-dimensional shape data, and left-eye texture data and right-eye texture data; and a rendering unit that generates a left-eye display image and a right-eye display image on the basis of the data set. The three-dimensional shape data has a vertex count corresponding to the load information. The left-eye texture data and the right-eye texture data correspond to the three-dimensional shape data.

In addition, according to the present disclosure, there is provided an information processing method including: receiving a request including load information regarding a load; and causing, by a processor, a data set to be sent in accordance with the request. The data set includes three-dimensional shape data, and left-eye texture data and right-eye texture data. The three-dimensional shape data has a vertex count corresponding to the load information. The left-eye texture data and the right-eye texture data correspond to the three-dimensional shape data.

Effects of the Invention

According to the present disclosure as described above, it is possible to suppress a decrease in the subjective image quality of a display image generated on the basis of three-dimensional shape data.

It is to be noted that the above-described effects are not necessarily limitative. Any of the effects indicated in this description or other effects that may be understood from this description may be exerted in addition to the above-described effects or in place of the above-described effects.

MODES FOR CARRYING OUT THE INVENTION

The following describes a preferred embodiment of the present disclosure in detail with reference to the accompanying drawings. It is to be noted that, in this description and the accompanying drawings, components that have substantially the same functional configuration are indicated by the same reference signs, and thus redundant description thereof is omitted.

It is to be noted that description is given in the following order.

<1-2. Flow of Process>

<<2. Principle according to the Present Technology>>

1. System Overview

First, an overview of an information processing system according to an embodiment of the present disclosure is described with reference toFIG. 1.FIG. 1is an explanatory diagram schematically illustrating the configuration of a transmission system according to the embodiment of the present disclosure.

As illustrated inFIG. 1, a transmission system1000according to the present embodiment is an information processing system including a distribution server1, a display control apparatus2, HMD3, and a communication network5. The transmission system1000according to the present embodiment provides a stereoscopic effect brought about by binocular parallax and motion parallax to a user U who is wearing the HMD3and viewing an image displayed on the HMD3.

The distribution server1and the display control apparatus2are coupled via the communication network5. It is possible to transmit and receive information between the distribution server1and the display control apparatus2. In addition, the display control apparatus2and the HMD3are also coupled in a wired or wireless manner, and it is possible to transmit and receive information between the display control apparatus2and the HMD3.

The communication network5is a wired or wireless transmission path for information sent from an apparatus coupled to the communication network5. For example, the communication network5may include a public network such as the Internet, a telephone network, or a satellite communication network, and various LANs (Local Area Networks) including Ethernet (registered trademark), WAN (Wide Area Network), and the like. In addition, the communication network5may include a private network such as IP-VPN (Internet Protocol-Virtual Private Network).

The distribution server1stores three-dimensional shape data and texture data corresponding to the three-dimensional shape data, and sends (transmits) a data set to the display control apparatus2in accordance with a request to send data (that is also referred to simply as request below) from the display control apparatus2. The data set includes the three-dimensional shape data and the texture data.

On the basis of the data set received from the distribution server1and viewpoint information regarding the viewpoint of the user U received from the HMD3, the display control apparatus2generates (renders) a display image at the viewpoint, and sends the display image to the HMD3. In the present embodiment, the display image generated by the display control apparatus2includes a left-eye display image and a right-eye display image. The left-eye display image is displayed in front of the left eye of the user U by the HMD3described below. The right-eye display image is displayed in front of the right eye of the user U. In addition, the following also refers the left-eye display image and the right-eye display image collectively as stereo display image or simply as display image in some cases.

The HMD3is a display apparatus (display unit) that displays a stereo display image received from the display control apparatus2. It is to be noted that the HMD3includes a sensor which acquires viewpoint information regarding the viewpoint of the user U wearing the HMD3, and sends the viewpoint information to the display control apparatus2. The viewpoint information sent by the HMD3may include, for example, information indicating the position of the viewpoint of the user U and the attitude of the user U.

It is to be noted thatFIG. 1illustrates the display control apparatus2and the HMD3as different apparatuses, but the display control apparatus2and the HMD3may also be integrated. That is, one information processing apparatus may also have a function of the display control apparatus2, and a function of the HMD3serving as a display unit which is worn on the head of a user and displays a left-eye display image and a right-eye display image.

The above-described configuration allows the transmission system1000to provide the user U with a stereoscopic effect brought about by binocular parallax and motion parallax.

Here, the above-described three-dimensional shape data and texture data corresponding to the three-dimensional shape data may be acquired, for example, by well-known three-dimensional capture technology that uses, for example, a method in which a distance measurement device such as a ToF (Time of Flight) sensor is used or a technique such as stereo matching is used. The following respectively refers the three-dimensional shape data and texture data acquired by this three-dimensional capture technology as original three-dimensional shape data and original texture data, and further refers to both collectively as original data in some cases.

This original data has an extremely large data amount in some cases. Therefore, the transmission load and the processing load are also heavy, and it is difficult under some conditions to perform a process within requested time. For example, depending on the band (band of the communication network5) of the transmission path between the distribution server1and the display control apparatus2, it may be difficult to transmit the original data as it is. In addition, depending on the processing performance (such as the processing speed of a processor and the size of a memory) of the display control apparatus2, it may be difficult to generate a display image from the original data.

It is thus considered to change a data amount to be transmitted from the distribution server1to the display control apparatus2in accordance with such a load. Such a mechanism is described below.

The display control apparatus2sends a request including load information regarding a load to the distribution server1. The load information may include, for example, at least one of transmission path band information regarding the band of the transmission path between the distribution server1and the display control apparatus2or processing performance information regarding the processing performance of the display control apparatus2.

The distribution server1sends a data set including three-dimensional shape data and texture data corresponding to the three-dimensional shape data to the display control apparatus2. The data set has the data amount corresponding to the load information included in the request received from the display control apparatus2. Here, each of the three-dimensional shape data and texture data included in the data set to be sent may be data having a data amount reduced more than those of the original three-dimensional shape data and original texture data. A process of reducing the data amount may include, for example, a vertex reduction process of reducing a vertex included in the three-dimensional shape data.

As described above, changing the data amount to be transmitted from the distribution server1to the display control apparatus2in accordance with the load information may smooth the transmission and the generation of a display image. However, the subjective image quality of the display image may decrease along with a reduction in the data amount. For example, reducing a vertex included in the three-dimensional shape data may decrease the shape accuracy of the three-dimensional model corresponding to the three-dimensional shape data, and the subjective image quality of a display image may decrease. For example, the three-dimensional model corresponding to the three-dimensional shape data loses unevenness or the like, and it is possible as a result that a user is not able to obtain a stereoscopic effect.

Accordingly, focusing on the above-described circumstances, the present embodiment has been created. The distribution server1according to the present embodiment uses even a left camera image and right camera image to generate left-eye texture data and right-eye texture data. The left camera image and the right camera image are used to generate (capture) three-dimensional shape data. That is, a data set to be sent from the distribution server1according to the present embodiment to the display control apparatus2includes not a single piece of texture data, but multiple pieces of texture data: left-eye texture data; and right-eye texture data. The display control apparatus2then generates a left-eye display image and a right-eye display image on the basis of the three-dimensional shape data subjected to the vertex reduction process and the left-eye texture data and the right-eye texture data. The three-dimensional shape data, the left-eye texture data, and the right-eye texture data are included in the data set received from the distribution server1. This configuration suppresses a decrease in the subjective image quality of a display image even in a case where the three-dimensional shape data subjected to the vertex reduction process is used for display. For example, it may be possible to reproduce unevenness, which is absent from the three-dimensional model corresponding to the three-dimensional shape data.

1-2. Flow of Process

The above has described the schematic configuration of the transmission system1000according to the present embodiment. Next, a schematic flow of a process of the transmission system1000according to the present embodiment is described.

As described above, the distribution server1according to the present embodiment sends the display control apparatus2the three-dimensional shape data and texture data each having a reduced data amount. Here, it is desirable to perform a process for reducing the data amount in advance. For example, a plurality of data sets having different data amounts may be generated and stored in advance on the basis of the original data. The distribution server1then selects a data set having the data amount corresponding to the load information from the plurality of stored data sets, and sends the selected data set to the display control apparatus2. This allows the distribution server1to quickly respond to a request of the display control apparatus2.

Accordingly, the following describes a flow of the process for generating a data set with reference toFIG. 2, and then describes a flow of a process for transmitting data with reference toFIG. 3.

(Process for Generating Data Set)

FIG. 2is a schematic diagram schematically illustrating a flow of a process of generating a data set according to the present embodiment. The following describes an example in which the process of generating a data set illustrated inFIG. 2is performed by the distribution server1illustrated inFIG. 1. The present technology is not, however, limited to this example. The process illustrated inFIG. 2may be performed by another apparatus. The data set generated by the other apparatus in advance may be stored in the distribution server1.

The distribution server1performs a process of reducing data included in a data set DS10, and generates a plurality of data sets DS11to DS13. The generated data sets DS11to DS13are stored in the distribution server1.

The data set DS10illustrated inFIG. 2includes original three-dimensional shape data F10, original texture data T10, a left camera image G10L, and a right camera image G10R. The original three-dimensional shape data F10includes multiple pieces of vertex data V101to V107. In addition, the left camera image G10L and the right camera image G10R may be images used to generate the original three-dimensional shape data F10. For example, the left camera image G10L is an image acquired by a left camera performing imaging, and the right camera image G10R is an image acquired by a right camera performing imaging. The left camera images an object from the left side. The right camera images an object from the right side.

As illustrated inFIG. 2, the generated data sets DS11to DS13respectively include pieces of three-dimensional shape data F11to F13, and pieces of left-eye texture data T11L to T13L and pieces of right-eye texture data T11R to T13R. The pieces of left-eye texture data T11L to T13L and the pieces of right-eye texture data T11R to T13R correspond to the pieces of three-dimensional shape data F11to F13. As illustrated inFIG. 2, the generated data sets DS11to DS13have different data amounts.

In the example illustrated inFIG. 2, the data set DS11has the largest data amount of the data sets DS11to DS13. In the example illustrated inFIG. 2, the three-dimensional shape data F11included in the data set DS11has not been subjected to the vertex reduction process. The number of pieces of vertex data V111to V117included in the three-dimensional shape data F11is the same as the number of pieces of vertex data included in the original three-dimensional shape data F10. The present embodiment is not, however, limited to this example. The distribution server1may perform the vertex reduction processes on the pieces of three-dimensional shape data included in all the data sets that may be sent.

In contrast, the three-dimensional shape data F12and three-dimensional shape data F13included in the data set DS12and data set DS13are generated by performing the vertex reduction process on the original three-dimensional shape data F10. In the example illustrated inFIG. 2, pieces of vertex data V121to V124included in the three-dimensional shape data F12are less than pieces of vertex data V101to107included in the original three-dimensional shape data F10, and more than pieces of vertex data V131to V132included in the three-dimensional shape data F13. It is to be noted that the number of vertices (vertex count) included in each piece of three-dimensional shape data is not limited to the number of pieces of vertex data illustrated inFIG. 2.

In addition, the pieces of left-eye texture data T11L to T13L and pieces of right-eye texture data T11R to T13R included in the pieces of data set DS11to DS12may be generated to have the pixel counts corresponding to the vertex counts of the respective pieces of three-dimensional shape data F11to F13. As illustrated inFIG. 2, as the corresponding three-dimensional shape data has a higher vertex count, the left-eye texture data and the right-eye texture data may each have a higher pixel count. This is because even an increase in the pixel count of each of the left-eye texture data and right-eye texture data does not considerably contribute to improvement in the image quality in a case where the three-dimensional shape data has a low vertex count. This configuration makes it possible to efficiently reduce the data amount.

It is to be noted thatFIG. 2illustrates an example in which the three data sets DS11to DS13are generated, but the number of data sets to be generated is not limited to the example illustrated inFIG. 2. As a larger number of data sets are generated, the data amount transmitted in accordance with the band of a transmission path and processing performance is more finely adjustable. However, depending on the number of data sets to be generated, the processing cost for generating the data sets and the retaining cost of the data sets are requested. It is thus desirable to determine the number of data sets to be generated by taking into consideration the processing cost and the retaining cost.

FIG. 3is a schematic diagram schematically illustrating a flow of a data transmission process according to the present embodiment. The distribution server1selects a data set to be sent to the display control apparatus2from the plurality of data sets DS11to DS13generated in advance as described above with reference toFIG. 2(S11). In step S11, for example, on the basis of the load information included in a request received from the display control apparatus2, the distribution server1may select a data set including three-dimensional shape data having the vertex count corresponding to the load information. Such selection makes it possible to adjust the transmission of a data set from the distribution server1to the display control apparatus2and the load for a rendering process performed by the display control apparatus2.

Subsequently, the data set selected in step S11is transmitted (sent) from the distribution server1to the display control apparatus2(S12). The display control apparatus2then generates (renders) a display image on the basis of the data set received from the distribution server1(S13). The display image generated in step S13includes a left-eye display image D10L and a right-eye display image D10R. The left-eye display image D10L is displayed in front of the left eye of a user. The right-eye display image D10R is displayed in front of the right eye of the user. In addition, in step S13, the three-dimensional shape data and the left-eye texture data are used to generate the left-eye display image D10L, and the three-dimensional shape data and the right-eye texture data are used to generate the right-eye display image D10R.

The left-eye display image D10L and right-eye display image D10R generated by the display control apparatus2are displayed by the HMD3(S14).

2. Principle According to the Present Technology

The above has described the overview of the present embodiment. As described above, in the present embodiment, not a single piece of texture data, but left-eye texture data and right-eye texture data are transmitted. The transmitted left-eye texture data and right-eye texture data are then used to render a display image. This makes it possible to suppress a decrease in the image quality of the display image while reducing the data amount. Here, the technical principle is described that suppresses a decrease in the image quality of a display image by performing rendering by using left-eye texture data and right-eye texture data in the present embodiment. It is to be noted that the following description refers to the left-eye texture data and the right-eye texture data collectively as stereo texture data in some cases.

First, an association between three-dimensional shape data and texture data is described in a case of performing rendering by using a single piece of texture data with reference toFIGS. 4 and 5.FIG. 4is an explanatory diagram for describing an association between a vertex of a three-dimensional model and a single piece of texture data. In addition,FIG. 5is a diagram illustrating an example of the data structure of three-dimensional shape data.

FIG. 4illustrates a three-dimensional model M21that is a cube, and a single piece of texture data T21corresponding to the three-dimensional model M21. Three-dimensional space coordinates (x0, y0, z0) indicating a vertex in the three-dimensional model M21illustrated inFIG. 4correspond to texture coordinates (u0, v0) on texture data T21.

As illustrated inFIG. 5, in three-dimensional shape data F21retained in a calculator, (x0, y0, z0, u0, v0) that is a combination of these corresponding coordinates may be treated as one piece of vertex data. Then, in the example illustrated inFIG. 5, N arrays of pieces of such vertex data are included in the three-dimensional shape data F21. That is, the three-dimensional shape data F21includes information for associating the coordinates of a vertex in the three-dimensional model with the corresponding coordinates in the texture data.

Here, when three from each head of the three-dimensional shape data F21illustrated inFIG. 5are treated as a triangular patch, it is possible to form the three-dimensional model M21in the three-dimensional space coordinate system of xyz illustrated inFIG. 4. The three-dimensional model M21has a stereoscopic shape including a triangular patch group. Further, an association between the three-dimensional space coordinates (x, y, z) and the texture coordinates (u, v) makes it possible to acquire a triangular region in the texture data T21corresponding to each triangular patch. Mapping (pasting) the triangular region to a triangular patch in the three-dimensional model M21while transforming the triangular region with Affine transformation thus makes it possible to render a three-dimensional model using texture data.

The above has described an association between three-dimensional shape data and texture data in a case of performing rendering by using a single piece of texture data. Subsequently, an association between three-dimensional shape data and texture data is described for performing rendering by using stereo texture data in the present embodiment.

FIG. 6is an explanatory diagram for describing an association between a vertex of a three-dimensional model and texture data in the present embodiment.FIG. 6illustrates a three-dimensional model M22, and left-eye texture data T22L and right-eye texture data T22R. The three-dimensional model M22is a cube. The left-eye texture data T22L and the right-eye texture data T22R correspond to the three-dimensional model M22.

Three-dimensional space coordinates (x0, y0, z0) indicating a vertex in the three-dimensional model M22illustrated inFIG. 6correspond to texture coordinates (u0, v0) on left-eye texture data T22L. In addition, the three-dimensional space coordinates (x0, y0, z0) indicating the vertex in the three-dimensional model M22illustrated inFIG. 6similarly correspond to texture coordinates (u0, v0) on right-eye texture data T22R.

As illustrated inFIG. 6, it is possible to represent texture coordinates in the left-eye texture data T22L and texture coordinates in the right-eye texture data T22R as the same coordinates. The texture coordinates in the left-eye texture data T22L and the texture coordinates in the right-eye texture data T22R correspond to the same vertex in the three-dimensional model M22. The data structure of three-dimensional shape data according to the present embodiment may be therefore similar to that of the example described with reference toFIG. 5.

FIG. 7is a schematic diagram for describing rendering that uses stereo texture data in the present embodiment. As illustrated inFIG. 7, mapping the left-eye texture data T22L to the three-dimensional model M22and performing rendering at the viewpoint corresponding to the left eye of a user make it possible to generate (render) a left-eye display image D22L. In addition, as illustrated inFIG. 7, mapping the right-eye texture data T22R to the three-dimensional model M22and performing rendering at the viewpoint corresponding to the right eye of the user make it possible to generate (render) a right-eye display image D22R.

The process for performing rendering illustrated inFIG. 7may be performed by the display control apparatus2illustrated inFIG. 1. It is to be noted that information of an association between a vertex position of a three-dimensional model with texture data is represented as three-dimensional shape data as described with reference toFIG. 5. The use of a data set received from the distribution server1thus allows the display control apparatus2to perform rendering as illustrated inFIG. 7. The data set includes three-dimensional shape data and stereo texture data corresponding to the three-dimensional shape data.

As described above, the use of stereo texture data for rendering the texture of a common three-dimensional model allows the rendered stereo display image to fuse at a position different from the surface of the three-dimensional model. This is the same as the principle that, for example, it is possible to provide a stereoscopic effect in spite of a flat display surface in a stereoscopic display that allows an image displayed on a flat screen to provide binocular parallax. The present technology suppresses a decrease in the subjective image quality of a display image rendered at a user viewpoint by using the effect that performing rendering by using such stereo texture allows unevenness different from that of the shape of the three-dimensional model to be recognized.

As described above, the original three-dimensional shape data is acquired by three-dimensional capture technology such as a method in which a distance measurement device is used or a method in which a technique such as stereo matching is used. The original three-dimensional shape data is acquired in various methods, but any of the methods may have an error.

In addition, even if it is possible to acquire a shape with high accuracy, a complicated shape requires an extremely large data amount or an extremely high vertex count of three-dimensional shape data to reproduce the shape with high accuracy. As described with reference toFIGS. 1 to 3, a data set including three-dimensional shape data subjected to the vertex reduction process is transmitted and used for rendering in accordance with the load information regarding the band of a transmission path, processing performance, or the like in the present embodiment. Thus, depending on the load information, three-dimensional shape data having an insufficient vertex count to reproduce a shape with high accuracy or three-dimensional shape data including an error may be used for rendering.

When rendering is performed by using three-dimensional shape data including an error caused by any of the acquisition (measurement), the transmission, or the process and a single piece of texture in this way, a shape including an error is reproduced and a user viewing a stereo display image also recognizes the shape including the error. In contrast, even in a case where three-dimensional shape data including such an error is used, the three-dimensional shape data and stereo texture are used to perform rendering in the present embodiment. This causes an error of a shape to look smaller, making it possible to suppress a decrease in the subjective image quality. The following describes the principle that such an error of a shape looks smaller in the present embodiment.

FIG. 8is an explanatory diagram for describing an error in a three-dimensional model.FIG. 8illustrates the true shape of a three-dimensional object OBJ31having a protrusion (beak) and the shape of a three-dimensional model M32. The three-dimensional model M32is three-dimensionally modeled on the three-dimensional object OBJ31by three-dimensional capture technology. The three-dimensional object OBJ31originally includes a protrusion B as illustrated inFIG. 8. In contrast, the three-dimensional model M32has no protrusion because of an error caused by measurement or the like. It is to be noted thatFIG. 8illustrates, as a point P, the tip position corresponding to the tip of the protrusion B of the three-dimensional object OBJ31in the three-dimensional space in which the three-dimensional model M32is present.

Here, in a case where the three-dimensional shape data corresponding to the three-dimensional model M32illustrated inFIG. 8and a single piece of texture data are used to perform rendering, it is difficult for a user viewing the generated stereo display image to recognize the protrusion. In contrast, the use of stereo texture makes it possible to cause a protrusion to look present in the present embodiment even if the three-dimensional shape data corresponding to a three-dimensional model including an error like the three-dimensional model M32is used to perform rendering.

To provide a stereoscopic effect brought about by stereo texture, texture data is used that is generated on the basis of a camera image acquired by a stereo camera disposed to have horizontal distance close to human interocular distance. The following describes the principle of texture data generation based on a camera image acquired by a stereo camera in the present embodiment.

FIG. 9is a schematic diagram schematically illustrating imaging by a stereo camera. InFIG. 9, a left camera C31L and a right camera C31R are disposed as a stereo camera at substantially the same interval as the human interocular distance to allow the three-dimensional object OBJ31to be imaged. It is to be noted thatFIG. 9illustrates the imaging ranges of the left camera C31L and the right camera C31R with one-dot chain lines.

The positional relationship between the stereo camera and the three-dimensional object corresponds to the positional relationship between the stereo camera and the three-dimensional model in the three-dimensional space. The known positional relationship between the stereo camera and the three-dimensional model in three-dimensional space makes it possible to generate the texture data corresponding to the three-dimensional model as follows from a camera image acquired by the stereo camera.

FIG. 10is a schematic diagram schematically illustrating a flow of a process of generating texture data. It is possible to generate a perspective projection image from the positional relationship between a stereo camera and a three-dimensional model by making a perspective projection of the shape of the three-dimensional model viewed from each camera position on the perspective projection surface. In the example illustrated inFIG. 10, the three-dimensional model M32is projected that is viewed from the position of a left camera C32L illustrated inFIG. 9, and a left perspective projection image P31L is generated. Similarly, the three-dimensional model M32is projected that is viewed from the position of a right camera C32R illustrated inFIG. 9, and a right perspective projection image P31R is generated.

The composition of each perspective projection image is the same as that of a camera image acquired by each camera performing imaging. This makes it possible to establish an association between the perspective projection image and the camera image for each triangular region corresponding to a triangular patch including three vertices in a three-dimensional model. In the example illustrated inFIG. 10, a triangular region A31L corresponding to a triangular patch A30of the three-dimensional model M32in the left perspective projection image P31L and a triangular region A32L of a left camera image G32L are associated. Similarly, a triangular region A31R corresponding to the triangular patch A30in the right perspective projection image P31R and a triangular region A32R of a right camera image G32R are associated.

It is possible to generate texture data by mapping (pasting) each triangular region in each camera image on the basis of an association for each triangular region acquired as described above while transforming the triangular region. In the example illustrated inFIG. 10, the triangular region A32L of the left camera image G32L is mapped to a triangular region A33L of left-eye texture data T33L, and a triangular region A32R of the right camera image G32R is mapped to the triangular region A33L of right-eye texture data T33R.

It is to be noted that a perspective projection image generated from a perspective projection from one camera position has a surface that does not have the three-dimensional model, and the texture data may thus have a region whose texture is not obtained from the camera image. In this case, for the region whose texture is not obtained from the camera image, texture may be acquired from the original texture data illustrated inFIG. 2, for example.

A case is considered where stereo texture is generated by using the three-dimensional model M32including an error as illustrated inFIG. 10.FIG. 11is a schematic diagram illustrating the relationship between the three-dimensional model M32including an error and the stereo camera (left camera C31L and right camera C31R).

InFIG. 11, the arrow extending from each of the left camera C31L and the right camera C31R to the point P represents a light ray of the point P in an image that appears in each camera. The point P is the tip position of the protrusion present in the true shape. The three-dimensional model M32has no protrusion because of an error. The respective arrows extending from the left camera C31L and the right camera C31R to the point P thus intersect at not the point P, but a point PRand a point PLon the surface of the three-dimensional model M32. Distance d between these point PRand point PLis mapped as it is as the positional difference between the left-eye texture data and the right-eye texture data, and recorded as it is as left-right parallax.

The left-eye texture data T33L and right-eye texture data T33R illustrated inFIG. 10have a design difference because of this left-right parallax. The three-dimensional model M32corresponding to the three-dimensional shape data has no protrusion. This causes the pattern of the protrusion to be recorded as having left-right parallax on texture.

In a case where the stereo texture data generated as described above is mapped to the three-dimensional model M32, and viewed and listened to from a camera position, it is possible to obtain a stereoscopic effect similar to that of the true shape even if the shape of the three-dimensional model M32includes an error.FIG. 12is a schematic diagram illustrating a viewing and listening situation in which a camera position and the position of an eye of a user match each other. InFIG. 12, a light ray represented as an arrow extends from the point PLto a left eye E32L of a user present at the position of the left camera C31L, and a light ray represented as an arrow extends from the point PRto a right eye E32R of the user present at the position of the right camera C31R. Here, it looks to the user like the texture at the point PLand the texture at the point PRfuse at the position of the point P at which the above-described two light rays intersect, and the surface of the three-dimensional model M32is present at the position of the point P.

It is to be noted thatFIG. 12is only for the point P, but all the light rays appearing in the left and right cameras in addition to the point P are reproduced. This makes it possible to obtain the original stereoscopic effect even if the shape of the three-dimensional model M32is different from the true shape. Moreover, in a case of viewing and listening with the camera positions matching with the positions of the eyes of a user, the light rays at the time of imaging by the cameras are reproduced and the true shape looks reproduced to the user even if the three-dimensional model M32has any shape.

It is not, however, possible to reproduce the actual light rays if the shape of the three-dimensional model M32includes an error in a case of viewing and listening at positions different from the camera positions.FIG. 13is a schematic diagram illustrating a viewing and listening situation in which the camera position and the position of an eye of a user are different from each other. In the situation illustrated inFIG. 13, the texture at the point PLappearing in the left eye E32L of a user and the texture at the point PLappearing in the right eye E32R of the user fuse at a point P′, and the surface of the three-dimensional model M32looks present at the position of the point P′. Here, the point P′ illustrated inFIG. 13is a position different from the point P that is the true tip position of the protrusion. This is because the three-dimensional model M32includes an error. The point P′, however, looks present at a protruding position as compared with a case where the protrusion looks absent. This suppresses a decrease in the subjective image quality caused by an error even in a case of viewing and listening at positions different from the camera positions.

Here, an example of a case is described where a technique (that is referred to as existing technique below) for performing rendering by using a single piece of texture data in place of stereo texture data is used for the present embodiment as a comparative example.FIGS. 14 and 15are explanatory diagrams each for describing a comparative example according to the present embodiment.FIGS. 14 and 15are diagrams in a case of mapping a single piece of texture data to the three-dimensional model M32, and viewing and listening in the same positional relationships as those ofFIGS. 12 and 13, respectively.

The existing technique does not allow left and right eyes to fuse points at different positions on the surface of the three-dimensional model M32like the point PLand point PRinFIGS. 12 and 13. Therefore, in this comparative example, for example, as illustrated inFIGS. 14 and 15, a user uses the left and right eyes to view a common point PCpositioned on the surface of the three-dimensional model M32. That is, the existing technique does not allow a user to recognize a point positioned on a protrusion in a case where the three-dimensional model M32includes an error, but has no protrusion. In other words, it is not possible for the existing technique to render a point that looks present at a position out of the shape of the three-dimensional model M32as illustrated inFIGS. 12 and 13.

As described above, according to the present embodiment, it is possible to have a user view a shape closer to the true shape as compared with the existing technique. In other words, performing rendering by using stereo texture makes it possible to suppress a decrease in the subjective image quality of a display image at each viewpoint in the present embodiment even in a case where the three-dimensional shape data corresponding to a three-dimensional shape model including an error is used for rendering.

The above has described the technical principle according to the present embodiment. Subsequently, configuration examples and operation examples of the distribution server1and the display control apparatus2are described one by one for achieving the above-described functions and processes.

3. Distribution Server

FIG. 16is a block diagram illustrating an example of the configuration of the distribution server1according to the present embodiment illustrated inFIG. 1. As illustrated inFIG. 16, the distribution server1is an information processing apparatus including a controller10, a communication unit17, and a storage unit19.

The controller10controls each component of the distribution server1. In addition, the controller10also functions as a data set generation unit11, a communication control unit13, and a selection unit15as illustrated inFIG. 16.

The data set generation unit11generates a plurality of data sets having different data amounts on the basis of the original three-dimensional shape data, the original texture data, the left camera image, and the right camera image as described with reference toFIG. 2.

FIG. 17is a block diagram illustrating an example of the configuration of the data set generation unit11. As illustrated inFIG. 17, the data set generation unit11includes a vertex reduction section111, a perspective projection section112, and a texture generation section115.

The vertex reduction section111performs the vertex reduction process on the original three-dimensional shape data. The three-dimensional shape data outputted from the vertex reduction section111and subjected to the vertex reduction process is provided to the perspective projection section112, and associated with stereo texture data described below. The three-dimensional shape data is then stored in the storage unit19as a data set. That is, the three-dimensional shape data included in the data set is generated by the vertex reduction section111performing the vertex reduction process.

FIG. 18is an explanatory diagram illustrating the vertex reduction process by the vertex reduction section111. As illustrated inFIG. 18, the six vertices and seven polygons of a three-dimensional model M41corresponding to three-dimensional shape data that has not been subjected to the vertex reduction process are respectively reduced to the four vertices and three polygons of a three-dimensional model M42corresponding to three-dimensional shape data that has been subjected to the vertex reduction process. Performing the vertex reduction process in this way makes a rougher shape, but allows data to be considerably reduced. This makes it possible to considerably reduce the loads for transmitting and processing data. It is to be noted that the technique of the vertex reduction section111for the vertex reduction process is not limited in particular, but, for example, a well-known vertex reduction technique such as QEM (Quadric Error Metrics) may be used.

FIG. 19is a schematic diagram illustrating gradual vertex reduction. The example illustrated inFIG. 19demonstrates that, as the vertex count becomes smaller in the order of three-dimensional models M51, M52, and M53, the shape becomes different and the pieces of respective vertex data included in pieces of three-dimensional shape data F51, F52, and F53corresponding to the respective shapes have smaller data amounts.

It is to be noted that, in a case where a data set is generated in which there is no need to reduce a vertex of the three-dimensional shape data like the data set DS11illustrated inFIG. 2, the vertex reduction section111does not have to perform the vertex reduction process, but may output the original three-dimensional shape data as it is.

FIG. 17is referred to again, and the data set generation unit11is continuously described. The perspective projection section112makes perspective projections from the left and right camera positions corresponding to the left-eye texture data and right-eye texture data by using the three-dimensional shape data as described with reference toFIG. 10, and generates perspective projection images.

As illustrated inFIG. 17, the perspective projection section112includes a left perspective projector113L and a right perspective projector113R. The left perspective projector113L makes a perspective projection of the shape of the three-dimensional model from the position of the left camera corresponding to the left-eye texture data to the perspective projection surface, and generates a left perspective projection image. The three-dimensional model corresponds to the three-dimensional shape data provided from the vertex reduction section111. Similarly, the right perspective projector113R makes a perspective projection of the shape of the three-dimensional model from the position of the left camera corresponding to the right-eye texture data to the perspective projection surface, and generates a right perspective projection image. The three-dimensional model corresponds to the three-dimensional shape data provided from the vertex reduction section111.

The texture generation section115establishes associations between the perspective projection images generated by the perspective projection section112and the camera images, and maps the camera images to generate left-eye texture data and right-eye texture data as described with reference toFIG. 10. It is to be noted that the texture generation section115may establish an association between a perspective projection image and a camera image for each region (e.g., for each triangular region corresponding to a triangular patch) based on a vertex included in three-dimensional shape data as described with reference toFIG. 10.

As illustrated inFIG. 17, the texture generation section115includes a left generation processor116L, a right generation processor116R, a left resolution changer117L, and a right resolution changer117R.

The left generation processor116L establishes an association between the left perspective projection image and the left camera image for each triangular region. The left generation processor116L then maps a triangular region of the left camera image to the corresponding triangular region in the left-eye texture data to generate the left-eye texture data. The right generation processor116R similarly establishes an association between the right perspective projection image and the right camera image for each triangular region. The right generation processor116R then maps a triangular region of the left camera image to the corresponding triangular region in the right-eye texture data to generate the right-eye texture data. It is to be noted that the left generation processor116L and the right generation processor116R may each acquire texture from the original texture data for a region whose texture is not obtained from the camera image, and generate the texture data as described above.

The left resolution changer117L performs a resolution change process on the left-eye texture data generated by the left generation processor116L, and outputs the left-eye texture data. The right resolution changer117R similarly performs the resolution change process on the right-eye texture data generated by the right generation processor116R, and outputs the right-eye texture data. The left-eye texture data outputted from the left resolution changer117L and the right-eye texture data outputted from the right resolution changer117R are associated with the three-dimensional shape data outputted from the vertex reduction section111, and stored in the storage unit19as a data set.

The left resolution changer117L and the right resolution changer117R may perform the resolution change processes to cause the left-eye texture data and right-eye texture data included in each data set to each have the pixel count corresponding to the vertex count of the three-dimensional shape data included in the data set. In a case where the three-dimensional shape data has a low vertex count, reducing the pixel count of each of the left-eye texture data and right-eye texture data does not lead to a considerable decrease in the image quality. This configuration thus makes it possible to efficiently reduce the data amount.

The above has described the configuration example of the data set generation unit11with reference toFIG. 17. It is to be noted that the data set generation unit11generates a plurality of data sets having different data amounts as described above. This allows the data set generation unit11to repeat the generation of a data set as the same times as the desired number of data sets while appropriately changing a parameter, for example, to gradually change the vertex count of the three-dimensional shape data included in the data set and the pixel count of each piece of texture data.

Alternatively, the controller10may include the plurality of data set generation units11, and generate data sets in parallel.FIG. 20is a schematic diagram schematically illustrating that data sets are generated in parallel in a case where the controller10includes the plurality of data set generation units11.

N data set generation units11-1to11-N illustrated inFIG. 20may each have a configuration similar to that of the data set generation unit11illustrated inFIG. 17. The data set generation units11-1to11-N respectively generate data sets DS-1to DS-N on the basis of the original three-dimensional shape data, the original texture data, the left camera image, and the right camera image. The respective data sets DS-1to DS-N include pieces of three-dimensional shape data having different vertex counts, and the pieces of left-eye texture data and right-eye texture data corresponding to the pieces of three-dimensional shape data.

As illustrated inFIG. 20, parallel processes make it possible to more efficiently generate a plurality of data sets having gradually different data amounts.

FIG. 16is referred to again, and the controller10of the distribution server1is continuously described. The communication control unit13controls communication with another apparatus established by the communication unit17. For example, the communication control unit13controls the communication unit17to cause the communication unit17to receive a request including the load information from the display control apparatus2. In addition, the communication control unit13controls the communication unit17to cause the communication unit17to send a data set selected by the selection unit15described below to the display control apparatus2in accordance with the request received from the display control apparatus2.

The selection unit15selects a data set to be sent by the communication unit17from a plurality of data sets generated by the data set generation unit11and stored in the storage unit19on the basis of the load information included in a request received by the communication unit17. As described above, the selection unit15may select a data set including three-dimensional shape data having the vertex count corresponding to the load information, and the left-eye texture data and right-eye texture data corresponding to the three-dimensional shape data.

The load information may include transmission path band information regarding the band of the transmission path between the distribution server1and the display control apparatus2as described above. For example, in a case where the band of the transmission path between the distribution server1and the display control apparatus2is not sufficient wide to transmit the data amount of a data set including the original three-dimensional shape data, the selection unit15selects a data set including three-dimensional shape data having a lower vertex count than that of the original three-dimensional shape data.

For example, the HMD3finds a convenient use when allowing a user to freely walk around within some range. It is thus desirable that the HMD3establish wireless communication. Then, in a case where the display control apparatus2and the HMD3are integrated to provide the HMD3with the function of the display control apparatus2, the transmission path between the distribution server1and the display control apparatus2(integrated with the HMD3) is supposed to have a narrow band. In contrast, in a case where the display control apparatus2and the HMD3are different apparatuses and the display control apparatus2is coupled to the communication network5in a wired manner, the transmission path between the distribution server1and the display control apparatus2is supposed to have a wide band. In any of such cases, the selection unit15is able to appropriately select a data set on the basis of the band of the transmission path.

In addition, the load information may include processing performance information regarding the processing performance of the display control apparatus2as described above. The display control apparatus2may come in a variety of processing performances. The display control apparatus2and the HMD3may be different apparatuses, and the display control apparatus2may be high-spec PC having a high processing performance. Alternatively, in a case where the display control apparatus2and the HMD3are integrated and the HMD3has the function of the display control apparatus2, the display control apparatus2has a lower processing performance than that of the high-spec PC in some cases. In addition, incorporating a smartphone into HMD also allows the smartphone to function as the display control apparatus2and the HMD3. In this case, the display control apparatus2is supposed to have a lower processing performance. In any of such cases, the selection unit15is able to appropriately select a data set on the basis of the processing performance.

The communication unit17performs information communication with another apparatus under the control of the above-described communication control unit13. For example, the communication unit17functions as a receiving unit, and receives a request including the load information regarding a load from the display control apparatus2. In addition, the communication unit17functions as a sending unit, and sends a data set in accordance with the received request. The data set includes three-dimensional shape data having the vertex count corresponding to the load information, and left-eye texture data and right-eye texture data corresponding to the three-dimensional shape data.

The storage unit19stores a program and a parameter for causing each component of the distribution server1to function. For example, the storage unit19stores the above-described original three-dimensional shape data, original texture data, left camera image, and right camera image in advance, and provides the data set generation unit11therewith. In addition, the storage unit19stores a plurality of data sets generated by the data set generation unit11.

The above has described the configuration example of the distribution server1according to the present embodiment. Next, an operation example of the distribution server1according to the present embodiment is described. It is to be noted that the distribution server1according to the present embodiment generates a plurality of data sets in advance, and sends a data set selected from the plurality of data sets to the display control apparatus2in accordance with a request from the display control apparatus2as described above. Accordingly, the following describes an operation example of the distribution server1for generating the data set with reference toFIG. 21, and then describes an operation example of the distribution server1for transmitting a data set with reference toFIG. 22.

FIG. 21is a flowchart illustrating an operation example of the distribution server1for generating a data set. As illustrated inFIG. 21, the vertex reduction section111of the data set generation unit11first performs the vertex reduction process on the original three-dimensional shape data (S101).

Next, the perspective projection section112of the data set generation unit11uses the three-dimensional shape data subjected to the vertex reduction process in step S101to make perspective projections from the left and right camera positions, and generates perspective projection images (S103).

Subsequently, the texture generation section115of the data set generation unit11establishes associations between the perspective projection images and the left and right camera images, and generates left-eye texture data and right-eye texture data (S105).

Further, the texture generation section115performs the resolution change processes on the left-eye texture data and right-eye texture data generated in step S105to provide the left-eye texture data and the right-eye texture data with the pixel count corresponding to the vertex count of the three-dimensional shape data subjected to the vertex reduction process in step S101(S107).

The data set generation unit11then associates the three-dimensional shape data subjected to the vertex reduction process in step S101with the left-eye texture data and right-eye texture data each subjected to the resolution change process in step S107, and causes the storage unit19to store the three-dimensional shape data, and the left-eye texture data and the right-eye texture data as a data set (S109).

The above has described the operation example of the distribution server1for generating a data set with reference toFIG. 21. It is to be noted that the series of processes illustrated inFIG. 21may be repeated as the same times as the desired number of data sets while a parameter is appropriately changed, for example, to gradually change the vertex count of the three-dimensional shape data included in the data set and the pixel count of each piece of texture data. Alternatively, the series of processes illustrated inFIG. 21may be performed in parallel by the plurality of data set generation units11as illustrated inFIG. 20.

Next, an operation example of the distribution server1for transmitting a data set is described with reference toFIG. 22.FIG. 22is a flowchart illustrating the operation example of the distribution server1for transmitting a data set. As illustrated inFIG. 22, the communication unit17first receives a request including the load information regarding a load from the display control apparatus2under the control of the communication control unit13(S151).

Next, the selection unit15selects a data set to be sent by the communication unit17from a plurality of data sets stored in the storage unit19on the basis of the load information included in the request received in step S151(S153). As described above, the data set selected in step S153includes three-dimensional shape data having the vertex count corresponding to the load information, and the left-eye texture data and right-eye texture data corresponding to the three-dimensional shape data.

Subsequently, the communication unit17sends the data set selected in step S153to the display control apparatus2in accordance with the request under the control of the communication control unit13(S155). The request is received from the display control apparatus2in step S151.

4. Display Control Apparatus

The above has described the configuration example and operation example of the distribution server1according to the present embodiment. Next, a configuration example of the display control apparatus2is described.FIG. 23is a block diagram illustrating the configuration example of the display control apparatus2according to the present embodiment. As illustrated inFIG. 23, the display control apparatus2is an information processing apparatus including a controller20, a communication unit27, and a storage unit29.

The controller20controls each component of the display control apparatus2. In addition, the controller20also functions as a rendering unit21and a communication control unit23as illustrated inFIG. 23.

The rendering unit21generates (renders) a left-eye display image and a right-eye display image on the basis of a data set received by the communication unit27described below from the distribution server1as described above with reference toFIGS. 3, 7, and the like. In addition, on the basis of viewpoint information regarding the viewpoint of a user, the rendering unit21may also generate a left-eye display image and right-eye display image at the viewpoint. The viewpoint information is received by the communication unit27from the HMD3.

The communication control unit23controls communication with another apparatus established by the communication unit27. For example, the communication control unit23controls the communication unit27to cause the communication unit27to send the distribution server1a request including the load information regarding a load. It is to be noted that the communication control unit23may acquire the load information from the storage unit29or acquire the load information from the outside via the communication unit27. In addition, the communication control unit23controls the communication unit27to cause the communication unit27to receive a data set from the distribution server1. In addition, the communication control unit23controls the communication unit27to cause the communication unit27to receive the viewpoint information regarding the viewpoint of a user from the HMD3. In addition, the communication control unit23controls the communication unit27to cause the communication unit27to send a left-eye display image and a right-eye display image to the HMD3, and causes the HMD3to display the left-eye display image and the right-eye display image. The left-eye display image and the right-eye display image are generated by the rendering unit21.

The communication unit27performs information communication with another apparatus under the control of the above-described communication control unit23. For example, the communication unit27functions as a sending unit, and sends a request including the load information regarding a load to the distribution server1. In addition, the communication unit27functions as a receiving unit, and receives a data set from the distribution server1. The data set includes three-dimensional shape data having the vertex count corresponding to the load information, and left-eye texture data and right-eye texture data corresponding to the three-dimensional shape data. In addition, the communication unit27receives the viewpoint information regarding the viewpoint of a user from the HMD3, and sends the HMD3a left-eye display image and right-eye display image generated by the rendering unit21.

The above has described the configuration example of the display control apparatus2according to the present embodiment. Next, an operation example of the display control apparatus2according to the present embodiment is described.FIG. 24is a flowchart illustrating the operation example of the display control apparatus2.

As illustrated inFIG. 24, the communication unit27first sends a request including the load information regarding a load to the distribution server1under the control of the communication control unit23(S201). Next, the communication unit27receives a data set from the distribution server1under the control of the communication control unit23(S203). As described above, the data set received in step S203includes three-dimensional shape data having the vertex count corresponding to the load information, and the left-eye texture data and right-eye texture data corresponding to the three-dimensional shape data.

Next, the rendering unit21generates a left-eye display image and a right-eye display image on the basis of the data set received in step S203(S205). Subsequently, the communication control unit23controls the communication unit27to cause the communication unit27to send a left-eye display image and a right-eye display image to the HMD3, thereby causing the HMD3to display the left-eye display image and the right-eye display image. The left-eye display image and the right-eye display image are generated in step S205.

5. Modification Example

The above has described the embodiment of the present disclosure. The following describes some modification examples of the embodiment of the present disclosure. It is to be noted that the modification examples described below may be individually applied to the embodiment of the present disclosure or may be applied to the embodiment of the present disclosure in combination. In addition, each modification example may be applied in place of the configuration described in the embodiment of the present disclosure or may be additionally applied to the configuration described in the embodiment of the present disclosure.

In the above-described embodiment, the example has been described in which the selection unit15selects a data set to be sent by the communication unit17on the basis of the load information, but the present technology is not limited to this example. For example, further on the basis of the number of objects included in a data set to be sent by the communication unit17, the selection unit15may select the data set to be sent by the communication unit17. This example is described as a modification example 1.

FIG. 25is an explanatory diagram for describing the modification example 1. In the present modification example, a data set may be generated for each object. Then, in the present modification example, the selection unit15may select a data set for each object. Further, as a data set to be sent by the communication unit17includes more objects, the selection unit15may select a data set having a smaller data amount. For example, the selection unit15may select data sets on the basis of the number of objects to cause data sets to be sent to have a constant total data amount. For example, in a case where objects double in number, a data set may be selected to cause each object to have half a data amount.

It is to be noted that the number of objects included in a data set to be sent by the communication unit17may be identified, for example, on the basis of the original data stored in the storage unit19. In addition, in a case where the viewpoint information regarding the viewpoint of a user is obtained from the display control apparatus2, the number of objects included in a data set to be sent by the communication unit17may be identified in accordance with the visual field of the user identified on the basis of the viewpoint information.FIG. 25illustrates visual fields W11to W13of a user as an example.

The visual field W11of the user includes one object OBJ111, and thus a data set to be sent by the communication unit17also includes one object. In this case, the selection unit15may select a data set DS21including the three-dimensional shape data F21having a large vertex count, and left-eye texture data T21L and right-eye texture data T21R each having a high pixel count as illustrated inFIG. 25. As a result, a data set DS31to be sent includes three-dimensional shape data F31, and left-eye texture data T31L and right-eye texture data T31R as illustrated inFIG. 25.

The visual field W12of the user includes two objects OBJ121and OBJ122, and thus a data set to be sent by the communication unit17also includes two objects. In this case, the selection unit15may select, for each object, a data set DS22including three-dimensional shape data F22whose vertex count is reduced as compared with the three-dimensional shape data F21as illustrated inFIG. 25. In addition, as illustrated inFIG. 25, the left-eye texture data T22L and right-eye texture data T22R included in the data set DS22have lower pixel counts than those of the left-eye texture data T21L and right-eye texture data T21R. As a result, a data set DS32to be sent includes pieces of three-dimensional shape data F32-1and F32-2, and pieces of left-eye texture data T32-1L and T32-2L and pieces of right-eye texture data T32-1R and T32-2R as illustrated inFIG. 25.

The visual field W13of the user includes three objects OBJ131, OBJ132, and OBJ133, and thus a data set to be sent by the communication unit17also includes three objects. In this case, the selection unit15may select, for each object, a data set DS23including three-dimensional shape data F23whose vertex count is further reduced as compared with the three-dimensional shape data F22as illustrated inFIG. 25. In addition, as illustrated inFIG. 25, the left-eye texture data T23L and right-eye texture data T23R included in the data set DS23have further lower pixel counts than those of the left-eye texture data T22L and right-eye texture data T22R. As a result, a data set DS33to be sent includes pieces of three-dimensional shape data F33-1to F33-3, and pieces of left-eye texture data T33-1L to T33-3L and pieces of right-eye texture data T33-1R to T33-3R as illustrated inFIG. 25.

As described above, according to the present modification example, selecting a data set to be sent further on the basis of the number of objects included in the data set makes it possible to send an appropriate data set.

In the above-described embodiment, the example has been described in which the distribution server1generates a data set, but the present technology is not limited to this example. For example, the function of the above-described data set generation unit11may be installed in another information processing apparatus. The other information processing apparatus may generate a plurality of data sets in advance, and provide the distribution server1therewith.

In addition, in the above-described embodiment, the example has been described in which two pieces of texture data are generated on the basis of the two camera images of a left camera image and a right camera image, but the present technology is not limited to this example. For example, three or more pieces of texture data may be generated on the basis of three or more camera images. In this case, two images of the three or more camera images may be regarded as a left camera image and a right camera image, and two pieces of texture data of the three or more pieces of generated texture data may be regarded as left-eye texture data and right-eye texture data.

It is to be noted that, in a case where three or more pieces of texture data are generated, the distribution server1may receive the viewpoint information from the display control apparatus2, and select the left-eye texture and right-eye texture data included in a data set to be sent from the three or more pieces of texture data on the basis of the viewpoint information.

In the above-described embodiment, the example has been described in which three-dimensional shape data acquired on the basis of three-dimensional capture technology is the original three-dimensional shape data, but the present technology is not limited to this example. For example, three-dimensional shape data that is not based on sensing in real space, but generated on a computer is the original three-dimensional shape data, and the present technology is applicable. In this case, for example, a left camera image and a right camera image may be generated by imaging (rendering), with a virtual stereo camera, a three-dimensional model corresponding to the original three-dimensional shape data, and the present technology may be applied.

6. Hardware Configuration Example

The above has described the embodiment of the present disclosure. Finally, the hardware configuration of the information processing apparatus according to the embodiment of the present disclosure is described with reference toFIG. 26.FIG. 26is a block diagram illustrating an example of the hardware configuration of the information processing apparatus according to the embodiment of the present disclosure. It is to be noted that an information processing apparatus900illustrated inFIG. 26may achieve, for example, the distribution server1, display control apparatus2, and HMD3illustrated inFIGS. 1, 16, and 23. Information processing by the distribution server1, display control apparatus2, and HMD3according to the present embodiment is achieved in cooperation between software and hardware described below.

As illustrated inFIG. 26, the information processing apparatus900includes CPU (Central Processing Unit)901, ROM (Read Only Memory)902, RAM (Random Access Memory)903, and a host bus904a. In addition, the information processing apparatus900includes a bridge904, an external bus904b, an interface905, an input device906, an output device907, a storage device908, a drive909, a coupling port911, a communication device913, and a sensor915. The information processing apparatus900may include a processing circuit such as DSP or ASIC in place of or in addition to the CPU901.

The CPU901functions as an arithmetic processing device and a control device, and controls the overall operation in the information processing apparatus900according to various programs. In addition, the CPU901may be a microprocessor. The ROM902stores a program to be used by the CPU901, an arithmetic parameter, and the like. The RAM903temporarily stores a program used in execution of the CPU901, a parameter appropriately changed in the execution, and the like. The CPU901may be included, for example, in the controller10and the controller20.

The CPU901, the ROM902, and the RAM903are coupled to each other by the host bus904aincluding a CPU bus and the like. The host bus904ais coupled to the external bus904bsuch as a PCI (Peripheral Component Interconnect/Interface) bus via the bridge904. It is to be noted that the host bus904a, the bridge904, and the external bus904bdo not necessarily have to be separately included, but the functions thereof may be implemented in a single bus.

The input device906is achieved by a device through which a user inputs information, such as a mouse, a keyboard, a touch panel, a button, a microphone, a switch, and a lever, for example. In addition, the input device906may be a remote control device using infrared rays or other electric waves, or an external coupling device such as a mobile phone or PDA supporting an operation of the information processing apparatus900, for example. Further, the input device906may include an input control circuit or the like that generates an input signal on the basis of information inputted by the user using the above-described input means and outputs the generated input signal to the CPU901, for example. The user of the information processing apparatus900may input various kinds of data to the information processing apparatus900or instructs the information processing apparatus900to perform a processing operation by operating this input device906.

The output device907includes a device that is able to visually or aurally notify a user of acquired information. Examples of such a device include a display device such as a CRT display device, a liquid crystal display device, a plasma display device, an EL display device, and a lamp, an audio output device such as a speaker and a headphone, a printing device, and the like. The output device907outputs results acquired through various kinds of processing performed by the information processing apparatus900, for example. Specifically, the display device visually displays results acquired through various kinds of processing performed by the information processing apparatus900, in various forms such as text, images, tables, and graphs. Meanwhile, the audio output device converts audio signals including reproduced audio data, acoustic data, and the like into analog signals, and aurally outputs the analog signals.

The storage device908is a device for data storage that is formed as an example of a storage unit of the information processing apparatus900. For example, the storage device908is achieved by a magnetic storage unit device such as HDD, a semiconductor storage device, an optical storage device, a magneto-optical storage device, or the like. The storage device908may include a storage medium, a recording device that records data in the storage medium, a reading device that reads data from the storage medium, a deletion device that deletes data recorded in the storage medium, and the like. This storage device908stores a program to be executed by the CPU901, various kinds of data, various kinds of data acquired from the outside, and the like. The above-described storage device908may be included, for example, in the storage unit19and the storage unit29.

The drive909is a reader/writer for a storage medium, and is incorporated in or externally attached to the information processing apparatus900. The drive909reads out information recorded in a removable storage medium such as a magnetic disk, optical disc, magneto-optical disk, or semiconductor memory mounted thereon, and outputs the information to the RAM903. In addition, the drive909is also able to write information into the removable storage medium.

The coupling port911is an interface coupled to an external apparatus, and is a coupling port to an external apparatus that is able to transmit data through USB (Universal Serial Bus) and the like, for example.

The communication device913is a communication interface including, for example, a communication device and the like for coupling to a network920. The communication device913is, for example, a communication card or the like for wired or wireless LAN (Local Area Network), LTE (Long Term Evolution), Bluetooth (registered trademark), or WUSB (Wireless USB). In addition, the communication device913may be a router for optical communication, a router for ADSL (Asymmetric Digital Subscriber Line), a modem for various kinds of communication, or the like. For example, this communication device913is able to transmit and receive signals and the like to and from the Internet and another communication device in accordance with a predetermined protocol such as, for example, TCP/IP. The communication device913may be included, for example, in the communication unit17and the communication unit27.

The sensor915may be, for example, various sensors such as an acceleration sensor, a gyro sensor, a geomagnetic sensor, an optical sensor, a sound sensor, a ranging sensor, and a force sensor. The sensor915acquires information regarding the state of the information processing apparatus900itself such as the attitude and moving speed of the information processing apparatus900, and information regarding the surrounding environment of the information processing apparatus900such as the brightness and noise around the information processing apparatus900. In addition, the sensor915may include a GPS sensor that receives a GPS signal and measures the latitude, longitude, and altitude of the apparatus.

It is to be noted that the network920is a wired or wireless transmission path for information sent from an apparatus coupled to the network920. For example, the network920may include a public network such as the Internet, a telephone network, or a satellite communication network, and various LANs (Local Area Networks) including Ethernet (registered trademark), WAN (Wide Area Network), and the like. In addition, the network920may include a private network such as IP-VPN (Internet Protocol-Virtual Private Network).

The above has described an example of the hardware configuration that makes it possible to achieve a function of the information processing apparatus900according to the embodiment of the present disclosure. The respective components described above may be achieved by using general-purpose members, or may be achieved by hardware specific to the functions of the respective components. It is thus possible to appropriately change hardware configurations to be utilized in accordance with a technical level at the time of carrying out the embodiment of the present disclosure.

It is to be noted that it is possible to create a computer program for achieving each function of the information processing apparatus900according to the embodiment of the present disclosure as described above and install the computer program in PC or the like. In addition, it is also possible to provide a computer-readable recording medium having such a computer program stored therein. The recording medium is, for example, a magnetic disk, an optical disc, a magneto-optical disk, a flash memory, or the like. In addition, the above-described computer program may be distributed, for example, via a network without using a recording medium.

According to the embodiment of the present disclosure as described above, it is possible to suppress a decrease in the subjective image quality of a display image generated on the basis of three-dimensional shape data.

A preferred embodiment(s) of the present disclosure has/have been described above in detail with reference to the accompanying drawings, but the technical scope of the present disclosure is not limited to such an embodiment(s). A person skilled in the art may find various alterations and modifications within the scope of the appended claims, and it should be understood that they will naturally come under the technical scope of the present disclosure.

For example, the respective steps in the above-described embodiment do not necessarily have to be processed in chronological order in accordance with the order described as a flowchart. For example, the respective steps in the processes according to the above-described embodiment may be processed in order different from the order described as a flowchart, or may be processed in parallel.

In addition, the effects described herein are merely illustrative and exemplary, but not limitative. That is, the technology according to the present disclosure may exert other effects that are apparent to those skilled in the art from the description herein, in addition to the above-described effects or in place of the above-described effects.

It is to be noted that the following configurations also fall within the technical scope of the present disclosure.

An information processing apparatus including:

a receiving unit that receives a request including load information regarding a load; and

a sending unit that sends a data set in accordance with the request, the data set including three-dimensional shape data, and left-eye texture data and right-eye texture data, the three-dimensional shape data having a vertex count corresponding to the load information, the left-eye texture data and the right-eye texture data corresponding to the three-dimensional shape data.

The information processing apparatus according to (1), further including a selection unit that makes selection of the data set to be sent by the sending unit from a plurality of data sets on the basis of the load information, the plurality of data sets each including the three-dimensional shape data, and the left-eye texture data and the right-eye texture data, the left-eye texture data and the right-eye texture data corresponding to the three-dimensional shape data.

The information processing apparatus according to (2), in which the selection unit makes the selection further on the basis of a number of objects included in the data set to be sent by the sending unit.

The information processing apparatus according to (2) or (3), further including a data set generation unit that generates the plurality of data sets.

The information processing apparatus according to (4), in which the data set generation unit includes a vertex reduction section that generates the three-dimensional shape data through a vertex reduction process, the three-dimensional shape data being included in the data set.

The information processing apparatus according to (4) or (5), in which the data set generation unit further includes a perspective projection section that makes perspective projections from respective camera positions corresponding to the left-eye texture data and the right-eye texture data by using the three-dimensional shape data, and generates perspective projection images.

The information processing apparatus according to (6), in which the data set generation unit further includes a texture generation section that establishes associations between the perspective projection images and camera images, and generates the left-eye texture data and the right-eye texture data, the camera images being acquired by performing imaging from the camera positions.

The information processing apparatus according to (7), in which the texture generation section establishes the associations between the perspective projection images and the camera images for each region based on a vertex included in the three-dimensional shape data.

The information processing apparatus according to any one of (1) to (8), in which the left-eye texture data and the right-eye texture data each has a pixel count corresponding to a vertex count of the three-dimensional shape data, the left-eye texture data and the right-eye texture data being included in each data set, the three-dimensional shape data being included in the data set.

The information processing apparatus according to any one of (1) to (9), in which the load information includes transmission path band information regarding a band of a transmission path between a sending apparatus and the information processing apparatus, or processing performance information regarding processing performance of the sending apparatus, the sending apparatus sending the request.

An information processing apparatus including:

a sending unit that sends a request including load information regarding a load;

a receiving unit that receives a data set including three-dimensional shape data, and left-eye texture data and right-eye texture data, the three-dimensional shape data having a vertex count corresponding to the load information, the left-eye texture data and the right-eye texture data corresponding to the three-dimensional shape data; and

a rendering unit that generates a left-eye display image and a right-eye display image on the basis of the data set.

The information processing apparatus according to (11), in which the load information includes transmission path band information regarding a band of a transmission path between a receiving apparatus and the information processing apparatus, or processing performance information regarding processing performance of the information processing apparatus, the receiving apparatus receiving the request.

The information processing apparatus according to (11) or (12), in which the rendering unit generates the left-eye display image and the right-eye display image further on the basis of information regarding a viewpoint of a user.

The information processing apparatus according to (13), further including a display unit that is worn on a head of the user, and displays the left-eye display image and the right-eye display image.

An information processing method including:

receiving a request including load information regarding a load; and

causing, by a processor, a data set to be sent in accordance with the request, the data set including three-dimensional shape data, and left-eye texture data and right-eye texture data, the three-dimensional shape data having a vertex count corresponding to the load information, the left-eye texture data and the right-eye texture data corresponding to the three-dimensional shape data.

An information processing method including:

sending a request including load information regarding a load;

receiving a data set including three-dimensional shape data, and left-eye texture data and right-eye texture data, the three-dimensional shape data having a vertex count corresponding to the load information, the left-eye texture data and the right-eye texture data corresponding to the three-dimensional shape data; and

generating, by a processor, a left-eye display image and a right-eye display image on the basis of the data set.

REFERENCE SIGNS LIST