Patent Publication Number: US-2017358280-A1

Title: Method of controlling head-mounted display system

Description:
CROSS-REFERENCE TO RELATED APPLICATIONS 
     This application is a Continuation of U.S. patent application Ser. No. 15/296,345 filed on Oct. 18, 2016, which is a Continuation of PCT International Application No. PCT/JP2016/073782 filed in Japan on Aug. 12, 2016, which claims the benefits of Patent Application No. 2015-167470 filed in Japan on Aug. 27, 2015, Patent Application No. 2015-167471 filed in Japan on Aug. 27, 2015, and Patent Application No. 2015-167472 filed in Japan on Aug. 27, 2015, the entire contents of each of which are hereby incorporated by reference. 
    
    
     TECHNICAL FIELD 
     The present disclosure relates to a program and method for causing a computer to provide a virtual space into which a user is immersed by using a head-mounted display (HMD). 
     RELATED ART 
     Patent Literature 1 discloses a display device which displays a publication as a virtual object in a virtual space to be provided to a user by using a HMD and allows the publication to be manipulated in response to a user&#39;s operation. Patent Literature 2 discloses a display control device which moves, in response to a motion of a HMD, a display position of a display object displayed in front of user&#39;s eyes by using a HMD. 
     CITATION LIST 
     Patent Literature 
     [Patent Literature 1] 
     Japanese Patent Application Publication Tokukai No. 2014-071498 
     [Patent Literature 2] 
     Japanese Patent Application Publication Tokukai No. 2014-197066 
     SUMMARY 
     The HMD system disclosed in Patent Literature 1 is a system for displaying a virtual publication, which imitates a real book, in a real space image captured by a camera which is provided in the HMD. Thus, there is a limit on the extent to which the HMD system disclosed in Patent Literature 1 can provide a unique experience of a virtual space into which the user is immersed. This also applies to the HMD system disclosed in Patent Literature 2. 
     The present disclosure provides a program for providing, to a head-mounted display, a virtual space into which a user is immersed, the program causing a computer to execute the steps of: generating a virtual space into which a user is immersed; from among a plurality of display objects having a sequence for making up a given content; causing the main image to be displayed in the virtual space in such a manner as to be associated with the basic sightline; and outputting the virtual space to the head-mounted display. 
     The present disclosure provides a program for providing, to a head-mounted display, a virtual space into which a user is immersed, the program causing a computer to execute the steps of: generating a virtual space into which a user is immersed; generating a plurality of display objects having a sequence for making up a given content in the virtual space, and then placing the plurality of display objects in the virtual space according to the sequence; in response to a user&#39;s operation, causing a selected one of the plurality of display objects to be displayed in a main view with an emphasis in the virtual space; and outputting the virtual space to the head-mounted display. 
     The present disclosure provides a program for providing, to a head-mounted display, a virtual space into which a user is immersed, the program causing a computer to execute the steps of: generating a virtual space into which a user is immersed so as to contain a virtual space image, which serves as a background, and display objects superimposed on the virtual space image; updating positions or displays of the display objects; and in updating the positions or displays of the display objects, performing processing for reducing visual effects of the display objects. 
     The present disclosure enables provision of a new experience in a virtual space to a user. 
    
    
     
       BRIEF DESCRIPTION OF DRAWINGS 
         FIG. 1  is a view illustrating a HMD system according to at least one embodiment. 
         FIG. 2  is a view illustrating a hardware configuration of a control circuit section according to at least one embodiment. 
         FIG. 3  is a view illustrating an orthogonal coordinate system, on a three-dimensional space, which is specified on the assumption that a head of a user who wears a HMD is a center thereof according to at least one embodiment. 
         FIG. 4  is a view of a XYZ-space illustrating an example of a virtual space according to at least one embodiment. 
         FIG. 5  is a view illustrating an example of a method for determining a sightline direction according to at least one embodiment. 
         FIG. 6  is a functional block diagram of a control circuit section for realizing the functions of the HMD system according to at least one embodiment. 
         FIG. 7  is a flow chart of a process for implementing a function of the HMD system according to at least one embodiment. 
         FIG. 8A  is a YZ-plane view of a vision region when viewed from an X-direction according to at least one embodiment. 
         FIG. 8B  is an XZ-plane view of the vision region when viewed from a Y-direction according to at least one embodiment. 
         FIG. 9  is a flow chart of a process for implementing a function of the HMD system according to at least one embodiment. 
         FIG. 10  is a view illustrating a virtual space according to at least one embodiment. 
         FIG. 11A  is a view illustrating a page turn-over operation performed on display objects according to at least one embodiment. 
         FIG. 11B  is a view illustrating a page turn-over operation performed on display objects according to at least one embodiment. 
         FIG. 11C  is a view illustrating a page turn-over operation performed on display objects according to at least one embodiment. 
         FIG. 11D  is a view illustrating a page turn-over operation performed on display objects according to at least one embodiment. 
         FIG. 12  is a view illustrating a page turn-over operation performed on display objects according to at least one embodiment. 
         FIG. 13A  is a view illustrating a relationship between a main image and sub-images according to at least one embodiment. 
         FIG. 13B  is a view illustrating a relationship between a main image and sub-images according to at least one embodiment. 
         FIG. 14  is a view illustrating an example of a display object management table according to at least one embodiment. 
         FIG. 15  is a flow chart of a process for implementing a function of the HMD system according to at least one embodiment. 
         FIG. 16A  is a view illustrating a placement angle of the main image according to at least one embodiment. 
         FIG. 16B  is a view illustrating a placement angle of the main image according to at least one embodiment. 
         FIG. 16C  is a view illustrating a placement angle of the main image according to at least one embodiment. 
         FIG. 17  is a flow chart of a process for implementing a function of the HMD system according to at least one embodiment. 
         FIG. 18A  is a view illustrating a relationship between movement of a basic sightline and placement of the main image according to at least one embodiment. 
         FIG. 18B  is a view illustrating a relationship between movement of a basic sightline and placement of the main image according to at least one embodiment. 
         FIG. 18C  is a view illustrating a relationship between movement of a basic sightline and placement of the main image according to at least one embodiment. 
         FIG. 19  is a flow chart of a process for implementing a function of the HMD system according to at least one embodiment. 
         FIG. 20A  is a view illustrating a page jump operation performed on a display object according to at least one embodiment. 
         FIG. 20B  is a view illustrating a page jump operation performed on a display object according to at least one embodiment. 
     
    
    
     DETAILED DESCRIPTION 
     First, at least one embodiment will be discussed through a list form. A program for controlling an HMD in accordance with at least one embodiment includes the following configurations. 
     (Item 1) A program for providing, to a head-mounted display, a virtual space into which a user is immersed, the program causing a computer to execute the steps of: generating a virtual space into which a user is immersed; generating a plurality of display objects having a sequence for making up a given content in the virtual space, and then placing the plurality of display objects in the virtual space according to the sequence; in response to a user&#39;s operation, causing a selected one of the plurality of display objects to be displayed in a main view with an emphasis in the virtual space; and outputting the virtual space to the head-mounted display. 
     (Item 2) The program according to item 1, wherein the virtual space is generated as a panoramic image formed along a hemispherical plane thereof, the plurality of display objects are placed in a manner like a spiral along the hemispherical plane, and the display objects placed adjacent to each other in a circumferential direction of the spiral are consecutive display objects. 
     (Item 3) The program according to item 1 or 2, further causing the computer to execute the steps of: accepting a first action from the user; and, in response to the first action, causing the display object previously displayed in the main view to be displayed in one of sub-views without an emphasis in the virtual space, and causing the display object preceding to or subsequent to the display object previously displayed in the main view to be displayed in the main view in the virtual space. 
     (Item 4) The program according to item 3, further causing the computer to execute the steps of: determining a basic sightline of the user; and making a shift to a first mode in which the basic sightline is associated with a main image displayed in the main view, wherein the first action is accepted when a given user&#39;s operation is accepted in the first mode. 
     (Item 5) The program according to item 4, wherein in a case where a movement of the basic sightline occurs in the first mode, the main image is moved from one location to another location in pursuit of the basic sightline, and none of the display objects is placed in the location from which the main image is moved in the virtual space. 
     (Item 6) The program according to any one of items 1 to 5, further causing the computer to execute the steps of: accepting a second action from the user; and, in response to the second action, causing a selected one of the display objects displayed in the sub-views without the emphasis in the virtual space to be displayed in the main view in the virtual space, and causing the display object previously displayed in the main view to be displayed in one of the sub-views in the virtual space. 
     (Item 7) The program according to item 6, further causing the computer to execute the steps of: determining the basic sightline of the user; and making a shift to a second mode in which the basic sightline is not associated with a main image displayed in the main view, wherein the second action is accepted when a given user&#39;s operation is accepted in the second mode. 
     (Item 8) The program according to any one of items 1 to 7, further causing the computer to execute the steps of: determining the basic sightline of the user; making a shift to the first mode in which the basic sightline is associated with a main image displayed in the main view; and making a shift to the second mode in which the basic sightline is not associated with the main image displayed in the main view, wherein a mode is switchable between the first mode and the second mode. 
     (Item 9) The program according to any one of items 1 to 8, further causing the computer to execute the steps of: causing the display objects other than the display object displayed in the main view to be displayed in the sub-views without the emphasis in the virtual space; causing a selected one of the display objects displayed in the sub-views in the virtual space to be displayed in the main view in the virtual space; causing the display object previously displayed in the main view to be displayed in one of the sub-views in the virtual space, while updating displays of the display objects previously displayed in the sub-views in the virtual space; and in updating displays of the display objects previously displayed in the sub-views in the virtual space, performing processing for reducing visual effects on content items shown by the display objects. 
     (Item 10) A program for providing, to a head-mounted display, a virtual space into which a user is immersed, the program causing a computer to execute the steps of: generating a virtual space into which a user is immersed so as to contain a virtual space image, which serves as a background, and display objects superimposed on the virtual space image; updating positions or displays of the display objects; and, in updating the positions or displays of the display objects, performing processing for reducing visual effects of the display objects. 
     (Item 11) The program according to item 10, wherein the virtual space image is composed of at least one color. 
     (Item 12) The program according to item 11, wherein the virtual space image is composed of a single color. 
     (Item 13) The program according to item 11, wherein the virtual space image is composed of a plurality of colors in gradation. 
     (Item 14) The program according to any one of items 10 to 13, wherein during the processing for reducing visual effects of the display objects, a visual effect of the virtual space image is not reduced. 
     (Item 15) The program according to any one of items 10 to 14, wherein the virtual space is generated as a panoramic image formed along a hemispherical plane thereof, the plurality of display objects have a sequence for making up a given content and are placed in a manner like a spiral along the hemispherical plane, and the display objects placed adjacent to each other in a circumferential direction of the spiral are consecutive display objects. 
     (Item 16) A program for providing, to a head-mounted display, a virtual space into which a user is immersed, the program causing a computer to execute the steps of: generating a virtual space into which the user is immersed; determining a basic sightline of the user; selecting a main image from among a plurality of display objects; causing the main image to be displayed in the virtual space in such a manner as to be associated with the basic sightline; and outputting the virtual space to the head-mounted display. 
     (Item 17) The program according to item 16, wherein the main image contains two or more of the display objects making up a given content, wherein the two or more of the display objects are consecutive display objects. 
     (Item 18) The program according to item 16 or 17, wherein the basic sightline moves in response to a motion of the head-mounted display in a pitch direction, in a yaw direction, and in a roll direction, and the main image is so placed as to be tilted such that the basic sightline forms an angle of smaller than 90° with the main image in a plane including the yaw direction and the roll direction. 
     (Item 19) The program according to item 18, wherein the main image is so placed as to be tilted such that the basic sightline forms an angle of larger than 60° but smaller than 85° with the main image in the plane including the yaw direction and the roll direction. 
     (Item 20) The program according to any one of items 16 to 19, wherein the basic sightline moves in response to a motion of the head-mounted display in a pitch direction, in a yaw direction, and in a roll direction, and the main image is substantially orthogonal to the basic sightline in a plane including the pitch direction and the roll direction. 
     (Item 21) The program according to any one of items 18 to 20, wherein the main image is moved in response to the movement of the basic sightline, while maintaining an angular relationship between the basic sightline and the main image. 
     (Item 22) The program according to any one of items 16 to 21, wherein the basic sightline moves in response to a motion of the head-mounted display in a pitch direction, in a yaw direction, and in a roll direction, the basic sightline intersects a basic position of the main image, and the basic position is located at a position shifted in the yaw direction from a center of the main image. 
     (Item 23) The program according to any one of items 16 to 22, wherein a starting point of the basic sightline intersects a normal to the main image. 
     (Item 24) The program according to any one of items 16 to 23, further causing the computer to execute the steps of: setting a basic region based on the main image; and in a case where the basic sightline is moved to a position falling outside the basic region, moving the main image so that the basic sightline is located within the basic region. 
     (Item 25) 
     The program according to item 24, wherein the basic region is so set as to encompass the main image. 
     DETAILS OF EMBODIMENTS 
     The following will discuss at least one example of a program for controlling a head-mounted display system in accordance with at least one embodiment and at least one example of a method for providing a virtual space, the method executed by a computer under the control of the program, in accordance with at least one embodiment, with reference to the accompanying drawings. The present disclosure is not limited to the illustrative description. The present disclosure is indicated by the appended claims and all changes made within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. In the following description, members that are identical to their respective corresponding members illustrated in the drawings are assigned equivalent symbols. Duplicate explanations of such members are omitted. 
       FIG. 1  is a view illustrating an HMD system  100  in accordance with at least one embodiment, the HMD system  100  including an HMD  110 . The HMD system  100  includes the HMD  110  worn by a user around his/her head, a control circuit section  120 , a motion sensor  130 , a gaze sensor  140 , and an external controller  150 . 
     The HMD  110  includes: a display  112 , which is a non-transmissive display device; a sensor section  114 ; and the gaze sensor  140 . The control circuit section  120  causes an image for a right eye and an image for a left eye to be displayed on the display  112  so that a three-dimensional image having a binocular parallax is provided as a virtual space. With the display  112  placed in front of the eyes of the user, the user can be immersed in the virtual space. The virtual space contains, for example, a background, various objects that can be manipulated by the user, and various menu images. 
     The display  112  may include a sub-display for a right eye and a sub-display for a left eye, on which the image for a right eye and the image for a left eye are to be displayed, respectively. 
     Alternatively, the display  112  may be realized by one display device, provided that the image for a right eye and the image for a left eye can be provided thereon. For example, shutters may be alternately opened and closed at a high speed so that an image displayed on the display  112  is perceived by either a right eye or a left eye of the user. In this manner, the image for a right eye and the image for a left eye can be provided separately. 
     The control circuit section  120  is a computer that is connected to the HMD  110 . As illustrated in  FIG. 2 , the control circuit section  120  includes a processing circuit, a memory, a storage medium, an input-output interface, and a communication interface, all of which are connected to one another via a bus that serves as a data transmission path. The processing circuit is configured to include various types of processing circuits such as a central processing unit (CPU), a micro-processing unit (MPU), and a graphics processing unit (GPU). The processing circuit has functions of controlling both the control circuit section  120  and the whole HMD system  100 . The memory is configured to include a read only memory (ROM), a random access memory (RAM), etc. and temporarily stores control data including programs to be used by the processing circuit and operation parameters. The storage medium is configured to include a non-volatile storage device such as flash memory and a hard disc drive (HDD) and stores various kinds of images, data of objects, a simulation program, and a user authentication program. The storage medium may include a database containing a table for management of various kinds of data. The input-output interface is configured to include various wired connection terminals, such as a universal serial bus (USB) terminal, a digital visual interface (DVI) terminal, and a high-definition multimedia interface (HDMI (registered trademark)) terminal, and various processing circuits for wireless connections. The input-output interface is connected to, for example, the HMD  110 , various sensors including the motion sensor  130 , and an external controller. The communication interface is configured to include various wired connection terminals for performing communications with an external device via a network NW and various processing circuits for wireless connections and is configured to conform to various communication standards and protocols for performing communications via a local area network (LAN) and the Internet. 
     The control circuit section  120  causes a given application, which is stored in the memory or in the storage medium, to be run so that a virtual space is presented on the display  112 . The memory and the storage medium store programs for manipulating various objects displayed in the virtual space and displaying and controlling, for example, various menu images. The control circuit section  120  is not necessarily provided in the HMD  110 . Alternatively, the control circuit section  120  may be configured as a separate hardware (e.g., a known personal computer or a server computer using a network). Further alternatively, some of the functions of the control circuit section  120  may be implemented by the HMD  110 , and the remaining functions of the control circuit section  120  may be implemented by another piece of hardware. 
     The motion sensor  130  detects information on position and tilt of the HMD  110 . The motion sensor  130  includes the sensor section  114  and a sensing section  132 . The sensor section  114  may contain a plurality of light sources. The light sources are, for example, LEDs that emit infrared rays. The sensing section  132  is, for example, an infrared sensor. The sensing section  132  senses, as a sensing point of the HMD  110 , infrared rays emitted from the light sources to continuously detect pieces of position and angle information of the HMD  110  in a real space in response to a motion of the user. Then, the motion sensor  130  can obtain, based on a change of the information detected by the sensing section  132  over time, a change in position and angle of the HMD  110  over time to detect information on a motion of the HMD  110 . 
     With reference to  FIG. 3 , the following will discuss the position and tilt information obtained by the motion sensor  130 . A three-dimensional coordinate system is specified on the assumption that a head of the user who wears the HMD  110  is a center thereof. A perpendicular direction in which the user stands upright is assumed to be a yaw direction. A longitudinal direction, which is orthogonal to the yaw direction, connecting the center of the display  112  and the user is assumed to be a roll direction. A lateral direction which is orthogonal to the yaw direction and the roll direction is assumed to be a pitch direction. With such an arrangement, a change in position of the user in the three-dimensional space over time is obtained. Further, a pitch angle as a tilt angle of the HMD  110  about the pitch direction, a yaw angle as a tile angle of the HMD  110  about the yaw direction, and a roll angle as a tile angle of the HMD  110  about the roll direction are obtained. 
     Alternatively, the motion sensor  130  may include only one of the sensor section  114 , which is fixed near the display  112 , and the sensing section  132 . The sensor section  114  may include a geomagnetic sensor, an acceleration sensor, and a gyro sensor. By using at least one of these sensors, the sensor section  114  detects a position and tilt of the HMD  110  (the display  112 , in particular) worn by the user around his/her head. This enables detection of information on a motion of the HMD  110 . For example, an angular velocity sensor is capable of continuously detecting angular velocities of the HMD  110  about three axes in response to a motion of the HMD  110  to determine a change in angle about each axis over time. Such a configuration eliminates the need for the sensing section  132 . The sensing section  132  may be configured to include an optical camera. Such a configuration allows the sensing section  132  to detect the information on a motion of the HMD  110  based on image information and thus eliminates the need for the sensor section  114 . 
     The function of detecting position and tilt information of the HMD  110  by using the motion sensor  130  is referred to as “position tracking”. The relationship between the position tracking performed by the motion sensor  130  and a virtual camera  1  placed in the virtual space  2  will be discussed with reference to  FIG. 4 . For the sake of explanation of the positions of the virtual camera  1  and the motion sensor  130  in relation to each other, the following discussion will assume that in a case where the motion sensor  130  includes the sensing section  132 , the position of the motion sensor  130  corresponds to that of the sensing section  132 , and in a case where the motion sensor  130  does not include the sensing section  132 , the position of the motion sensor  130  corresponds to that of the sensor section  114 . 
     The virtual camera  1  is placed inside the virtual space  2 , and the motion sensor  130  is placed outside the virtual space  2  (in a real space). 
     The virtual space  2  is formed in the shape of a hemisphere and includes a plurality of meshes that are substantially square or substantially rectangular in shape. Each of the meshes is associated with spatial information of the virtual space  2 , and a vision region  23  is defined based on the spatial information. In the present embodiment, it is preferable that a center  21  of the hemisphere is so adjusted as to be constantly located, in a XZ-plane, on a line connecting the virtual camera  1  and the sensor  130 . For example, the virtual camera  1  may be constantly located on the center  21 . Further, in a case where the position of the virtual camera  1  is moved in an X-direction as the user wearing the HMD  110  moves, the area of the virtual space  2  may be altered so that the center  21  is positioned on the line connecting the virtual camera  1  and the motion sensor  130 . In these cases, the position of the virtual camera  1  is fixed in the virtual space  2 , but only the tilt of the virtual camera  1  is varied. Alternatively, the position of the virtual camera  1  may be moved in response to a movement of the motion sensor  130  in an X-direction, in a Y-direction, and in a Z-direction. Such a configuration allows for variable position settings of the virtual camera  1  in the virtual space  2 . 
     The gaze sensor  140  has an eye tracking function of detecting a direction in which sightlines of a right eye and a left eye of the user are pointed. The gaze sensor  140  preferably includes a sensor for a right eye and a sensor for a left eye. The sensor for a right eye detects a direction in which a sightline of a right eye of the user is pointed, and the sensor for a left eye detects a direction in which a sightline of a left eye of the user is pointed. In this manner, the gaze sensor  140  detects a sightline direction in which the user gazes. The gaze sensor  140  can be a known sensor having the eye tracking function. For example, the gaze sensor  140  may apply infrared light to a right eye and a left eye and then obtain light reflected off corneas or irises of the eyes to determine rotation angles of their eyeballs. 
     As illustrated in  FIG. 5 , the gaze sensor  140  detects sightline directions of a right eye R and a left eye L of a user U. In a case where the user U is looking somewhere near the user U, sightlines R 1  and L 1  are detected. Then, a gaze point N 1 , which is a point of intersection of the sightlines R 1  and L 1 , is determined. Meanwhile, in a case where the user U is looking somewhere distance from the user U, sightlines R 2  and L 2 , which form smaller angles with the roll direction than angles formed by the sightlines R 1  and L 1  with the roll direction, are detected. When the gaze point N 1  is determined, a sightline direction N 0  of the user U is determined accordingly. The sightline direction N 0  is a direction in which the user U is actually looking with his/her two eyes. The sightline direction N 0  is defined as, for example, a direction in which a straight line passing through a midpoint between the right eye R and the left eye L of the user U and the gaze point N 1  extends. 
     The external controller  150  is a device capable of performing communications through which various instructions can be sent to the control circuit section  120 , and may be constituted by a portable terminal capable of performing wireless communications. The external controller  150  can be any portable device including a processing circuit, a memory, a storage medium, a communication section, a display section, and an input section, all of which are connected to one another via a bus. Examples of the external controller  150  include a smartphone, a PDA, a tablet computer, a game console, and a notebook PC. The external controller  150  is preferably a portable terminal equipped with a touch panel. The user can exert an influence on an object displayed in the virtual space by performing any of various touching actions, including a tap, a swipe, and a hold, on the touch panel of the external controller  150 . 
     Any of the constituent members of the HMD system  100  may include a microphone-equipped headphone. Such a configuration allows the user to give voice instructions to a given object which is shown inside the virtual space. Further, any of the constituent members of the HMD system  100  may include a television receiver in order that a virtual television set provided inside the virtual space can receive a broadcast of a television program. Still further, any of the constituent members of the HMD system  100  may include a feature, such as a communication capability, in order that an e-mail and/or the like obtained by the user can be displayed. 
       FIG. 6  is a functional block diagram of the control circuit section  120  for implementing the functions of the HMD system  100 , i.e., the processing to display the virtual space  2 , the display of various menus in the virtual space  2 , and the manipulation of objects. The control circuit section  120  controls images to be outputted to the display  112 , based mainly on inputs from the motion sensor  130 , the gaze sensor  140 , and the external controller  150 . 
     The control circuit section  120  includes a display control section  200 , an object control section  300 , a communication control section  400 , and a storage section  500 . The display control section  200  includes a virtual space image generating section  210 , a HMD motion detecting section  220 , a sightline detecting section  230 , a basic sightline determining section  240 , a vision region determining section  250 , and a vision image generating section  260 . The object control section  300  includes an object selecting section  310 , a display form determining section  320 , a display adjusting section  330 , a mode setting section  340 , a judging section  350 , and a drawing section  360 . The communication control section  400  includes a requesting section  410  configured to request an external device  160  for various kinds data via a network NW and a communication processing section  420  configured to handle communications between the requesting section  410  and the external device. The storage section  500  includes a spatial information storing section  510 , an object information storing section  520 , and a user information storing section  530 . The storage section  500  stores various kinds of data that are required for operations to supply output information to the display  112  in response to inputs from the motion sensor  130 , the gaze sensor  140 , and the external controller  150 . 
     A flow of a process performed by the HMD system  100  to provide the virtual space  2  will be discussed with reference to  FIGS. 6 and 7  according to at least one embodiment. The virtual space  2  can be provided through interactions between the HMD  110  (the gaze sensor  140  and the motion sensor  130 ) and the control circuit section  120 . 
     First, the control circuit section  120  (virtual space image generating section  210 ) refers to the spatial information storing section  510  to generate a hemispherical virtual space image  22  that forms the virtual space  2  into which the user is immersed (S 120 - 1 ). When a motion of the user, such as a movement and a tilt, is inputted to the HMD  110  (S 110 - 1 ), the motion sensor  130  detects the position and tilt of the HMD  110  (S 130 - 1 ). Pieces of information obtained by the detection made by the motion sensor  130  are transmitted to the control circuit section  120 . Subsequently, the HMD motion detecting section  220  accepts position information and tilt information of the HMD  110 . Then, a vision direction is determined based on the position information and tilt information of the HMD  110  (S 120 - 2 ). 
     When the gaze sensor  140  detects motions of left and right eyeballs of the user (S 140 - 1 ), information indicative of directions in which sightlines of left and right eyes are pointed is transmitted to the control circuit section  120 . The sightline detecting section  230  determines the directions in which sightlines of the left and right eyes are pointed and then determines the sightline direction N 0  (S 120 - 3 ). The basic sightline determining section  240  determines the vision direction determined based on the tilt of the HMD  110  or the sightline direction N 0  of the user to be a basic sightline  5  (S 120 - 4 ). 
     The vision region determining section  250  determines the vision region  23  of the virtual camera  1  in the virtual space  2  (S 120 - 5 ). As illustrated in  FIG. 4 , the vision region  23  is an area corresponding to a user&#39;s vision in the virtual space image  22 . The vision region  23  is determined based on the basic sightline  5 , which is determined based on the position and tilt of the virtual camera  1 .  FIG. 8A  is a YZ-plane view of the vision region  23  when viewed from an X-direction, and  FIG. 8B  is an XZ-plane view of the vision region  23  when viewed from a Y-direction. 
     The vision region  23  has a first vision region  24  (see  FIG. 8A ), which is a range defined by the basic sightline  5  and a YZ-section of the virtual space image  22 , and a second vision region  25  (see  FIG. 8B ), which is a range defined by the basic sightline  5  and a XZ-section of the virtual space image  22 . The first vision region  24  is set as a range including a polar angle a formed about the basic sightline  5 . The second vision region  25  is set as a range including an azimuth angle β formed about the basic sightline  5 . 
     The vision image generating section  260  generates a vision image  26  based on the vision region  23  (S 120 - 6 ). The vision image  26  includes two two-dimensional images, i.e., a two-dimensional image for a left eye and a two-dimensional image for a right eye. These two-dimensional images are superimposed on the display  112 , so that a three-dimensional image as the virtual space  2  is provided to the user. The HMD  110  receives information on the vision image  26  from the control circuit section  120  and then causes the vision image  26  to be displayed on the display  112  (S 110 - 2 ). 
     At least one example of a virtual space to be provided to the user and a flow of a process performed by the HMD system  100  to provide the virtual space to the user will be discussed with reference to  FIG. 6 ,  FIG. 9 , and the subsequent drawings. First, the display control section  200  refers to the spatial information storing section  510  and the object information storing section  520  to generate the virtual space  2  as illustrated in  FIG. 10  and display objects DO (S 120 - 7 ). In at least one embodiment, a plurality of display objects DO makes up a content, like a comic, containing characters and illustrations, and include individual pages for making up the content. That is, a plurality of display objects DO have a sequence based on a story of the content. Note, however, that the display objects DO may include a character object C that is independent of the sequence of the content. When the object selecting section  310  selects a plurality of display objects DO which are to be placed in the virtual space  2 , the display form determining section  320  causes the plurality of display objects DO to be placed in a manner like a spiral along a hemispherical plane of the virtual space  2  (S 120 - 8 ). As illustrated in  FIG. 10  and  FIGS. 11A to 11D , a plurality of display objects DO are placed in accordance with the sequence of the content in a circumferential direction so as to draw a spiral in a Y-direction, the spiral starting from a portion close to a bottom of the hemispherical plane to a top of the hemispherical plane. That is, the display objects DO placed adjacent to one another in the circumferential direction of the spiral are display objects DO on consecutive pages in the story. In an initial state, the display objects DO placed in a manner like a spiral are placed as sub-images S, which are displayed in sub-views without an emphasis or other effect. 
     The display object DO is displayed in such a manner as to be superimposed on the virtual space image  22 . In the present embodiment, the virtual space image  22  is composed of at least one color so that the user can concentrate on a content in the virtual space  2 . For example, the virtual space image  22  may be composed of a single color or may be composed of a plurality of colors in gradation. 
     The mode setting section  340  sets control performed by the control circuit section  120  to a first mode (S 120 - 9 ). In the present embodiment, the mode setting section  340  sets a mode in an initial state to the first mode. As discussed later, the first mode is a control mode in which a main image M, which is a selected one of the display objects DO, is displayed in a main view with an emphasis in the virtual space  2  in such a manner as to be associated with the basic sightline  5 . 
     When an input of a given motion (e.g., variations in position and angle of the HMD  110 ) is made from the HMD  110  (S 110 - 3 ), the basic sightline determining section  240  determines the basic sightline  5 . Then, the object selecting section  310  selects, from among the display objects DO, a main image M to be displayed in the virtual space in the initial state in such a manner as to be associated with the basic sightline  5  (S 120 - 10 ). In a case where the user views the content for the first time, a display object DO corresponding to a top page of the content is selected as the main image M. Further, in a case where a bookmark is set on a last viewed page at the previous viewing of the content, a display object DO corresponding to the last viewed page is selected as the main image M. The display form determining section  320  determines that the display object DO is to be displayed as the main image M in the virtual space  2 . The display adjusting section  330  sets a placement position and placement angle of the main image M in the virtual space  2  (discussed in detail below). 
     The object control section  300  outputs, to the display control section  200 , information on a display form of the display object DO. The vision image generating section  260  outputs, to the HMD  110 , information on the virtual space  2  to be produced by the virtual space image  22  having the display object DO displayed thereon. Upon receipt of the information on the virtual space  2 , the HMD  110  causes the virtual space  2  to be displayed on the display  112  (S 110 - 4 ), thereby providing the virtual space  2  into which the user is immersed. As illustrated in  FIG. 10 , the virtual space  2  into which the user is immersed is preferably such that the virtual camera  1  is located on the center  21  of the hemisphere, and that the main image M and the sub-image S are contained within the vision region  23 , which is set based on the basic sightline  5 . This allows the user to have a new experience that the user goes on reading the comic in such an environment that the user is surrounded by individual pages of the comic. 
     The user can input a first action in the virtual space  2  (S 110 - 5 ). The first action is an instruction for updating a display object DO being displayed as the main image M in the first mode. The first action may be a given motion of the HMD  110  (an action of shaking a head to the left or to the right), a given sightline movement obtained by the gaze sensor  140  (an action of moving eyes to the left or to the right), or an input from the external controller  150  (a swipe to the left or to the right). 
     In a case where the control circuit section  120  is in the first mode, the first action enables turning over a page shown in the main image M (S 120 - 11 , S 120 - 12 ). For example, in a case where the user inputs an operation corresponding to a movement to the right (by shaking the head to the right, by moving the eyes to the right, or by the swipe to the right), an image being displayed as a main image in  FIG. 11A  is shifted to the right by one position and is changed into the sub-image, as illustrated in  FIG. 11B  (S 120 - 13 ). Further, a sub-image  10 , which is located immediately on the left side of the main image in  FIG. 11A , is shifted to the right by one position and is changed into the main image (S  120 - 14 ). Meanwhile, all the other sub-images are shifted to the right by one position while remaining unchanged as the sub-images. In this manner, the page shown in the main image M is updated to the subsequent page. 
     In a case where the user inputs an operation corresponding to a movement to the left (by shaking the head to the left, by moving the eyes to the left, or by the swipe to the left), an image being displayed as a main image in  FIG. 11C  is shifted to the left by one position and is changed into the sub-image, as illustrated in  FIG. 11D  (S 120 - 13 ). Further, a sub-image  9 , which is located immediately on the right side of the main image in  FIG. 11C , is shifted to the left by one position and is changed into the main image (S 120 - 14 ). Meanwhile, all the other sub-images are shifted to the left by one position while remaining unchanged as the sub-images. In this manner, the page shown in the main image M is updated to the preceding page. 
     In updating the page shown in the main image M, the display adjusting section  330  may move the positions of the display objects DO or may update only images to be displayed without moving the positions of the display objects DO. In these cases, the display adjusting section  330  may perform processing for reducing visual effects on content items shown by the display objects DO before the positions of the display objects DO are moved or before the content items shown by the display objects DO are updated, as illustrated in  FIG. 12 . The processing for reducing the visual effects may be processing for disabling the content items recognizable. Example of such processing includes processing of blurring an image as illustrated in  FIG. 12 , processing of decreasing a resolution, and processing of creating a smoke effect. This allows for reduction of visually induced motion sickness (so-called virtual reality (VR) sickness) which is caused when the user experiences a visual effect that occurs independent of a motion of the HMD  110 . 
     In the present embodiment, the virtual space image  22  is composed of at least one color. In such a case, it is preferable that the processing for reducing a visual effect is not performed on the virtual space image  22 . The virtual space image  22  composed of at least one color originally has a small amount of information on a visual effect. For this reason, the user is less affected by the visual effect that occurs independent of a motion of the HMD  110 . As such, the user is less likely to suffer from the VR sickness. Thus, performing the processing for reducing a visual effect only on a display object DO enables reduction of a processing load on the control circuit section  120 . 
       FIGS. 13A, 13B, and 14  illustrate examples of the main image M selected by the object selecting section  310  and a relationship between the main image M and the sub-images S, which relationship is determined by the display adjusting section  330 . As illustrated in  FIGS. 13A and 13B , the virtual camera  1  is located at the center of a region enclosed with the sub-images, and a main image is placed farther forward to the front of the virtual camera  1  than sub-images are, so that the main image is recognized by the user as being displayed with an emphasis so as to be scaled up. At this time, the display adjusting section  330  can place the main image M forward and slightly scale up the main image M (e.g., from 1.2 times to 1.5 times of the original size) so that the main image M is displayed with a further emphasis. In the present embodiment, it is preferable that the user is positioned constantly at the center  21  of the virtual space  2 . In this situation, it is assumed that only the orientation of the basic sightline  5  is changed in response to a motion of the HMD  110 . Furthermore, it is preferable that, in response to a change in orientation of the basic sightline  5 , all of the sub-images S and the main image M are so placed as to be orthogonal to the basic sightline  5  in a horizontal plane (XZ-plane). 
       FIG. 14  illustrates an example of a display object management table T 1  stored in the object information storing section  520 . The display object management table T 1  stores page numbers of pages that make up a content and also stores their associated image files, image sizes, and display styles. The display form determining section  320  refers to the display object management table T 1 . In a case where the display style is set to “Normal” in the display object management table T 1 , the display form determining section  320  causes only a single display object DO to be shown in the main image M, as illustrated in  FIG. 13A . In contrast, in a case where the display style is set to “Spread” in the display object management table T 1 , the display form determining section  320  causes two or more consecutive display objects DO to be displayed in the main image M, as illustrated in  FIG. 13B . In this case, the size of the main image M is set to a sum of sizes of the plurality of display objects DO. This allows the user to enjoy viewing visually impactful spread pages. 
     A display angle of the main image M, which display angle is set by the display adjusting section  330 , will be discussed with reference to  FIG. 15  and  FIGS. 16A to 16C . When a motion (information on the basic sightline  5 ) is inputted from the HMD  110  (S 110 - 3 ) in a situation where the first mode is set by the mode setting section  340  (S 120 - 9 ), the object selecting section  310  selects a display object DO to be displayed as the main image M (S 120 - 10 ). The display adjusting section  330  determines a basic position BP of the selected main image M (S 120 - 15 ). As illustrated in  FIG. 16A , the basic position BP, which is a given position included in the main image M, serves as a reference for a setting of an initial position of a sub-image as displayed in a main view. In the present embodiment, the basic position BP is located at a center of the main image M in a pitch direction and is set to a position shifted in a yaw direction from a center of the main image M in the yaw direction (at any position that falls within the basic sightline range BR). In an initial state, the position of the main image M is so set that the basic sightline  5  intersects the basic position BP. Further, the angle of the main image M is so set that a normal to the main image M intersects a starting point P, which is a point of intersection of the basic sightline  5  and the virtual camera  1  (S 120 - 16 ). 
     In S 120 - 16 , the display adjusting section  330  first sets the position of the main image M to such a position that the basic sightline  5  meets the basic position BP (to such a position that the basic sightline  5  intersects the basic position BP) (S 120 - 17 ). Then, as illustrated in  FIG. 16B , the display adjusting section  330  determines an angle which the basic sightline  5  forms with the main image M in a horizontal plane (a plane including a pitch direction and a roll direction). In the present embodiment, the position and angle of the main image M are so set that the basic sightline  5  is substantially orthogonal to the main image M. Subsequently, as illustrated in  FIG. 16C , the display adjusting section  330  determines an angle which the basic sightline  5  forms with the main image M in a vertical plane (a plane including a yaw direction and a roll direction). In the present embodiment, the display adjusting section  330  sets the angle which the basic sightline  5  forms with the main image M in the vertical plane, by rotating the main image M by an angle θ about a pitch axis so that a lower end of the main image M is closer to the virtual camera  1  side. In at least one embodiment, the angle θ is an angle at which the basic sightline  5  forms an angle of smaller than  90 ° with the main image M, and in at least one embodiment an angle at which the basic sightline  5  forms an angle of larger than 60° but smaller than 85° with the main image M. The normal to the main image M may intersect the starting point P of the basic sightline  5  in the vertical plane. 
     The drawing section  360  draws the main image M with its position and angle set in the manner as discussed above and then outputs the main image M to the display control section  200 . Subsequently, the display control section  200  outputs the main image M to the HMD  110  (S 110 - 4 ). This allows the user to view the main image M which is orthogonal to the direction of the basic sightline  5  in a horizontal plane and to view the main image M which is slightly tilted upward relative to the direction of the basic sightline  5  in a vertical plane. In the present embodiment, it is preferable that the basic sightline  5  is defined as a vision direction which is detected according to a tile of the HMD  110 . A line of sight of the user tends to gradually go a little downward relative to the roll direction, which is a direction that the user&#39;s head faces the front. Since the main image M is placed at the position and angle as discussed above, the main image M viewed by the user when a natural sightline direction (gaze direction GD) of the user points to the main image M can be angled so as to be substantially perpendicular to the sightline direction. Further, the basic position BP may be so set that the normal to the main image M intersects the starting point P of the basic sightline  5 . With this arrangement, the main image M can be naturally angled so as to be perpendicular in a vertical plane when the user views the main image M. Note that the angle  0  which the basic sightline  5  forms with the main image M may be altered by, for example, changing the basic position BP to suit a preference of the user. In this arrangement, the display adjusting section  330  may adjust the angle θ by rotating the main image M about a pitch axis that passes through the center of the main image M. With this arrangement, the virtual space  2  can be configured to be adapted to differences among individuals in the amount of angle by which the gaze direction of the user points downward and to suit a preference of the user in angle at which the user gazes the main image M. 
     Next, processing performed when a change of the basic sightline  5  of the user is inputted after the main image M is displayed will be discussed with reference to  FIG. 17  and  FIGS. 18A to 18C . When a change of the basic sightline  5  is inputted in response to a change in motion of the HMD  110  (S 110 - 5 ), the judging section  350  determines a basic region BRE (S 120 - 19 ). In the present embodiment, the basic region BRE is set as a region that encompasses the main image M. The judging section  350  judges whether the basic sightline  5  intersects the basic region BRE (S 120 - 20 ). 
     In a case where the basic sightline  5  intersects the basic region BRE as illustrated in  FIG. 18A , the display adjusting section  330  skips adjustment of the placement position and angle of the main image M. With this arrangement, the position and angle of the main image M remain unchanged even in the event of the occurrence of a slight change in tilt of the HMD  110  when the user points his/her sightline at the corner of the main image M. This allows the user to easily view the whole content. 
     In contrast, in a case where the basic sightline  5  does not intersect the basic region BRE as illustrated in  FIG. 18C , the display adjusting section  330  adjusts the placement position and placement angle of the main image M. First, as illustrated in  FIG. 18B , the display adjusting section  330  moves the position of the main image M so that the basic sightline  5  intersects the basic region BRE (S 120 - 21 ). In S 120 - 21 , it is preferable that the display adjusting section  330  moves the main image M to such a position that the basic sightline  5  intersects the basic position BP. Further, the display adjusting section  330  adjusts an angular relationship between the basic sightline  5  and the main image M as the main image M has been moved (S 120 - 22 ). Specifically, in order to have the angular relationship as illustrated in  FIGS. 16A to 16C , the display adjusting section  330  performs a step of making the basic position BP and the basic sightline  5  (S 120 - 23 ) meet each other, a step of adjusting an angle between the basic sightline  5  and the main image M in a horizontal phase (S 120 - 24 ), and a step of adjusting an angle between the basic sightline  5  and the main image M in a vertical phase (S 120 - 25 ). In S 120 - 21 , the main image M may be moved in response to the movement of the basic sightline  5 , while maintaining an angular relationship between the basic sightline  5  and the main image M. 
     The drawing section  360  draws the main image M with its position and angle set as discussed above and then outputs the main image M to the display control section  200 . Subsequently, the display control section  200  outputs the main image M to the HMD  110  (S 110 - 6 ). With this arrangement, the main image M can be displayed in pursuit of the basic sightline  5  that moves in response to a user&#39;s relatively large motion to change his/her posture. This allows the user to enjoy the content with ease in a posture that the user likes. 
     The following will discuss a second mode set by the mode setting section  340  with reference to  FIGS. 19, 20A, and 20B . When a given input from the user is accepted, the mode setting section  340  sets control performed by the control circuit section  120  to a second mode (S 120 - 26 ). Thus, as illustrated in  FIG. 20A , the display form determining section  320  changes a display form of a current main image from a main view with an emphasis to a sub-view and cancels the aforementioned association of the basic sightline  5  with the main image M (S 120 - 27 ). This allows the user to view many sub-images S collectively. 
     Subsequently, the user inputs a given second action to a target sub-image S (S 110 - 7 ). The second action is, for example, an action of pointing the basic sightline  5  (a vision direction that is shifted in response to a tilt of the HMD  110  or a sightline direction determined by the gaze sensor  140 ) at a target sub-image for a certain period of time or more. Alternatively, the second action may be an action of selecting a target sub-image S (“DESIRED IMAGE” indicated by the letter “A” in  FIG. 20A ) by using the external controller  150 . In a case where the control circuit section  120  is set to the second mode (S 120 - 28 ), the object control section  300  accepts the second action, and the object selecting section  310  identifies the display object DO (DESIRED IMAGE indicated by the letter “A”) selected by the second action (S 120 - 29 ). 
       
     As illustrated in  FIG. 20B , the display form determining section  320  sets the selected display object DO as a main image, and the display adjusting section  330  sets the position and angle of that display object DO in the manner as discussed above (S 120 - 30 ). Further, the mode setting section  340  sets processing performed by the control circuit section  120  to the first mode so that manipulations on the main image can be performed in response to a movement of the basic sightline  5  of the user (S 120 - 31 ). The drawing section  360  draws the main image M with its position and angle set in the manner as discussed above and then outputs the main image M to the display control section  200 . Subsequently, the display control section  200  outputs the main image M to the HMD  110  (S 110 - 8 ). This allows the user to perform a manipulation of a page jump to a target sub-image. After the page jump is performed, the destination sub-image can be displayed at an easy-to-read angle. 
     Although the above description has discussed the embodiments of the present disclosure, the present disclosure is not limited to the embodiments discussed above. As those skilled in the art will understand, various changes of the embodiments can be made without departing from the spirit and scope of the present disclosure in the appended claims. 
     For example, in the above embodiment, the processing performed by the control circuit section  120  is set to the first mode in the initial state. However, the processing may be set to the second mode in the initial state. Further, the first mode and the second mode may be set as necessary by a given user&#39;s input. 
     Further, in the states illustrated in  FIGS. 13A and 13B  in the first mode, at the occurrence of a movement of the basic sightline  5 , none of the display objects DO may be placed in a vacant location, in the virtual space  2 , from which the main image M has been moved to another location. This allows the user to easily know the position of a page corresponding to the main image M in the content even after the basic sightline  5  is moved. At the occurrence of a movement of the basic sightline  5 , the sub-images S may be moved to fill the vacant location, in the virtual space  2 , from which the main image M has been moved to another location. 
     REFERENCE SIGNS LIST 
       100  Head-mounted display (HMD) system 
       110  HMD 
       112  Display 
       114  Sensor section 
       120  Control circuit section 
       130  Motion sensor 
       132  Sensing section 
       140  Gaze sensor 
       150  External controller 
       200  Display control section 
       300  Object control section 
       400  Communication control section 
       500  Storage section 
       1  Virtual camera 
       2  Virtual space 
       5  Basic sightline 
       22  Virtual space image 
       23  Vision area 
       26  Vision image 
     BP Basic position 
     BR Basic sightline range 
     BRE Basic region 
     DO Display object 
     M Main image 
     S Sub-image