Patent Description:
Some methods have been known for displaying an area or object that is hidden behind an obstacle and is hardly visible when viewed from a user. For example, Patent Literature <NUM> discloses a method of taking an image of a blind area of a driver with a camera, converting the taken image into an image from the view point of the driver, and displaying the resulting image and a translucent image of a vehicle from the view point of the driver in a superimposed manner.

Document <CIT> discloses a display terminal that reduces a sense of discomfort given to a user when the state of an observation space is presented to a head-mounted display in accordance with a change in the direction of the user's head. In the display terminal, a setting unit sets an observation position, an observation direction, and an observation axis in the observation space, the observation axis intersecting an observation line extending in the observation direction from the observation position. A display unit generates a video observing the observation space in the observation direction from the observation position, and displays the generated video on a screen disposed in a real space. An acquisition unit detects a display direction in which the screen displays the video in the real space, and acquires the amount of rotation around a reference axis in the real space among changes in the display direction. An update unit updates, while the observation line is kept intersecting the observation axis, the observation position and the observation direction by rotating the observation position around the observation axis by the acquired amount of rotation around the reference axis.

Document <CIT> discloses a video presentation method that includes: a head movement measurement step for measuring operation of a head in the real space of an observer; a viewpoint position and visual line direction calculation step for calculating a viewpoint position and a visual line direction in a video space represented to the observer from the measured operation of the head; a video generation step for generating video in which the viewpoint position and the visual line direction of the observer are virtually shifted centering around a subject reflected in the video, based on the calculated viewpoint position and visual line direction; and a video representation step for representing the generated video to the observer.

Patent Literature <NUM>: Unexamined <CIT>.

In some techniques related to a virtual reality system, augmented reality system, or substitutional reality system, the system detects the orientation of a user wearing a head mounted display, changes the sight line direction in a three-dimensional space in accordance with the detected orientation, generates an image representing a view of the three-dimensional space in the sight line direction, and causes the image to be displayed on the head mounted display. In the technique disclosed in Patent Literature <NUM>, the image of the blind area and the translucent image of the vehicle are displayed in a superimposed manner on the preliminary assumption that the driver pays attention to the blind area of the driver.

If a display device has an ability to detect the position of the display device, a system is capable of display control by transferring the observation site in the virtual space in accordance with movement of the head of a user. This configuration enables the user to observe a portion of an object hidden behind another object, for example, when the user moves the head so as to look into the space behind the other object for the hidden portion.

Unfortunately, if the display device has no ability to detect the position of the display device in the real space, the system is incapable of display control in accordance with movement of the user's head and thus cannot achieve observation of the hidden portion. If the display device has an ability to detect the position of the display device in the real space with a low detection accuracy, the system may fail to transfer the observation site to an appropriate site and cannot achieve appropriate observation of the hidden portion.

The disclosure, which has been accomplished to solve the above issues, is to provide a display control device that causes a display device to display a view of a virtual space or the like from an observation site in the virtual space or the like and appropriately transfers the observation site in the virtual space or the like without detection of the position of the display device in a real space, and to provide a display control method, a program, and a non-transitory computer-readable information recording medium.

A display control device according to one aspect of the disclosure for solving the above issues controls a display device. The display control device is as defined in the appended claim <NUM>.

A display control method performed by the above-described display control device can be achieved when a computer connected to or integrated with the display device executes a program stored on a non-transitory computer-readable information recording medium or the like and functions as the display control device for controlling the display device.

This non-transitory information recording medium can be distributed and sold independently from computers. The non-transitory information recording medium indicates a tangible information recording medium. Examples of the non-transitory information recording medium include a compact disk, flexible disk, hard disk, magnetic disk, digital video disk, magnetic tape, and semiconductor memory. In contrast, a transitory information recording medium indicates a transmission medium (transmission signal) itself. Examples of the transitory recording medium include an electrical signal, optical signal, and electromagnetic wave. A temporary storage area indicates an area for temporarily storing data and programs. A typical example of the temporary storage area is a volatile memory, such as a random access memory (RAM).

According to the disclosure, a display control device, which causes a display device to display a view of a virtual space or the like from an observation site in the virtual space or the like, can appropriately transfer the observation site in the virtual space or the like without detection of the position of the display device in a real space.

An embodiment of the disclosure will now be described. This embodiment is provided for exemplary purposes only and does not limit the scope of the disclosure. Persons skilled in the art can replace some or all of the components of the embodiment with their equivalents to configure other embodiments, which are also encompassed in the scope of the disclosure. The embodiment of the disclosure will now be described with reference to the accompanying drawings, in which the identical or corresponding components are provided with the same reference symbol.

<FIG> is a schematic block diagram illustrating a hardware configuration of a display device <NUM> according to the embodiment. The display device <NUM> includes, for example, a head mounted display equipped with various sensors and controllers.

The head mounted display may be provided with a computer (for example, smartphone, tablet computer, or phablet) installed in an attachment of the head mount display. In this case, the display device <NUM> is achieved by execution of a program in the computer (for example, smartphone) for causing the computer to function as the above-described components.

Alternatively, the display device <NUM> may be configured by a head mounted display equipped with various sensors and a display screen and a computer connected to the head mounted display via wire or wireless communication for controlling the head mounted display. In this case, the computer acquires values measured by the sensors from the head mounted display, generates an image based on the measured values, and causes the image to be displayed on the display screen of the head mounted display.

With reference to <FIG>, the display device <NUM> is composed of a control unit <NUM>, a read only memory (ROM) <NUM>, a RAM <NUM>, a display <NUM>, a sensor unit <NUM>, and an operation unit <NUM>. These components are connected to each other via buses <NUM>.

The control unit <NUM> includes, for example, a central processing unit (CPU) and controls the entire display device <NUM>.

The ROM <NUM> is a non-volatile memory that stores programs and various types of data for enabling the control unit <NUM> to control the entire display device <NUM>.

The RAM <NUM> is a volatile memory that temporarily stores information generated by the control unit <NUM> and data necessary for generation of the information.

The display <NUM> includes a liquid crystal display (LCD) and backlights. The display <NUM> displays, for example, an image output from the control unit <NUM> under the control of the control unit <NUM>.

The sensor unit <NUM> is equipped with an orientation sensor and an acceleration sensor to detect the orientation of the display device <NUM>, that is, the orientation of the display <NUM>. The sensor unit <NUM> may detect a shift in orientation instead of detecting the orientation itself.

The operation unit <NUM> includes input components, such as buttons, a keyboard, and a touch panel. The operation unit <NUM> receives an operation input from a user of the display device <NUM> and outputs a signal corresponding to the received operation to the control unit <NUM>.

In the above example, the display control device according to the embodiment is configured by the control unit <NUM> that controls the display device <NUM> in cooperation with the ROM <NUM> and the RAM <NUM>. The display device <NUM> is integrated with the display control device in such a manner as to include the display control device.

Alternatively, the control unit <NUM>, the ROM <NUM>, and the RAM <NUM> may be configured by an external computer communicably connected to a head mounted display. The display <NUM> and the sensor unit <NUM> may be the constituent elements of the head mounted display connected to this computer. The operation unit <NUM> may be input components connected to this computer. In this case, the display control device according to the embodiment is achieved outside the head mounted display by cooperation of the control unit <NUM> with the ROM <NUM> and the RAM <NUM>. This display control device controls the display device <NUM> including the display <NUM> and the sensor unit <NUM> of the head mounted display.

The following description is directed to a positional relationship between the display device <NUM> and a user <NUM> in a real space <NUM> and a virtual space (three-dimensional space) <NUM> displayed in the display device <NUM> according to the embodiment. <FIG> is a top view of the real space <NUM> at a time ta before a shift in orientation of the display device <NUM> and the user <NUM>. <FIG> is a top view of the virtual space <NUM> at the time ta before the shift in orientation of the display device <NUM> and the user <NUM> in the real space <NUM>. <FIG> is a top view of the real space <NUM> at a time tb after the shift in orientation of the display device <NUM> and the user <NUM>. <FIG> is a top view of the virtual space <NUM> at the time tb after the shift in orientation of the display device <NUM> and the user <NUM> in the real space <NUM>. The embodiment will be described with reference to these figures. Although the reference symbols in the figures are provided with lower case alphabets as required for representing the elapsed time from the time ta to the time tb, these lower case alphabets are omitted as required in the following description.

With reference to <FIG> and <FIG>, the user <NUM> wears the display device <NUM> in front of the eyes in the real space <NUM>. While the user <NUM> is wearing the display device <NUM>, a display direction <NUM> of the display device <NUM> is opposite to a sight line direction of the user <NUM> looking at the display device <NUM>.

With reference to <FIG> and <FIG>, still or moving objects <NUM> to <NUM> are placed in the virtual space <NUM>. In this embodiment, the objects <NUM> and <NUM> remain still, while the object <NUM> moves in the direction of arrow A and the object <NUM> moves in the direction of arrow B in the virtual space <NUM>. The objects <NUM> to <NUM> are generated by computer graphics. The positions of the objects <NUM> to <NUM> in the virtual space <NUM>, their sizes, and shapes, and the moving velocities of the objects <NUM> and <NUM> are defined in advance. In an exemplary case where the display device <NUM> displays a station platform reproduced by computer graphics, examples of the objects <NUM> and <NUM> include trains and people moving on the station platform, while examples of the objects <NUM> and <NUM> include columns and signboards standing still on the station platform.

It should be noted that an orientation <NUM> in the real space <NUM> may be identical to or deviated from an orientation <NUM> in the virtual space <NUM>. These orientations are deviated from each other in this embodiment.

<FIG> is a schematic block diagram illustrating a functional configuration of the display device <NUM> according to the embodiment of the disclosure. As illustrated in <FIG>, the display device <NUM> includes a detector <NUM>, a changer <NUM>, a generator <NUM>, a specifier <NUM>, a display controller <NUM>, and a display <NUM>.

The detector <NUM> detects a shift in orientation of the display device <NUM> in the real space <NUM>. For example, the detector <NUM> detects a shift in the display direction <NUM> of the display device <NUM> relative to the reference axis in the real space <NUM> on the basis of the values measured by the sensor unit <NUM>. A typical example of the reference axis in the real space <NUM> is the gravity direction in the real space <NUM> or the up-down direction of the display <NUM> of the display device <NUM>.

In this embodiment, the functions of the detector <NUM> are performed by cooperation of the control unit <NUM> and the sensor unit <NUM>.

The changer <NUM> changes the sight line direction <NUM> in the virtual space <NUM> in accordance with the shift in orientation of the display device <NUM> detected by the detector <NUM>. For example, the changer <NUM> acquires a rotational angle θ about the reference axis in the real space <NUM> with reference to the shift in the display direction <NUM> of the display device <NUM> from the time ta until the time tb detected by the detector <NUM>. Specifically, before the user <NUM> turns the head, the display direction 203a extends from the display device 100a to the user 202a as illustrated in <FIG>. After the user <NUM> turns the head, the display direction 203b extends from the display device 100b to the user 202b as illustrated in <FIG>. The angle between the display direction 203a and the display direction 203b is defined as the rotational angle θ about the reference axis. The rotational angle θ corresponds to a so-called yaw angle. The user <NUM> turns the head to the left in <FIG>.

The interval between the time ta and the time td can be defined depending on the usage. For example, the interval may be a refresh interval (for example, vertical synchronization period) of the display <NUM> of the display device <NUM>. Since the operation is repeated as explained below, the time tb in a certain repeating unit is regarded as the time ta in the subsequent repeating unit.

The changer <NUM> rotates the sight line direction 303a by the rotational angle θ about the reference axis at the observation site 302a in the virtual space <NUM>, thereby updating the sight line direction 303a to the sight line direction 303b.

In this embodiment, the functions of the changer <NUM> are performed by the control unit <NUM>.

The generator <NUM> generates an image representing a view of the virtual space <NUM> from the observation site <NUM> in the sight line direction <NUM>. For example, the generator <NUM> uses a technique, such as perspective projection, to generate the image representing the view from the observation site <NUM> in the sight line direction <NUM>, on the basis of the certain positions and shapes of the objects <NUM> to <NUM> in the virtual space <NUM>.

The specifier <NUM> specifies at least one focus object determined on a predetermined basis to have a low value of the velocity component, which is not resulting from movement of each of the objects <NUM> to <NUM> in the virtual space <NUM>, in the inter-image velocity of the object transferred from one position to another position in respective images generated by the generator <NUM>.

The inter-image velocity and the velocity component not resulting from movement of an object will now be described with reference to <FIG> and <FIG>. <FIG> illustrates an image 501a representing a view of the virtual space <NUM> from the observation site <NUM> in the sight line direction <NUM> at the time ta. <FIG> illustrates an image 501b representing a view of the virtual space <NUM> from the observation site <NUM> in the sight line direction <NUM> at the time tb. When the sight line direction <NUM> is changed, each of the still objects <NUM> and <NUM> in the virtual space <NUM> among the objects <NUM> to <NUM> is transferred from one position in the generated image 501a to another position in the generated image 501b at an inter-image velocity equal to a velocity component resulting from the change in the sight line direction <NUM>. For example, the object <NUM> is transferred from one position in the generated image 501a to another position in the generated image 501b at an inter-image velocity V2 equal to a velocity component S2x resulting from the change in the sight line direction <NUM>. In contrast, when the sight line direction <NUM> is changed, each of the objects <NUM> and <NUM> moving in the virtual space <NUM> is transferred from one position to another position in the respective generated images at an inter-image velocity, which is equal to the resultant velocity from the velocity component resulting from the change in the sight line direction <NUM> and the velocity component resulting from the movement of the object. For example, the object <NUM> is transferred from one position in the generated image 501a to another position in the generated image 501b at an inter-image velocity V1, which is equal to the resultant velocity from the velocity component S1x resulting from the change in the sight line direction <NUM> and the velocity component Ax resulting from the movement of the object. In general, if a user pays attention to a moving object, the user is likely to change the sight line direction <NUM> in accordance with the movement of the object while capturing the object at the center of the view (that is, the center of the generated image). Accordingly, the focus object to which the user pays attention is expected to have a lower velocity component resulting from the change in the sight line direction <NUM>, that is, a lower velocity component not resulting from the movement of the focus object in the inter-image velocity in comparison to that of the other objects, regardless of the change in the sight line direction <NUM>. In this embodiment, the focus object determined on a predetermined basis to have a low value of the velocity component not resulting from the movement of the object is specified among the objects.

Specifically, the specifier <NUM> acquires preceding and subsequent positions for each of the objects <NUM> to <NUM>. The preceding position indicates the position of the object in the virtual space <NUM> immediately before the change in the sight line direction <NUM> in the image generated immediately before the change in the sight line direction <NUM>. The subsequent position indicates the position of the object in the image to be generated immediately after the change in the sight line direction <NUM>. More specifically, the specifier <NUM> calculates preceding positions based on the known positions and moving velocities of the objects <NUM> to <NUM> in the virtual space <NUM>, the observation site <NUM>, and the sight line direction <NUM>. The specifier <NUM> also calculates subsequent positions based on the preceding positions, the transferred observation site <NUM>, and the changed sight line direction <NUM>. The specifier <NUM> then calculates the difference between the subsequent and preceding positions, as the velocity component resulting from the change in the sight line direction <NUM>, that is, the velocity component not resulting from movement of each object. It should be noted that the specifier <NUM> regards the positions of the representative points (for example, geometric centers or gravitational centers) of the objects <NUM> to <NUM> as the positions of the objects <NUM> to <NUM>.

A method of calculating the velocity component resulting from the change in the sight line direction <NUM> will now be explained focusing on the object <NUM> with reference to <FIG>. For example, the object <NUM> is assumed to be located at a point Pa at the time ta as illustrated in <FIG>. In this case, the specifier <NUM> calculates the position Pia of the point Pa in the image 501a generated at the time ta, based on the point Pa, the observation site <NUM>, and the sight line direction <NUM>. The specifier <NUM> also calculates the position Pib of the point Pa in the image 501b to be generated at the time tb, based on the point Pa, the observation site <NUM> at the time tb, and the sight line direction <NUM> at the time tb. The specifier <NUM> then calculates the difference between the positions Pib and Pia, as the velocity component S1x resulting from the change in the sight line direction <NUM>. That is, the specifier <NUM> calculates an inter-image velocity by subtracting the position Pib from the position Pia on the assumption that object <NUM> remains still, as the velocity component S1x resulting from the change in the sight line direction <NUM>. The specifier <NUM> calculates a velocity component resulting from the change in the sight line direction <NUM> for each of the objects <NUM> to <NUM> in the virtual space <NUM> in the same manner.

The specifier <NUM> then determines whether the velocity component not resulting from the movement of the object is a low value or not on the predetermined basis. In an exemplary method of determining whether the velocity component not resulting from the movement of the object is a low value or not on the predetermined basis, the specifier <NUM> determines the minimum value of the velocity components resulting from the change in the sight line direction <NUM> calculated for the individual objects <NUM> to <NUM> to be a low value on the predetermined basis. The specifier <NUM> then specifies at least one focus object among the objects <NUM> to <NUM> having the minimum value of the velocity component resulting from the change in the sight line direction <NUM>. For example, when the object <NUM> has the minimum velocity component S1x of the velocity components of the objects <NUM> to <NUM> resulting from the change in the sight line direction <NUM>, the specifier <NUM> determines the object <NUM> to be a focus object. Alternatively, the specifier <NUM> may determine the velocity component to be a low value on a predetermined basis if the velocity component resulting from the change in the sight line direction <NUM> is a predetermined threshold or less or is less than a predetermined threshold. Alternatively, the specifier <NUM> may determine the velocity component to be a low value on a predetermined basis if the ratio of the velocity component to the detected change in the sight line direction <NUM> is less than a predetermined threshold. Specifically, the specifier <NUM> may determine the velocity component to be a low value on a predetermined basis if the value, which is calculated by dividing the velocity component resulting from the change in the sight line direction <NUM> by the velocity of the detected change in the sight line direction <NUM>, is less than a predetermined threshold.

In this embodiment, the functions of the specifier <NUM> are performed by the control unit <NUM>.

The generator <NUM> increases the transparency of a hiding object, which is an object other than the focus object specified by the specifier <NUM> among the objects <NUM> to <NUM> and which hides the focus object in a view of the virtual space <NUM> in the sight line direction <NUM>, and then generates an image. For example, the generator <NUM> renders the background with a certain background color (for example, black) and then renders the objects in the descending order of distance from the observation site <NUM>. If the pixels to represent an object other than the focus object have been provided with a color other than the background color, the generator <NUM> determines that these pixels have already been provided with the color of the focus object, that is, these pixels correspond to the focus object hidden behind the other object. The generator <NUM> then determines the other object to be the object hiding the focus object. The generator <NUM> increases the transparency of the other object and then generates an image. For example, if the specifier <NUM> determines the object <NUM> to be the focus object, the generator <NUM> determines the object <NUM> to be the object hiding the object <NUM>. The generator <NUM> then increases the transparency of the object <NUM> as illustrated in <FIG> and then generates the image 501b. The generator <NUM> increases the transparency of the object <NUM> and is thus able to generate an image having a high visibility of the portion of the object <NUM> focused by the user but hidden behind the object <NUM>. In the case where a focus object is hidden behind another focus object, the generator <NUM> generates an image without increasing the transparency of the other focus object hiding the focus object.

In this embodiment, the functions of the generator <NUM> are performed by the control unit <NUM>.

The display controller <NUM> causes the image generated by the generator <NUM> to be displayed on the display <NUM>. For example, the display controller <NUM> causes the image 501a illustrated in <FIG> at the time ta and the image 501b illustrated in <FIG> at the time tb to be individually displayed on the display <NUM>.

In this embodiment, the functions of the display controller <NUM> are performed by the control unit <NUM>.

The operation of the display device <NUM> according to the embodiment of the disclosure will now be explained. <FIG> is a flowchart of a display operation executed by the control unit <NUM> of the display device <NUM>. This operation is activated by, for example, reception of an instruction to start this operation by the operation unit <NUM>.

First, the display device <NUM> sets the observation site <NUM> and the sight line direction <NUM> to be the predetermined initial states in the virtual space <NUM> (Step S101).

The display device <NUM> then initializes the status of the virtual space <NUM> (Step S102). In this initialization step, the display device <NUM> obtains and sets the positions, shapes, orientations, and appearances of objects to be placed in the virtual space <NUM>, and obtains and sets an image of the background supposed to be placed at an infinite distance in the virtual space <NUM>, for example.

The display device <NUM> then detects the orientation of the display device <NUM>, and acquires the rotational angle θ of the display direction of the display <NUM> about the reference axis in the real space <NUM> (Step S103). Since Step S103 is repetitively executed, the acquired rotational angle θ is equivalent to the difference between the display direction <NUM> previously measured by a sensor or the like and the currently measured display direction <NUM>.

The display device <NUM> then updates the sight line direction <NUM> depending on the acquired rotational angle θ (Step S104).

The display device <NUM> then selects one of the objects <NUM> to <NUM> in the virtual space <NUM> for which the velocity component resulting from the change in the sight line direction <NUM> has not yet calculated (Step S105).

The display device <NUM> then calculates the preceding and subsequent positions of the object selected in Step S105 (Step S106).

The display device <NUM> then calculates the difference between the subsequent and preceding positions calculated in Step S106 as the velocity component resulting from the change in the sight line direction <NUM> for the selected object (Step S107). The display device <NUM> then determines whether the velocity components resulting from the change in the sight line direction <NUM> have been calculated for all the objects <NUM> to <NUM> (Step S108). If calculation of the velocity components resulting from by the change in the sight line direction <NUM> for all the objects <NUM> to <NUM> has not yet been completed (Step S108; No), the display device <NUM> returns to Step S105.

If calculation of the velocity components resulting from the change in the sight line direction <NUM> for all the objects <NUM> to <NUM> has been completed (Step S108; Yes), the display device <NUM> specifies a focus object having the minimum value of the calculated velocity components (Step S109).

The display device <NUM> determines whether any object hides the focus object specified in Step S109 (Step S110). If no object hides the focus object (Step S110; No), the display device <NUM> proceeds to Step S112.

If any object hides the focus object (Step S110; Yes), the display device <NUM> increases the transparency of the object hiding the focus object by a predetermined degree (Step S111).

The display device <NUM> then generates an image representing a view of the virtual space <NUM> from the observation site <NUM> in the sight line direction <NUM> (Step S112).

The display device <NUM> then waits until a vertical synchronization interrupt of the display <NUM> (Step S113), and transmits the generated image to the display <NUM> to exhibit the image to the user (Step S114).

The display device <NUM> then updates the status of the virtual space <NUM> (Step S115). For example, in the virtual space <NUM> generated by computer graphics that vary with time, the display device <NUM> executes a physical simulation for updating the position and orientation of an object using the parameters (for example, velocity, acceleration, angular velocity, and angular acceleration) set in the object, and deforms the object under predetermined conditions.

The display device <NUM> then returns to Step S103. The display device <NUM> repeats the above-explained operation, for example, until receiving an instruction to end this operation by the operation unit <NUM>. The repetition period of this operation accords with the vertical synchronization period because of the waiting step (Step S113).

As explained above, the display device <NUM> according to the embodiment of the disclosure changes the sight line direction <NUM> in the virtual space <NUM> in response to a shift in orientation of the display device <NUM> in the real space <NUM>. The display device <NUM> then specifies at least one focus object determined on a predetermined basis to have a low value of the velocity component, which is not resulting from the movement of each of the objects <NUM> to <NUM> in the virtual space <NUM>, in the inter-image velocity of the object. This configuration enables the display device <NUM> to specify a focus object of which the movement is being followed by the user's eyes in the virtual space <NUM>, by detection of a shift in orientation of the display device <NUM> without detection of the position of the display device <NUM>.

In addition, if the specified focus object is hidden behind another object, the display device <NUM> increases the transparency of the object hiding the focus object and then generates an image. The display device <NUM> can therefore make the portion of the focus object hidden behind the other object visible.

The above-described embodiment of the disclosure is a mere example and should not be construed as limiting the applicable range of the disclosure. That is, the embodiment of the disclosure can be applied in various manners and a variety of embodiments are encompassed in the scope of the disclosure.

For example, in the above-described embodiment, the display device <NUM> may enable the user to look into a space behind an obstacle for a focus object by changing the observation site <NUM> in accordance with the change in the sight line direction <NUM>. The example illustrated in <FIG> assumes that a portion of the focus object 401a is hidden behind the object 404a when the user looks at the focus object 401a from the observation site 302a in the sight line direction 303a. In this example, the user is likely to turn and move the head in the real space <NUM> to look into a space behind the object 404a for the portion of the focus object 401a. Unfortunately, the display device <NUM> without an ability to detect the position of the display device <NUM> cannot detect the movement of the head of the user. In this case, the display device <NUM> may detect a turn of the head, that is, a shift in orientation of the display device <NUM>, change the sight line direction 303a in the virtual space <NUM> in accordance with the shift in orientation, and transfer the observation site 302a in accordance with the change in the sight line direction 303a.

Specifically, in the above-described embodiment, the changer <NUM> may change the observation site <NUM> toward the direction opposite to the direction of change in the sight line direction <NUM>, in accordance with the change in the sight line direction <NUM>. For example, if the user turns the head in the counterclockwise direction by a rotational angle θ from the state illustrated in <FIG> at the time ta to the state illustrated in <FIG> at the time tb, the changer <NUM> acquires the rotational angle θ about the reference axis in the real space <NUM> based on the shift in the display direction <NUM> of the display device <NUM> detected by the detector <NUM>. As illustrated in <FIG>, the changer <NUM> then rotates the sight line direction 303a by the rotational angle θ about the reference axis at the observation site 302a in the virtual space <NUM>, thereby updating the sight line direction 303a to the sight line direction 303b. The changer <NUM> also transfers the observation site 302a such that the observation site 302a rotates by the rotational angle θ about the point Pa of the focus object 401a specified by the specifier <NUM> toward the direction opposite to the direction of change in the sight line direction <NUM>, thereby updating the observation site 302a to the observation site 302b. In the example illustrated in <FIG>, since the sight line direction 303a is rotated in the counterclockwise direction to become the sight line direction 303b, the changer <NUM> conducts rotational transfer of the observation site 302a in the clockwise direction by the rotational angle θ to obtain the observation site 302b. This configuration enables the display device <NUM> to transfer the observation site <NUM> toward the direction opposite to the direction of change in the sight line direction <NUM> in accordance with the change in the sight line direction <NUM>, thereby updating the observation site <NUM> to an observation site appropriate for looking into a space for the focus object while bringing a reduced feeling of strangeness to the user <NUM>, even if the display device <NUM> has no ability to detect the position of the display device <NUM>.

In the case where a plurality of focus objects are specified, the changer <NUM> transfers the observation site <NUM> toward the direction opposite to the direction of change in the sight line direction <NUM> about one of the focus objects having the minimum value of the velocity component not resulting from the movement of the object, that is, the minimum value of the velocity component resulting from the change in the sight line direction <NUM>. The angle of rotation of the observation site <NUM> by the changer <NUM> toward the direction opposite to the direction of change in the sight line direction <NUM> is not necessarily be equal to the rotational angle θ of the sight line direction <NUM>. For example, the changer <NUM> may transfer the observation site <NUM> by a rotational angle proportional to the rotational angle θ of the sight line direction <NUM> toward the direction opposite to the direction of change in the sight line direction <NUM>.

In the above-described embodiment, the display device <NUM> detects a shift in orientation of the display device <NUM> and specifies a focus object based on the shift. Alternatively, the display device <NUM> may further have an ability to detect the position of the display device <NUM>. In this case, the changer <NUM> may transfer the observation site <NUM> in the virtual space <NUM> in accordance with the detected position of the display device <NUM>.

For example, with reference to <FIG>, the changer <NUM> acquires a displacement d from the time ta until the time tb of a position <NUM> of the display device <NUM> detected by the detector <NUM>. Specifically, the representative point (for example, geometric center or gravitational center) of the display device <NUM> is located at the position 204a before the user <NUM> turns the head, as illustrated in <FIG>. The representative point of the display device <NUM> is located at the position 204b after the user <NUM> turns the head, as illustrated in <FIG>. The displacement d of the display device <NUM> therefore corresponds to the difference vector between the positions 204a and 204b. With reference to <FIG>, the changer <NUM> transfers the observation site 302a by the displacement d in the virtual space <NUM>, thereby updating the observation site 302a to the observation site 302b. The generator <NUM> then generates an image representing a view of the virtual space <NUM> from the observation site 302b in the sight line direction 303b. The specifier <NUM> then specifies the focus object determined on a predetermined basis to have a low value of the velocity component, which is not resulting from the movement of the object, in the inter-image velocity of the object transferred from one position to another position in the respective images generated by the generator <NUM>, as in the above-described embodiment.

In general, a display device <NUM> capable of detecting not only the orientation but also the position of the display device <NUM> is expected to specify a focus object at the intersection of the previous and current sight line directions <NUM>, assuming that the user changes the sight line direction while capturing the focus object at the center of the view. Unfortunately, the display device <NUM> having a low accuracy of detecting the orientation and position of the display device <NUM> possibly fails to properly determine the intersection. In contrast, the disclosed specifier <NUM> specifies the focus object determined on a predetermined basis to have a low value of the velocity component, which is not resulting from the movement of the object, in the inter-image velocity of the object transferred from one position to another position in the respective images generated by the generator <NUM>, and can therefore specify the focus object despite of a low accuracy of detecting the orientation and position of the display device <NUM>.

In the above-described embodiment, the display device <NUM> specifies a focus object in response to detection of a shift in orientation of the display device <NUM>. That is, the display device <NUM> stops specification of a focus object while the display device <NUM> is maintaining the orientation in the above-described embodiment. Alternatively, the display device <NUM> may specify the most lately specified focus object again, for example, for a predetermined period from the most lately detection of a shift in orientation of the display device <NUM>. No detection of a shift in orientation of the display device <NUM> seems to occur because the user has already turned the display device <NUM> to the direction appropriate for observation of the focus object. The display device <NUM> can thus specify the most lately specified focus object again for a predetermined period from the most lately detection of a shift in orientation of the display device <NUM>, despite of no detection of a shift in orientation of the display device <NUM>.

In the above-described embodiment, the specifier <NUM> specifies the focus object having the minimum value of the velocity component, which is not resulting from the movement of the object, in the inter-image velocity of the object transferred from one position to another position in the respective images generated by the generator <NUM>. In such a case where the specifier <NUM> specifies a single focus object, the focus object may be frequently switched in some situations. In order to prevent the frequent switching of the focus object, for example, the display device <NUM> may stop specification of another focus object for a predetermined period from specification of one focus object.

Although the display device <NUM> detects a shift in yaw angle as a shift in orientation of the display device <NUM> in the above-described embodiment, this configuration should not be construed as limiting the invention. The display device <NUM> may also detect, for example, a shift in roll or pitch angle as a shift in orientation of the display device <NUM>.

The above-described functions of the display device <NUM> may be achieved by hardware including various electric and electronic devices. Alternatively, these functions may be achieved by, for example, electric and electronic circuits and controllers installed in an existing personal computer or information terminal, to configure the display device <NUM>.

Alternatively, the display <NUM> and the sensor unit <NUM> of the display device <NUM> may be configured by a head mounted display, and the display control device including the control unit <NUM>, the ROM <NUM>, and the RAM <NUM> of the display device <NUM> may be configured by a personal computer or information terminal connected to the head mounted display, for example.

Alternatively, the functions of the display device <NUM> may be achieved by an existing personal computer or information terminal by installation of a program for providing the functional configuration of the display device <NUM> demonstrated in the above embodiment and by execution of the program in a CPU or the like that controls the personal computer or information terminal. The CPU or the like executes the program and can thus perform the display control method for controlling the display device <NUM>.

This program may be applied in any procedure other than the above example. For example, the program may be stored in a non-transitory computer-readable recording medium, such as a compact disc read-only memory (CD-ROM), digital versatile disc (DVD), or magneto optical disc (MO), for example. Alternatively, the program may be stored in a storage on a network, such as the Internet to be downloaded.

The foregoing describes some example embodiments for explanatory purposes. Although the foregoing discussion has presented specific embodiments, persons skilled in the art will recognize that changes may be made in form and detail without departing from the scope of the invention. Accordingly, the specification and drawings are to be regarded in an illustrative rather than a restrictive sense. This detailed description, therefore, is not to be taken in a limiting sense, and the scope of the invention is defined only by the included claims.

Claim 1:
A display control device (<NUM>) for controlling a display device (<NUM>), the display control device comprising:
a generator (<NUM>) configured to generate, at a first time (ta), an (501a) representing a view of a virtual space (<NUM>) including a plurality of still and moving objects (<NUM>-<NUM>), wherein the view is observed from an observation site (302a) in the virtual space in a sight line direction (303a) in the virtual space;
a display controller (<NUM>) configured to cause the generated image to be displayed on a display (<NUM>) included in the display device;
a changer (<NUM>) configured to change the sight line direction to a second sight line direction (303b) in the virtual space in accordance with detecting a shift in orientation of the display device in a real space from the first time until a second time (tb); and
a specifier (<NUM>) configured to specify a focus object among the objects wherein the changer is further configured to transfer the observation site around the specified focus object toward a direction opposite to a direction of change in the sight line direction;
the display control device being characterized in that:
the specifier is configured to calculate, for each object among the plurality of still and moving objects, an inter-image velocity of said each object based on a difference between
a position of said each object in the image generated at the first time and
a position of said each object in the image (501b) representing the view of the virtual space in the second sight line direction, and
specify, as the focus object, an object whose calculated inter-image velocity is equal to or less than a predetermined threshold and is a minimum value among the plurality of still and moving objects.