Projection image adjusting system and projection image adjusting method

The projection image adjusting system has a storage, a receiving section, and a controller. The storage stores virtual-environment setting information on a set-up situation of a projection display apparatus set so as to have a desired image-projection state on an object on which an image is projected in a virtual space created by a computer and also stores a control set-up value for the projection display apparatus in the desired image-projection state. The receiving section receives real-environment setting information on a set-up situation of the projection display apparatus in a real space. The controller controls the projection display apparatus in the real space. Based on the virtual-environment setting information and the real-environment setting information, the controller corrects the control set-up value so as to decrease a difference between an image-projection state in the real space and the desired image-projection state.

BACKGROUND

1. Technical Field

The present disclosure relates to a projection image adjusting system and a projection image adjusting method for projecting image on a 3D structure.

2. Description of the Related Art

A technique for projecting image on a 3D structure, so-called projection mapping is becoming popular. Patent Literature 1 discloses a technique for supporting installation of a projection display apparatus for projection mapping on an event site. Specifically, it discloses technique that generates a layout chart for installation of a projection display apparatus simulated on a virtual space, allowing the user to have an easy set-up with reference to the layout chart of the projection display apparatus on the event site.

CITATION LIST

Patent Literature

SUMMARY

When a user locates a projection display apparatus in a real space with reference to a layout chart obtained by simulation, it is difficult to set the display on the first try at the right position shown by the simulation. For adjusting the position, the user has to move the projection display apparatus or has to adjust lenses carefully and repeatedly in the real space. This is very complicated work that imposes much time and effort on the user.

The present disclosure provides a projection image adjusting system and a projection image adjusting method capable of simplifying the installation and adjustment of the projection display apparatus.

The projection image adjusting system as an exemplary embodiment of the present disclosure has a storage, a receiving section, and a controller. The storage stores virtual-environment setting information on a set-up situation of a projection display apparatus set so as to have a desired image-projection state on an object on which an image is projected in a virtual space created by a computer and also stores a control set-up value for the projection display apparatus in the desired image-projection state. The receiving section receives real-environment setting information on a set-up situation of the projection display apparatus in a real space. The controller controls the projection display apparatus in the real space. Based on the virtual-environment setting information and the real-environment setting information, the controller corrects the control set-up value so as to decrease a difference between an image-projection state in the real space and the desired image-projection state, and based on the corrected control set-up value, the controller controls the projection display apparatus in the real space.

The projection image adjusting system as other exemplary embodiments of the present disclosure has a storage, a receiving section, and a controller. The storage stores virtual-environment setting information on a set-up situation of a projection display apparatus set so as to have a desired image-projection state on an object on which an image is projected in a virtual space created by a computer and also stores a shape and a size of a projection image in the desired image-projection state. The receiving section receives real-environment setting information on a set-up situation of the projection display apparatus in a real space. The controller creates a projection image and causes the projection display apparatus in the real space to project the projection image. Based on the virtual-environment setting information and the real-environment setting information, the controller corrects the shape and the size of the projection display apparatus stored in the storage so as to decrease a difference between an image-projection state in the real space and the desired image-projection state, and causes the projection display apparatus in the real space to project the corrected projection image.

According to the present disclosure, the virtual-environment setting information shows the set-up situation of the projection display apparatus when a desired image-projection state with respect to an object on which image is projected is obtained through simulation in the virtual space, whereas the real-environment setting information shows the set-up situation of the projection display apparatus in the real space. Based on the virtual-environment setting information and the real-environment setting information, the structure of the present disclosure corrects the control set-up value so as to decrease a difference between an image-projection state in the real space and a desired image-projection state. Further, based on the corrected control set-up value, the workings of the projection display apparatus are automatically controlled. In this way, installation and adjustment of the projection display apparatus is simplified.

DETAILED DESCRIPTION

Hereinafter, an exemplary embodiment will be described in detail, with reference to the accompanying drawings. However, details beyond necessity—for example, descriptions on well-known matters or on substantially identical structures—may be omitted to eliminate redundancy from the description below for easy understanding of those skilled in the art.

It is to be understood that the accompanying drawings and the description below are provided by the applicant for purposes of full understanding of those skilled in the art and are not to be construed as limitation on the scope of the claimed disclosure.

First Exemplary Embodiment

Hereinafter, the structure of the first exemplary embodiment will be described with reference toFIG. 1throughFIG. 8.

FIG. 1is a schematic view showing a state in which projection image adjusting system1of the embodiment is installed in a real space. Projection image adjusting system1adjusts an installed condition of projectors101and102in the real space. Projectors101and102are placed on bases T. Projectors101,102and adjustment PC200are connected by network cable L and video-signal cable V with each other. Position detecting module300has, for example, a GPS receiver, a geomagnetic sensor, an acceleration sensor, and a gyro sensor. Disposed at the center of the lower end of a projection wall of structure S, position detecting module300detects the position and direction of the projection wall of structure S. The position of the projection wall is represented by, for example, the latitude and longitude showing the center of the projection wall. The direction of the projection wall is represented by, for example, the angle to the Z-axis of the normal line on the projection wall. Position detecting module300communicates with adjustment PC200via radio wave, but they are not necessarily connected by air; they may communicate via wire. Projectors101and102are an example of a projection display apparatus, and structure S is an example of an object on which image is projected.

FIG. 2is a functional block diagram of projector101used in the first exemplary embodiment. The description will be focused on projector101because projector102has a structure the same as projector101. Projector101has signal processor1010and projection unit1017that projects image onto a projection wall. Signal processor1010further has wireless communicator1011for wireless communication, wired communicator1012for wire communication, storage1013, image processor1014, controller1015, and posture/position detector1016. Posture/position detector1016has, for example, a GPS receiver, a geomagnetic sensor, an acceleration sensor, and a gyro sensor. Posture/position detector1016detects the position, direction, and tilt of projector101. The position of projector101is represented by the latitude and longitude showing the installed position of projector101. The direction is represented by the angle of the reference direction of projector101with respect to the north-south direction. The tilt is represented by an angled position of projector101in the directions of the left-right axis and the front-back axis with respect to the level plane of the reference plane of projector101. Projection unit1017has lens controller1018that controls a shift amount, a zoom amount, and a focus amount of a lens.

FIG. 3is a functional block diagram of adjustment PC200in the embodiment. Adjustment PC200has wireless communicator201, wired communicator202, storage203, user-interface section (hereinafter, referred to as UI section)204, controller205, image processor206, and display207. Storage203stores programs, such as the operating system and applications, and various kinds of data, such as 3D modeling data on structure S and image data for projection. User-interface (UI) section204is a user interface that receives user's instruction such as selection. Controller205performs calculation on the programs and the data stored in storage203. Image processor206creates image from image data. Display207displays windows and images of an application program. Adjustment PC200has commonly used hardware—wireless communicator201is formed of a wireless LAN unit; storage203is formed of a hard disk and RAM; UI section204is formed of a keyboard, a mouse; controller205is formed of a CPU; and display207is formed of a graphics board and a display connected to the board. Controller205is an example of the controller, and storage203is an example of the storage.

Hereinafter, the workings of such structured projection image adjusting system1will be described. The flowchart ofFIG. 4illustrates the workflow for adjusting the installation (hereinafter, installation adjusting workflow) of projectors101,102so as to fit projection image within a projection range of structure S.FIG. 4shows processes to be performed in adjustment PC200on the left side and shows processes to be performed in projectors101,102or position detecting module300on the right.

In the step S100of the installation adjusting workflow, adjustment PC200performs, with use of an application program, layout design of the projector as an optimal form in a virtual space. Specifically, 3D modeling data on structure S on which image is to be projected in the real space is read into the application program. The 3D modeling data on structure S may be CAD data on structure S or may be data obtained by 3D measurement on site. Next, the models of projectors101,102(hereinafter, referred to as a projector model as necessary) are placed in the virtual space, and the position of each projector model, installation angle, and a zoom ratio of a lens are adjusted so as to fit structure S entirely in the projection range.

FIG. 5is an example of window210of an application program shown in display207of adjustment PC200. The model of structure S—represented by 3D modeling data that has been read into the application program—is shown as ‘Screen1’ (ID name), and the center of the lower end of the model is positioned at origin O, i.e., (X, Y, Z)=(0, 0, 0). The projector model corresponding to projector101is shown as ‘Projector1’ (ID name) and is disposed in a virtual space. Similarly, the projector model corresponding to projector102is shown as ‘Projector2’ (ID name) and is disposed in the virtual space. The plane shown by ‘Stage’ (ID name) in the virtual space is the reference plane (Y=0), which corresponds to the horizon plane in the real space. In the description below, the projector model shown as ‘Projector1’ (ID name) may be simply referred to as ‘Projector1’, and similarly, the projector model shown as ‘Projector2’ (ID name) may be simply referred to as ‘Projector2’. Projection area A1and Projection area B1, which are projected by ‘Projector1’ and ‘Projector2’, respectively, are shown in window210.

Of the two projector models shown in the window, the user can choose either of them by positioning the mouse pointer over a desired model and clicking.FIG. 5shows a state where the projector model of ‘Projector1’ has been chosen. When the user chooses ‘Projector1’, adjustment PC200highlights the projector model of ‘Projector1’ and projection area A1by heavy line in the window of the application program, and shows the information on the chosen projector model (i.e., ‘Projector1’) in the upper left box in window210. In the upper right box in window210, information on projector101in the real space that corresponds to ‘Projector1’ in the virtual space, which will be described later.

The items shown in the left box will be described below. The item of ‘POSITION’ represents the position of ‘Projector1’ (as the chosen projector model in the window) in the virtual space.FIG. 5shows the position of ‘Projector1’ as a distance from origin O (i.e., as a coordinate value). The item of ‘LOOK AT’ represents the position of projection image projected by ‘Projector1’. ‘Projector1’ projects an image such that the center of the image to be projected agrees with the position shown by ‘LOOK AT’. The information shown by ‘POSITION’ and ‘LOOK AT’ determines the projecting direction of the projector model. The information shown by the items of ‘POSITION’ and ‘LOOK AT’ is an example of the virtual-environment setting information.

The item of ‘LENS’ in window210shows information on the lens used for ‘Projector1’, having the following sub items: ‘Model’; ‘Shift’; and ‘ThrowRatio’. The item of ‘Model’ shows a model number of the lens of ‘Projector1’. The user can select a desired model number from multiple options, and according to the selected model number, the setting range of a zoom factor (a zoom amount) that will be described below is determined. The item of ‘Shift’ shows how much amount the image is shifted by the shifting function of the lens (a shifting amount) with respect to the position shown by ‘LOOK AT’ at which ‘Projector1’ projects the image. The item of ‘ThrowRatio’ shows a zoom factor of the lens. When the projector is disposed away from the projection wall by distance D and the lateral length of the projection range of the projector is represented by width W, the value of ‘ThrowRatio’ is obtained by D/W. The shifting amount that ‘Shift’ shows and the zoom factor that ‘ThrowRatio’ shows are an example of the control set-up value.

Similarly, when the user chooses ‘Projector2’ shown in window210by clicking, adjustment PC200highlights ‘Projector2’ and projection area B1by heavy lines; and at the same time, PC200shows each item of ‘POSITION’, ‘LOOK AT’, and ‘LENS’. Adjusting each item described above allows ‘Projector1’ and ‘Projector2’ to cover the entire range of ‘Screen1’.

When the adjustment in step S100ofFIG. 4is completed, the user locates projectors101and102in the real space in step S200. Specifically, prior to the installation, the user outputs a drawing derived from the setting values having undergone adjustment in the virtual space (in step S100). With reference to the setting values in the drawing, such as the position, direction, and tilt of projectors101and102, the user locates bases T, and projectors101and102in the real space. At the same time, the user locates position detecting module300at a position that corresponds to origin O in the virtual space. When the installation is completed, the user connects between adjustment PC200and projectors101,102by network cable L and video-signal cable V, and connects between adjustment PC200and position detecting module300by air.

Next, in step S300, the user performs matching between the reference point in the real space and origin O as the reference point in the virtual space of adjustment PC200. According to the structure of the embodiment, origin O in the virtual space is located at the center of the lower end of the projection wall of the modeling data on structure S. The user performs positional matching between origin O in the virtual space and position detecting module300disposed at the reference point in the real space.

FIG. 6shows an example of window210of the application program for the matching operation of the reference points. When the user puts the mouse pointer over the cross line showing origin O and clicks, the cross line is highlighted by heave lines and the window is put into the selection mode. The application program shows input areas (of the left and right boxes in the upper section of the window) for the matching operation; specifically, the left box contains information on the virtual space, whereas the right box contains information on position detecting module300in the real space. For example, suppose that position detecting module300of the embodiment has an IP address of ‘192.168.0.18’. When the user sets the IP address at ‘IP ADDRESS’ in the right box and pushes the ‘Connect’ button, adjustment PC200starts communicating with position detecting module300. When the connection between them is established, adjustment PC200reads out information on the position/direction (i.e., latitude LAT, longitude LON, altitude ALT, and direction DIR) from position detecting module300and shows the values. Further, based on the read-out values, adjustment PC200calculates the location as coordinate data and the direction of position detecting module300, making correspondence between the calculated coordinate position and origin O, and between the calculated direction and the z-axis direction in the virtual space.

Next, in step S400, adjustment PC200makes correspondence between a projector model in the virtual space and a projector in the real space, i.e., between ‘Projector1’ and projector101; and between ‘Projector2’ and projector102.FIG. 7shows an example of window210of the application program for making correspondence between ‘Projector1’ as a projector model in the virtual space and projector101in the real space. For example,FIG. 7shows a state in which ‘Projector1’ is being selected. Suppose that the IP address is ‘192.168.0.8’ in the embodiment. When the user sets the address to ‘IP ADDRESS’ in the upper right box and pushes the ‘Connect’ button, adjustment PC200starts communicating with projector101. When the connection between them is established, adjustment PC200reads out information on the position/direction (i.e., latitude LAT, longitude LON, altitude ALT, direction DIR, pitch PIT showing lateral tilt, and roll ROL showing longitudinal tilt) from posture/position detector1016of projector101and shows the values.

Adjustment PC200also shows the ‘Receive’ button next to ‘POSITION’ and the ‘Send’ button next to ‘LENS’ in the left box showing layout information on ‘Projector1’ in the virtual space, which makes correspondence between projector101in the real space and ‘Projector1’ in the virtual space. The procedures described above are also applied to make correspondence between ‘Projector2’ in the virtual space and projector102in the real space.

Next, in step S500, adjustment PC200transfers information on the lens—that has been defined in the virtual space by adjustment PC200in step S100—to projectors101and102in the real space. Specifically, when the user pushes the ‘Send’ button additionally shown next to ‘LENS’ inFIG. 7, adjustment PC200sends the shift amount, the zoom factor, and the focus value (focus amount) of the lens that have been defined in the virtual space to projector101in the real space. According to the shift amount, the zoom factor, and the focus value received from adjustment PC200, projector101provides the lens mounted on it with automatic adjustment.

Although it is not shown, the adjustment for a lens of projector102is also performed in the same manner. Under the state where the correspondence between ‘Projector2’ and projector102has been made and ‘Projector2’ is being selected, when the user pushes the ‘Send’ button, adjustment PC200sends the shift amount, the zoom factor, and the focus value (focus amount) of the lens for ‘Projector2’ that have been defined in the virtual space to projector102in the real space. According to the shift amount, the zoom factor, and the focus value received from adjustment PC200, projector102provides the lens mounted on it with automatic adjustment.

In step S200ofFIG. 4, the projectors are manually located at a place by the user; the position and the direction of the projectors may not agree with those of the projector models in the virtual space. In step S600, according to the information on the position/direction of projectors101and102in the real space, adjustment PC200updates positional information (i.e., information on the position/direction of Projectors1and2) on the projector models. Specifically, when the user pushes the ‘Receive’ button additionally shown next to ‘POSITION’ in the left box ofFIG. 7, adjustment PC200calculates the values of ‘POSITION’ and ‘LOOK AT’ and updates them, based on the positional information on projector101shown in the right box ofFIG. 7and the positional information on position detecting module300shown in the right box ofFIG. 6. These values are calculated by commonly used geometrical operation. In the description, each value of ‘POSITION’ and ‘LOOK AT’ before updating is an example of the virtual-environment setting information, whereas each value of ‘POSITION’ and ‘LOOK AT’ after updating is an example of the real-environment setting information. The item of ‘LENS’ shows information on the lens mounted on a currently selected projector model. Position detecting module300, posture/position detector1016, and controller205is an example of the receiving section that obtains the real-environment setting information.

FIG. 8shows an example of window210of the application program. The window shows a state where the information on the position/direction of ‘Projector1’ as a projector model in the virtual space has been updated, based on the information on the position/direction of projector101in the real space. As shown in the left box ofFIG. 5, ‘POSITION’ is defined as (X, Y, Z)=(−4.5, 3.0, 5.4) and ‘LOOK AT’ is defined as (X, Y, Z)=(−4.0, 3.0, 0.0) in the virtual space. However, according to the result of calculation based on the positional information of projector101in the real space, the aforementioned values are updated as follows: the value of ‘POSITION’ is (X, Y, Z)=(−4.4, 3.1, 5.4); and the value of ‘LOOK AT’ is (X, Y, Z)=(−4.2, 2.8, 0.0). In the window, projection area A2that reflects the updated values is shown by heavy (solid) lines, whereas desired projection area A1is shown by bold broken lines, which allows the user to know a positional difference between currently set projection area A2and desired projection area A1.

Next, in step S700, adjustment PC200performs final adjustment for projector101so that currently set projection area A2covers desired projection area A1. Specifically, according to the positional information (virtual-environment setting information) at the stage of layout design (in step S100) and the positional information (real-environment setting information) at the stage of updating real-space positional information (step S600), adjustment PC200calculates an amount of difference in the center position and size between desired projection area A1and currently set projection area A2. The amount of difference in the center position and size between desired projection area A1and currently set projection area A2is an example of the difference between the image projection state in the real space and a desired image projection state. Adjustment PC200calculates a control set-up value (i.e., ‘Shift’ and ‘ThrowRatio’ of the item of ‘LENS’ of ‘Projector1’) for minimizing the amount of difference in the center position and size between desired projection area A1and currently set projection area A2. These values are obtained geometrically, and after calculation, the obtained values are automatically shown at ‘Shift’ and ‘ThrowRatio’ of the item of ‘LENS’ of ‘Projector1’ as a projector model in the left box in window210. That is, the control set-up value (i.e., ‘Shift’ and ‘ThrowRatio’ of the item of ‘LENS’ of ‘Projector1’) is corrected. In addition to the values above, a focus value of a lens may be corrected as the control set-up value.

When the user pushes the ‘Send’ button disposed next to the item of ‘LENS’, adjustment PC200sends the updated (corrected) values—a shift amount of a lens (i.e., ‘Shift’), a zoom factor (i.e., ‘ThrowRatio’), and a focus value—to projector101in the real space. According to the shift amount of a lens (i.e., ‘Shift’), the zoom factor (i.e., ‘ThrowRatio’), and the focus value received from adjustment PC200, projector101provides the lens with automatic adjustment. Through the final adjustment described above, projection area A2in the real space is automatically corrected so as to agree with (or so as to minimize the amount of difference from) desired projection area A1in the virtual space. In step S700, final adjustment for projector102can be similarly performed; in that case, the user selects ‘Projector2’ as a projector model in window210.

According to the structure of the embodiment, as for a positional difference that is directly uncontrollable by adjustment PC200and has difficulty in adjustment at a later time, such as the position and the tilt of a projector located in the real space, adjustment PC200reads the aforementioned values in the real space and corrects a controllable item, for example, a shift amount and a zoom factor of a lens. Such a correction allows the real space and the virtual space to make correspondence with each other.

The structure above greatly decreases the time and effort when the user installs and makes adjustment for projectors101and102so as to agree with the predetermined layout that has designed in the virtual space.

Projection image adjusting system1of the embodiment has storage203, a receiving section (i.e., position detecting module300, posture/position detector1016, and controller205), and controller205. Storage203stores virtual-environment setting information on the set-up situation of projectors101and102so as to have a desired image-projection state on an object on which image is projected in a virtual space created by adjustment PC200, and also stores a control set-up value for projectors101and102. The receiving section receives real-environment setting information that shows the set-up situation of projectors101and102in the real space. Based on the virtual-environment setting information and the real-environment setting information, controller205corrects the control set-up value so as to eliminate a difference between the image-projection state in the real space and a desired image-projection state. Based on the corrected control set-up value, controller205controls the workings of projectors101and102.

In the structure above, the virtual-environment setting information shows the set-up situation of projectors101and102when a desired image-projection state with respect to an object on which image is projected is obtained through simulation in the virtual space, whereas the real-environment setting information shows the set-up situation of projectors101and102in the real space. Based on the virtual-environment setting information and the real-environment setting information, controller205corrects the control set-up value so as to eliminate a difference between the image-projection state in the real space and a desired image-projection state. Further, based on the corrected control set-up value, the workings of projectors101and102are automatically controlled, by which installation and adjustment for projectors101and102is simplified.

In the exemplary embodiment, the control set-up value is at least any one of a shift amount, a zoom amount, and a focus amount of the lens of projectors101and102. Based on the control set-up value, projectors101and102control the lens. Controlling the lens of projectors101and102provides an appropriate image projection suitable for the set-up position of projectors101and102with in the real space.

Other Exemplary Embodiments

In the first exemplary embodiment, projectors101and102(as a projection display apparatus) drive the lens according to the control set-up value. However, the structure is not limited to the above. When bases T for projectors101and102are structured so as to be movable in the directions of the X-axis, the Y-axis, and the Z-axis, projectors101and102with the bases can be moved in the directions of the X-axis, the Y-axis, and the Z-axis, according to the control set-up value, for example. Further, according to the control set-up value, bases T may be moved and the lens may be driven. For example, when adjustment for a projection range is not satisfactorily achieved by lens-driving only, movable bases T may be a help to the adjustment. Further, bases T may be formed separate from the projectors. In that case, bases T may be moved by direct control of adjustment PC200, or adjustment PC200may control the projectors so as to move on fixed bases T.

Further, as another possibility, when a desired image projection is obtained on a projection object in the virtual space, the shape and the size of the projection image is stored in storage203. In that case, based on the virtual-environment setting information and the real-environment setting information, controller205corrects the shape and the size of the stored projection image so as to minimize the difference between the image-projection state in the real space and a desired image-projection state. After correction, the corrected projection image is projected by projectors101and102. The specific structure will be described below.

Projection image adjusting system1has storage203, a receiving section (i.e., position detecting module300, posture/position detector1016, and controller205), and controller205. Storage203stores virtual-environment setting information on the set-up situation of projectors101and102so as to have a desired image-projection state on an object on which image is projected in a virtual space created by adjustment PC200, and also stores the shape and the size of the projection image in the desired image-projection state. The receiving section receives real-environment setting information that shows the set-up situation of projectors101and102in the real space. Controller205creates projection image, and the image is projected by projectors101and102. Based on the virtual-environment setting information and the real-environment setting information, controller205corrects the shape and the size of the stored projection image so as to minimize a difference between an image-projection state in the real space and a desired image-projection state. The corrected projection image is projected by projectors101and102.

The structure above differs from that of the first exemplary embodiment in that there is no need for driving the lens of projectors101and102, allowing projectors101and102to have a simplified structure and to have easy installation and adjustment.

Although the structure of the first exemplary embodiment projects an image onto a 3D object by using two projectors (i.e., projector101and projector102), but the number of projectors is not limited to two; only one or three-or-more projectors can be employed.

The structure of the embodiment has been described in detail as an example of the technology of the present disclosure with reference to accompanying drawings.

In addition to a component essential for solving problems, the accompanying drawings and the in-detail description can contain a component used for illustrative purpose in the technology but not essential for solving problems. It will be understood that not all the components described in the drawings and description are essential for solving problems.

Further, it will be understood that the aforementioned embodiment is merely an example of the technique of the present disclosure. That is, the technique of the present disclosure is not limited to the structure described above, allowing modification, replacement, addition, and omission without departing from the spirit and scope of the claimed disclosure.

INDUSTRIAL APPLICABILITY

The present disclosure is extensively used for a technique that supports installation and adjustment of a projection display apparatus.