Image display and photographing system, photographing device, display device, image display and photographing method, and computer-readable storage medium for computer program

An image display and photographing system includes a display device configured to display a target image on a surface; a photographing device configured to photograph the surface; a first detector configured to detect that an object moves away from a space between the surface and the photographing device; a second detector configured to detect that, after the first detector detects that the object moves away from the space, the surface is touched at a predetermined position; a third detector configured to detect that, after the second detector detects that the surface is touched at the predetermined position, the object moves away from the space; and a photographing control portion configured to control, after the third detector detects that the object moves away from the space, the photographing device to photograph the surface.

This application is based on Japanese patent application No. 2014-097556 filed on May 9, 2014, the contents of which are hereby incorporated by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a system for image display and photographing.

2. Description of the Related Art

Recent years have seen the widespread use of projectors which projects, onto a screen, an image displayed in a personal computer or smartphone. Such a projector is sometimes called an “image projection device”.

The screen may be a whiteboard. The screen may be a white plastic sheet put on a wall. In such cases, a user may take a note on the screen with a pen while an image is projected onto the screen. The user may take a photo of the screen with a digital camera for recording.

There has been proposed a technology which allows a user to take a photo easily. According to the technology, a projector has a digital camera built therein, and a lens for projection to a liquid crystal projector and a lens for the digital camera are used in common. Thereby, both the projected light of the video by a personal computer outputted from the projector and the incident light of a synthetic image of the image handwritten by using a marker pen or the like on a whiteboard pass through the same lens. This eliminates, in importing the synthetic video of the video by the personal computer and the handwritten image to the personal computer, the need to adjust the position and size thereof (English abstract of Japanese Laid-open Patent Publication No. 2004-252118).

Another technology has been proposed for changing an image displayed at a time when a user touches a predetermined position in a display surface (Japanese Laid-open Patent Publication No. 2005-242725). Yet another technology has been proposed for controlling the driving of a projector. According to the technology, it is detected whether or not a human is present in a drawing region where a projector draws an image; and the driving of the projector is so controlled as to change an image or an image pattern plotted in the drawing area depending on whether or not the human is detected in the drawing region (Japanese Laid-open Patent Publication No. 2011-107237).

It is bothersome for a user to operate a button of a projector while making a presentation. This is because, during the presentation, the user is a little away from the projector. The user is often next to a screen during the presentation.

In view of this, it would be convenient to enter a command into the projector by doing something on the screen. It would be also convenient to enter a photographing command by doing something on the screen. For example, a button corresponding to the photographing command is displayed together with an image on the screen. In response to the button touched, a digital camera takes an image.

According to this method, unfortunately, a user's human body is sometimes photographed above the image.

SUMMARY

The present invention has been achieved in light of such an issue, and an object thereof is to photograph an image in such a manner that an object such as a user's human body does not overlap the image even when a photographing command is entered by operating a button displayed on a screen.

According to an aspect of the present invention, an image display and photographing system includes a display device configured to display a target image on a surface; a photographing device configured to photograph the surface; a first detector configured to detect that an object moves away from a space between the surface and the photographing device; a second detector configured to detect that, after the first detector detects that the object moves away from the space, the surface is touched at a predetermined position; a third detector configured to detect that, after the second detector detects that the surface is touched at the predetermined position, the object moves away from the space; and a photographing control portion configured to control, after the third detector detects that the object moves away from the space, the photographing device to photograph the surface.

These and other characteristics and objects of the present invention will become more apparent by the following descriptions of preferred embodiments with reference to drawings.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Referring toFIG. 1, a projector1and a whiteboard5are used to make a presentation. The projector1is connected to a personal computer7operated by a presenter8or an assistant9. The personal computer7gives data on an image to be projected and a projective instruction to the projector1. The projector1follows the instruction from the personal computer7to project a target image onto the whiteboard5to be used as a screen (projection surface5A).

Because the whiteboard5is used as the projection surface5A, the presenter8may add supplemental information to a projected image or correct errors therein by writing the same onto the whiteboard5with a pen only for the whiteboard5. The projector1according to this embodiment has a photographing function. The presenter8performs operation to cause the projector1to photograph the whiteboard5, so that the handwritten contents can be recorded.

FIG. 2shows an example of the hardware configuration of the projector1. The projector1is provided with a projection unit11, a camera12, an object sensor13, a Central Processing Unit (CPU)15, a Random Access Memory (RAM)16, a non-volatile memory17, an image processing portion18, an interface19, a lamp driver20, a DC power source21, and so on.

The projection unit11is a display means for displaying an image by projecting the image onto the projection surface5A. The projection unit11includes a liquid crystal panel for displaying an image to be projected, a flood lamp provided in the back of the liquid crystal panel, and a group of lenses for forming an image on the projection surface. The DC power source21supplies power necessary for the flood lamp to emit light to the projection unit11through the lamp driver20.

The camera12is a photographing means for taking an image of the projection surface5A. The camera12has a two-dimensional image pickup device. The camera12outputs photographic data obtained by the image pickup device to the image processing portion18. The camera12may be a scanner camera for obtaining a two-dimensional photographic image with a one-dimensional image pickup device and a scanning optical system.

The object sensor13is a distance measuring sensor for detecting the position of an object which is present between the projection surface5A and the camera12. The object sensor13includes a light-emitting device for emitting infrared rays, and two image pickup devices. The object sensor13outputs, to the CPU15, data on two photographic images different in location of the eyepoint for distance measurement by using a triangulation method. Each of the image pickup devices has, on its light-receptive surface, a filter for infrared photography.

The CPU15loads a program for controlling the projector1from the non-volatile memory17into the RAM16to execute the program. The CPU15performs communication with the personal computer7through the interface19for communication with external devices. The CPU15controls the projector1to project an image in accordance with instructions from the personal computer7. The CPU15also executes the variety of processing of displaying a mark on the projection surface5A and of storing the photographic data captured by the camera12into the non-volatile memory17. The processing is discussed later. The non-volatile memory17is, for example, a flash memory.

The image processing portion18expands, in a memory, a bitmapped image corresponding to the data sent by the personal computer7to display an image to be projected in the liquid crystal panel of the projection unit11. The image processing portion18also performs processing for compressing the photographic data captured by the camera12. The image processing portion18includes, for example, an Application Specific Integrated Circuit (ASIC).

The interface19has a USB portion191which enables wired communication meeting the Universal Serial Bus (USB) standards. The interface19also has a Wi-Fi portion192which enables wireless communication meeting the Wi-Fi standards.

The projector1has functions related to photographing by the camera12, namely, a first function, a second function, and a third function.

The first function is to photograph the projection surface5A with an object not appearing in the projection surface5A. The first function is used when a handwritten content on the projection surface5A by the presenter8is recorded. To be specific, the first function makes it possible to obtain photographic data showing the projection surface5A having the handwritten content not hidden by the object, which is preferable in terms of recording.

The second function is to detect a touch, by an object, to a predetermined position in the projection surface5A as user's instructions for photographing, namely, as entry of a photographing command. The second function makes it possible to provide the presenter8with an operational environment in which instructions for photographing are given to the projector1by touching the projection surface5A. This makes the projector1convenient to use when the projector1is used for presentation.

The third function works when the projector1is ready to receive instructions for photographing. The third function is to display, on the projection surface5A, a mark51showing that the projector1is ready to receive instructions for photographing. The third function makes it possible to provide the presenter8with an operational environment which enables the presenter8to easily know whether instructions for photographing can be given. This makes the projector1convenient to use when the projector1is used for presentation.

The description goes on to the configuration and operation of the projector1, focusing on the first through third functions.

The projector1determines whether or not there is an object hiding the entirety or a part of the projection surface5A with respect to the camera12in order that a photographic image may not contain an object. To be specific, the CPU15compares two photographic images obtained by the object sensor13to determine the parallax of a subject, then to calculate a distance from a photographing device to the subject. The CPU15then compares the calculated distance and a distance (known distance) to the whiteboard5which is stored in advance. The known distance may be measured in advance by the projector1, or be entered by the user. If the calculated distance is shorter than the known distance, then the CPU15determines that there is an object. In other cases, the CPU15determines that there is no object.

Referring toFIG. 3A, an object30is present within a part corresponding to a field of view of the camera12for photographing in a space between the projection surface5A and the camera12. The part is hereinafter referred to as a photographic space40. In such a case, the projector1determines that the object30is present. Referring toFIG. 3B, the object30is present close to the whiteboard5. The object30is however not present in the photographic space40. In such a case, the projector1determines that the object30is not present.

The CPU15determines whether or not an object is present at regular intervals, e.g., of1through3seconds. If the result of latest determination (determination this time) is “object30not present”, and if the result of the previous determination is “object30present”, then the results show that the object30moves from within the photographic space40to outside the photographic space40between the previous determination and the determination this time. In other words, the results show that the object30moves away from the photographic space40. As discussed later, the projector1photographs the projection surface5A after the user gives instructions for photographing and the object30moves away from the photographic space40.

FIG. 4shows examples of the object30. While the presenter8makes a presentation with the projector1, what moves into and out of the photographic space40is mostly limited to the presenter8. In this embodiment, examples of the object30are: (1) a human body31; (2) a pen35held in a hand32; (3) a pointer36held in the hand32; (4) a hand32bpointing to somewhere; (5) a back of the hand32c;and (6) a flat of the hand32d.

Examples of a method for detecting a touch of the object30to a predetermined position of the projection surface5A includes: a method by using the object sensor13as shown inFIG. 5; and a method by using a shadow30sof the object30appearing in the projection surface5A as shown inFIG. 6. InFIGS. 5 and 6, a predetermined position of the projection surface5A is any position of a region5Ap onto which the mark51is projected.

Referring to (A) ofFIG. 5, the object30is away from the projection surface5A. Referring to (B) ofFIG. 5, the object30is in contact with the region5Ap of the projection surface5A. A so-called distance image is created based on the two photographic images having different locations of the eyepoint obtained by the object sensor13to identify the position of the region5Ap and the position of the object30in the three-dimensional space. If the object30overlaps the region5Ap, and further, if a distance Dx between the object30and the region5Ap is equal to or smaller than a threshold close to zero, e.g., 1-2 centimeters or so, then the projector1determines that the object30touches the predetermined position of the projection surface5A.

Referring to (A) ofFIG. 6, the object30is away from the projection surface5A. Referring to (B) ofFIG. 6, the object30is in contact with a part at which the mark51is projected onto the projection surface5A. When the mark51is projected, the object30blocks light emitted from the projector1, so that the shadow30shaving a shape corresponding to the object30appears on the projection surface5A. Photographing the projection surface5A in such a case obtains a two-dimensional photographic image having the mark51, the object30, and the shadow30scontained therein.

The object30and the shadow30sin the obtained two-dimensional photographic image are shifted from each other in position depending on a distance between the actual object30and the actual projection surface5A. If an amount of the shift in position is equal to or smaller than a threshold close to zero (equal to or smaller than the number of pixels corresponding to the actual distance of, for example, 1-2 centimeters or so), and further, if the object30overlaps the mark51in the photographic image, then the projector1determines that the object30is in contact with the predetermined position of the projection surface5A.

FIG. 7shows an example of the functional configuration of the projector1. The projector1is configured of a first detection portion101, a second detection portion102, a third detection portion103, a photographing control portion104, a mark display control portion105, a display stop control portion106, and so on. The portions are the functional elements implemented in response to the program executed by the CPU15.

The first detection portion101detects that the object30moves away from the photographic space40by, as described above, determining whether or not the object30is present in the photographic space40at regular intervals. Every time determining whether or not the object30is present therein, the first detection portion101identifies the position of the object30in the photographic space40. Based on the change in position of the object30, the first detection portion101detects a direction to which the object30moves away from the photographic space40, i.e., the direction of movement of the object30away from the photographic space40.

After the first detection portion101detects that the object30moves away from the photographic space40, the second detection portion102detects that the object30touches a predetermined position of the projection surface5A by, for example, making the distance measurement as shown inFIG. 5.

After the second detection portion102detects that the object30touches the predetermined position of the projection surface5A, the third detection portion103detects that the object30moves away from the photographic space40by, as with the first detection portion101, determining whether or not the object30is present in the photographic space40at regular intervals.

After the third detection portion103detects that the object30moves away from the photographic space40, the photographing control portion104controls the camera12to take an image of the projection surface5A.

The mark display control portion105controls the projection unit11to display any of marks related to photographing by the camera12in response to the presenter8moving into/out of the photographic space40as discussed later.

When the presenter8or another user selects a projection stop mode in which projection is stopped while the camera12takes an image, the display stop control portion106controls the projection unit11to stop displaying an image during a period from when the third detection portion103detects that the object30moves away from the photographic space40to when the camera12takes an image of the projection surface5A.

FIG. 8shows a first example of operation of displaying a mark by the projector1.FIG. 9shows a second example of operation of displaying a mark by the projector1. In the first and second examples, it is supposed that the presenter8writes a content on the whiteboard5by hand, and the projector1photographs the handwritten content. The projector1displays marks51,52,53, and54related to photographing in the following manner in accordance with the change in position of the presenter8as the object30.

Referring to (A) ofFIG. 8, the image50provided by the personal computer7is projected onto the whiteboard5. The illustrated image50is a bar graph.FIG. 8shows, in (A), that the presenter8has finished writing, by hand, the character string60“please make a modification.” on the whiteboard5with the image50projected.

At this point in time of (A) ofFIG. 8, a part of the right arm of the presenter8is present in the photographic space40. In such a case, the projector1displays the mark54above the image50. The mark54has a non-permission color (red color, for example) defined as a color showing a state in which the photographing control portion104does not permit the camera12to take an image. The mark54is displayed for the purpose of letting the presenter8know that he/she has to move away from the photographic space40in order to instruct the projector1to photograph.

After the mark54is displayed as shown in (A) ofFIG. 8, the presenter8moves away from the photographic space40as shown in (B) ofFIG. 8. The first detection portion101detects the movement of the presenter8. The mark display control portion105receives a notice of the detection. The mark display control portion105controls the projection unit11to finish displaying the mark54and to display the mark51having a photographing permission color, e.g., green color. The photographing permission color is defined as a color showing a state in which the photographing control portion104permits the camera12to take an image.

The mark51is displayed in the form of a button used for the presenter8to instruct the projector1to photograph. In short, the presenter8is capable of instructing photographing by touching a part of the whiteboard5at which the mark51is projected.

The mark51is displayed until a predetermined amount of time (one minute, for example) has elapsed since display of the mark51was started. While the mark51is displayed, the presenter8is allowed to instruct the projector1to photograph.

FIG. 8shows, in (C), a situation immediately after the presenter8touches, with the right hand, the part of the whiteboard5at which the mark51is projected onto the projection surface5A. In the illustrated example, the right hand which has touched the part is away from the projection surface5A, but is still in the photographic space40.

The second detection portion102detects the touch to the part at which the mark51is projected onto the projection surface5A. When being informed of the detection, the mark display control portion105controls the projection unit11in such a manner that the display of the mark51is finished, and instead, the mark52having a photographing waiting color (blue, for example) is displayed. The photographing waiting color is defined as a color showing a state in which the photographing control portion104waits for the third detection portion103to detect that the presenter8moves away from the photographic space40. The mark52is displayed in (C) ofFIG. 8.

The mark52is displayed in order to prompt the presenter8to move away from the photographic space40, which prevents the presenter8from appearing in the photographic image.

The presenter8moves away from the photographic space40after the display of the mark52, which is detected by the third detection portion103. When being informed of the detection, the photographing control portion104controls the camera12to take an image of the projection surface5A. In this way, the photographic image containing the image50and the handwritten character string60is recorded.

From when the photographing control portion104instructs the camera12to take an image to when photographic data obtained through the photographing is completely saved to the non-volatile memory17, the mark display control portion105controls the projection unit11to display a mark53having a color, e.g., yellow, defined as a color showing a state in which photographing is currently conducted as shown in (D) ofFIG. 8.

After the photographic data is completely saved, when at least a part of the human body of the presenter8moves into the photographic space40, the mark54is displayed as with the case of (A) ofFIG. 8.

The first example discussed above may be modified as follows: When the camera12takes an image of the projection surface5A with the projection stop mode selected, it is possible not to display the mark53. Alternatively, the display stop control portion106may control the projection unit11to display the mark53and not to display the image50. Yet alternatively, at the time of photographing, the display stop control portion106controls the projection unit11to stop projecting light, and to photograph the whiteboard5on which only the handwritten character string60as information appears and none of the image50and the mark53appears.

Unlike the first example shown inFIG. 8, according to the second example shown inFIG. 9, the mark54is not displayed, which is clear from the comparison between (A) ofFIG. 8and (A) ofFIG. 9. Except for the difference, the second example is the same as the first example.FIG. 9shows, in (B)-(D), the same situations as those in (B)-(D) ofFIG. 8, respectively. Referring to (B)-(D) ofFIG. 9, the marks51,52, and53are displayed in the second example as with the first example.

The second example may be modified as follows: The mark54can be displayed only for approximately 1-3 seconds after the display of the mark53having a color showing a state in which photographing is being conducted. As with the modification to the first example, when the camera12takes an image of the projection surface5A with the projection stop mode selected, the display stop control portion106may control the projection unit11to stop projecting one or both of the mark53and the image50.

FIG. 10shows an example of operation for changing the display position of the mark51, functioning as a button for the presenter8to instruct photographing, in accordance with the movement direction of the presenter8.

Referring to (A) ofFIG. 10, the presenter8writes the character string60by hand, and stands close to the whiteboard5. A part of the right arm of the presenter8is in the photographic space40.

When the presenter8moves away from the photographic space40to the right as shown inFIG. 10, from the state of (A) to the state of (B), the mark display control portion105controls the projection unit11to display the mark51in the right end which is the front end of the projection surface5A in the movement direction of the presenter8.

On the other hand, when the presenter8moves away from the photographic space40to the left as shown inFIG. 10, from the state of (A) to the state of (C), the mark display control portion105controls the projection unit11to display the mark51in the left end of the projection surface5A. In such a case, the left end of the projection surface5A corresponds to the front end of the projection surface5A in the movement direction of the presenter8.

In order to determine at which end, right and left, of the projection surface5A the mark51is to be displayed, the mark display control portion105obtains, from the first detection portion101, the movement direction detected thereby as discussed above. If the obtained movement direction is the direction from left to right, then the mark display control portion105displays the mark51in the right end of the projection surface5A. If the obtained movement direction is the direction from right to left, then the mark display control portion105displays the mark51in the left end of the projection surface5A.

As the result of detection of the movement direction by the first detection portion101, the mark display control portion105may obtain, from the first detection portion101, a position (last position) immediately before the presenter8moves away from the photographic space40to determine whether the mark51is to be displayed in the left or right of the projection surface5A depending on the last position. If the last position is located in the right half of the photographic space40, then the movement direction of the presenter8is determined to be the direction from left to right. The mark display control portion105therefore determines that the mark51is to be displayed in the right end of the projection surface5A. In contrast, if the last position is located in the left half of the photographic space40, then the movement direction of the presenter8is determined to be the direction from right to left. The mark display control portion105therefore determines that the mark51is to be displayed in the left end of the projection surface5A.

In any case, as shown inFIG. 10, the mark51is displayed at a position close to the presenter8who has moved away from the photographic space40. This enables the presenter8to touch the mark51(button) quickly, and after touching the same, to move away from the photographic space40quickly.

The description goes on to a method for displaying any of the marks51,52,53, and54selectively above the image50.

FIG. 11shows an example of the construction of an image projected by the projector1. The image processing portion18controls the projection unit11to project an image combined by overlaying two layers L1and L2on each other. The layer L1is to draw the image50given by the personal computer7. The layer L2is to draw a photographing-related mark. In the illustrated example, the mark51is drawn in the layer L2.

The use of the two layers L1and L2allows the mark51,52,53, or54to be displayed with the image50, remaining displayed. Only the mark may be projected by making the layer L1blank.

A modification to the style of the photographing-related mark is described below. As discussed above,FIGS. 8 and 9show examples in which the marks51,52,53, and54have colors different from one another. The present invention, however, is not limited to the examples. As shown inFIG. 12A, it is possible to display marks54b,51b,52b,and53bhaving a character string indicating what kind of situation the corresponding mark is displayed. The marks54b,51b,52b,and53bofFIG. 12Ahave character strings and colors different from one another. Instead of such a character string, a state may be displayed by using a graphic designed as a symbol. The mark has any shape of circle, triangle, rectangle, or star.

Alternatively, as shown inFIG. 12B, it is possible to display marks51c,52c,53c,and54chaving a frame-like shape corresponding to the periphery of the projection surface5A. In such a case, the marks51c,52c,53c,and54chave different colors, patterns, thicknesses, or line types from one another. This allows the presenter8to distinguish between the marks51c,52c,53c,and54c.In the example ofFIG. 12B, the marks51c,52c,53c,and54chave different colors from one another.

The frame-like mark51cshows a state in which photographing can be instructed by touching the frame-like mark51c.When the mark51cis displayed, the entire area including the mark51cand the projection surface5A surrounded thereby may be used as a button for the presenter8to instruct photographing. In such a case, the presenter8can instruct the projector1to photograph by touching any position in the projection surface5A.

The foregoing description is summarized with reference to the flowchart ofFIG. 13.

The projector1projects the image50given by the personal computer (PC)7onto the whiteboard5(Step S10).

The mark display control portion105controls the projection unit11to display the mark54having a non-permission color (Step S11). However, in the mode of not displaying the mark54as shown in the example ofFIG. 9, the process in Step S11is bypassed.

Each of the first detection portion101, the second detection portion102, and the third detection portion103starts monitoring an object, specifically, starts processing of detecting the movement of the object30in the photographic space40based on the output from the object sensor13(Step S12).

When the first detection portion101detects that the object30moves away from the photographic space40(YES in Step S13), the mark display control portion105controls the projection unit11to display, as an operational button for instructing photographing, the mark51having a photographing permission color (Step S14).

When the second detection portion102detects a touch, by the object30, to a position in a region of the projection surface5A where the mark51as the button is displayed (YES in Step S15), the mark display control portion105controls the projection unit11to display the mark52having a photographing waiting color (Step S17).

When a predetermined amount of time elapses without a touch, by the object30, to a position in the region where the mark51is displayed (NO in Step S15, and YES in Step S16), the processing returns to Step S11. In such a case, the mark display control portion105controls the projection unit11to finish displaying the mark52, and thereafter, controls the projection unit11to display the mark54in Step S11.

When the second detection portion102has detected the touch to the position in Step S15, and the third detection portion103detects that the object30moves away from the photographic space40(YES in Step S18), the photographing control portion104controls the camera12to take an image of the whiteboard5(Step S19). The photographing control portion104then saves the obtained photographic data to the non-volatile memory17(Step S20).

From when the photographing control portion104instructs the camera12to take an image to when the photographic data is completely saved, in other words, to when the camera12is ready for the next photographing (Step S19and Step S20), the mark display control portion105may control the projection unit11to display the mark53having a color showing the state in which photographing is being conducted.

After the save of the photographic data, the mark display control portion105controls the projection unit11to display the mark54having a non-permission color (Step S21). After that, in a case where the mark54is displayed only for a short time after photographing as shown in the example ofFIG. 9, the mark display control portion105controls the projection unit11to finish displaying the mark54at a time when a predetermined short time has elapsed.

The projection stop mode related to projection by the projection unit11while the camera12takes an image come in the following three options: Displaying only the mark53; displaying only the image50; and displaying none of the mark53and the image50. Independently of whether or not the image50is displayed, when the mark53is not displayed, the mark display control portion105controls the projection unit11not to display the mark53in Step S19. When the image50is not displayed, the display stop control portion106controls the projection unit11not to display the image50immediately before Step S19, and controls the projection unit11to display the image50immediately after Step S20.

The foregoing embodiments enable photographing of an image with the object30such as a user's human body not overlapping an image even when a photographing command is entered through user's action on a button (mark51) displayed on the screen.

In the foregoing embodiments, the projector1is exemplified in which the projection unit11and the camera12are integral with each other. The present invention is not limited thereto, and is also applicable to a case shown inFIG. 14where a display means for projecting an image is provided in a device independently of a device having a photographing means for photographing a projection surface.

Referring toFIG. 14, a system2for projection and photographing is provided with a portable projector3and a portable information terminal4having a camera (photographing means). The information terminal4is, for example, a smartphone. The projector3projects an image onto, for example, paper6used as a writable screen placed on a desk. The information terminal4photographs the paper6. The projector3and the information terminal4are so positioned that the entire area of a projection surface6A may be photographed.

As with the projector1, the system2monitors movement of an object in and out of a part corresponding to field of view for photographing (referred to as a photographic space42) of a space between the projection surface6A and the camera of the information terminal4. The system2then controls the camera of the information terminal4to photograph the projection surface6A so as to prevent the object from appearing in the photographic image. Stated differently, the system2has functional elements similar to the first detection portion101, the second detection portion102, the third detection portion103, the photographing control portion104, the mark display control portion105, and the display stop control portion106, all of which are shown in inFIG. 7.

The information terminal4or the projector3may be configured to provide all of the first detection portion101, the second detection portion102, the third detection portion103, the photographing control portion104, the mark display control portion105, and the display stop control portion106. Alternatively, the following configuration is also possible: The information terminal4is so configured to provide some of the first detection portion101, the second detection portion102, the third detection portion103, the photographing control portion104, the mark display control portion105, and the display stop control portion106, and the projector3is so configured to provide the other functional portions.

For example, the information terminal4may include a functional element to display a mark in the projector3as an operational guide or a button for a user of the system2to instruct photographing, and a functional element to detect that the object moves away from the photographic space42and to detect a touch to a position where the mark is displayed. The projector3may include a functional element to control the projector3to photograph the projection surface6A when the information terminal4detects that the object moves away from the photographic space42after the position where the mark is displayed is touched.

Another configuration is also possible in which a determination as to whether or not the object is present in the photographic space40or42is made through image recognition based on the photographic image of the projection surface5A or6A photographed by the camera12or the camera of the information terminal4.