Patent Abstract:
A remote supporting apparatus including a first receiving unit that receives information about a first annotation image from another device, a projecting unit that projects an image on a predetermined projection region in cooperation with the another device in accordance with the information about the first annotation image received from the first receiving unit, a recording unit that records an image of the projection region, a first transmitting unit, a second transmitting unit, a second receiving unit, and a displaying unit that displays an image in accordance with the information about the recorded image received by the second receiving unit.

Full Description:
BACKGROUND 
       [0001]    1. Technical Field 
         [0002]    The present invention generally relates to a remote supporting apparatus that performs an operation for projecting an image on a predetermined projection region in cooperation with another device, a remote supporting system that includes the remote supporting apparatus, and a remote supporting method to be utilized in the remote supporting apparatus. 
         [0003]    2. Related Art 
         [0004]    During presentations, images of materials (slides) prepared with presentation software in personal computers (PCs) are often projected on screens by projectors. 
         [0005]    In such image processing, an image sent from a remote place is projected with a projector, an image of the screen having the image projected thereon is recorded, and the image information obtained through the recording is transmitted to a remote place. Alternatively, a picture drawn by a user with a pen or the like is displayed on an electronic whiteboard at own site, and a picture that is originally drawn on an electronic whiteboard is also displayed at another site and is received and converted by a communication controller on the electronic whiteboard at the own site. 
       SUMMARY 
       [0006]    An aspect of the present invention provides a remote supporting apparatus including: a first receiving unit that receives information about a first annotation image from another device; a projecting unit that projects an image on a predetermined projection region in cooperation with the another device in accordance with the information about the first annotation image received from the first receiving unit; a recording unit that records an image of the projection region; a first transmitting unit that transmits image information about the recorded image obtained by the recording unit, as image information about an image to be displayed on the another device, to the another device; a second transmitting unit that transmits information about a second annotation image to be projected on a projection region at a site at which the another device is placed; a second receiving unit that receives information about a recorded image obtained by recording an image of the projection region at the site at which the another device is placed, the information being received from the another device; and a displaying unit that displays an image in accordance with the information about the recorded image received by the second receiving unit. 
     
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0007]    Embodiments of the present invention will be described in detail based on the following figures, wherein: 
           [0008]      FIG. 1  illustrates the basic structures of remote supporting apparatuses that constitute a remote supporting system; 
           [0009]      FIG. 2  illustrates the structure of the remote supporting system; 
           [0010]      FIG. 3  is a flowchart showing an operation to be performed by each server system; 
           [0011]      FIG. 4  is a flowchart showing an operation to be performed by each client system; 
           [0012]      FIG. 5  illustrates another example structure of a remote supporting apparatus; and 
           [0013]      FIG. 6  illustrates another example structure of a remote supporting system. 
       
    
    
     DETAILED DESCRIPTION 
       [0014]    A description will now be given, with reference to the accompanying drawings, of exemplary embodiments of the present invention.  FIG. 1  illustrates the structures of remote supporting apparatuses that form a remote supporting system. In  FIG. 1 , a server system  110  as a remote supporting apparatus and a whiteboard  160  are placed at site A, and a client system  200  as a remote supporting apparatus is placed at site B. 
         [0015]    The server system  110  at the site A includes a drawing server  112 , a video server  114 , a projector  116 , a video camera  118 , and a half mirror  120 . Meanwhile, a client system  240  at the site B includes a monitor  242 . The drawing server  112  and the video server  114  in the server system  110  at the site A are communicably connected to the client system  240  at the site B with the Internet  400 . 
         [0016]    In this exemplary embodiment, the same client system as the client system  240  at the site B is provided together with the server system  110 , so as to form a remote supporting apparatus at the site A. The same server system as the server system  110  at the site A is provided together with the client system  240 , so as to form a remote supporting apparatus at the site B. A whiteboard is also provided at the site B, thereby forming a remote supporting system. 
         [0017]      FIG. 2  illustrates the structure of the remote supporting system. In the remote supporting system illustrated in  FIG. 2 , the server system  110  and a client system  140  as a remote supporting apparatus, and the whiteboard  160  are set at the site A. The client system  140  includes a monitor  142 . Meanwhile, a server system  210  and the client system  240  as a remote supporting apparatus, and a whiteboard  260  are set at the site B. The server system  210  is formed with a drawing server  212 , a video server  214 , a projector  216 , a video camera  218 , and a half mirror  220 . The client system  140  at the site A is communicably connected to the drawing server  212  and the video server  214  in the server system  210  at the site B with the Internet  400 . 
         [0018]    The server system  110  at the site A performs an operation for projecting an image at the site A in response to a request from the client system  240  at the site B. Likewise, the server system  210  at the site B performs an operation for projecting an image at the site B in response to a request from the client system  140  at the site A. The operations are described below in greater details. 
         [0019]    In accordance with an operation instruction of the user at the site B, the client system  240  at the site B generates a drawing command as an instruction for projecting an annotation image such as an explanatory note, and information about the annotation image. The client system  240  then transmits the drawing command and the information to the drawing server  112  in the server system  110  at the site A via the Internet  400 . 
         [0020]    In response to the received drawing command, the drawing server  112  outputs the information about the annotation image together with the drawing command to the projector  116 . If there is image information that has already been held by the drawing server  112 , the drawing server  112  outputs the image information to the projector  116 . The projector  116  projects an image in accordance with the input image information, onto the whiteboard  160  via the half mirror  120 . 
         [0021]    An image is projected on the white board  160  by the projector  116 , and characters or the likes are written on the whiteboard  160  by the user of the site A. The video camera  118  records, through the half mirror  120 , the whiteboard  160  having an image projected thereon and characters or the likes written thereon. The video camera  118  has various settings for panning, tilting, zooming, and the likes, so that the video camera  118  has the same field angle and the same optical axis as the projector  116 . The image information obtained through the recording is output to the video server  114 . The video server  114  transmits the input image information to the client system  240  at the site B via the Internet  400 . 
         [0022]    The client system  240  at the site B displays the received image information on the monitor  242 . Accordingly, the user of the site B can recognize the image on the whiteboard  160  at the site A. 
         [0023]    The same operation as above is performed between the client system  140  at the site A and the server system  210  at the site B. In accordance with an operation instruction from the user of the site A, the client system  140  at the site A generates a drawing command as an instruction for projecting an annotation image, and information about the annotation image. The client system  140  then transmits the drawing command and the information to the drawing server  212  in the server system  210  at the site B via the Internet  400 . 
         [0024]    In response to the received drawing command, the drawing server  212  outputs the information about the annotation image together with the drawing command to the projector  216 . If there is image information that has already been held by the drawing server  212 , the drawing server  212  outputs the image information to the projector  216 . The projector  216  projects an image in accordance with the input image information, onto the whiteboard  260  via the half mirror  220 . 
         [0025]    An image is projected on the whiteboard  260  by the projector  216 , and characters or the likes are written on the whiteboard  260  by the user of the site B. The video camera  218  records, through the half mirror  220 , the whiteboard  260  having an image projected thereon and characters or the likes written thereon. The video server  214  transmits the input image information to the client system  140  at the site A via the Internet  400 . 
         [0026]    The client system  140  at the site A displays the received image information on the monitor  142 . Accordingly, the user of the site A can recognize the image on the whiteboard  260  at the site B. 
         [0027]    In the following, the operations of the server system  110  at the site A and the client system  240  at the site B are described, with reference to flowcharts. 
         [0028]      FIG. 3  is a flowchart showing the operation to be performed by the server system  110  at the site A. The drawing server  112  in the server system  110  at the site A outputs already held image information to the projector  116 . The projector  116  projects the image in accordance with the input image information (the initial image), onto the whiteboard  160  through the half mirror  120  (S 101 ). 
         [0029]    The video camera  118  starts recording the whiteboard  160  (S 102 ). The image information through the recording (the information about the recorded image) is output to the video server  114 . The video server  114  starts transmitting the input image information to the client system  240  at the site B (S 103 ). After that, the video camera  118  continues recording the whiteboard  160 , and the video server  114  keeps transmitting image information to the client system  240  at the site B every time image information is input. 
         [0030]    The drawing server  112  determines whether it has received a drawing command from the client system  240  at the site B (S 104 ) If having received a drawing command, the drawing server  112  outputs the information about the annotation image attached to the drawing command to the projector  116  in accordance with the drawing command. The projector  116  projects the image in accordance with the input information about the annotation image, onto the whiteboard  160  via the half mirror  120  (S 105 ). As a result, the initial image and an image such as an explanatory note in accordance with the information about the annotation image attached to the drawing command from the client system  240  at the site B are projected onto the whiteboard  160 . In the image recorded by the video camera  118 , problems such as uncolored portions might be caused in the image formed in accordance with the information about the annotation image from the client system  240  at the site B. With such problems being taken into consideration, the video server  114  may generate image information about a combined image formed by overlapping the image formed in accordance with the information about the annotation image from the client system  240  at the site B on the position of the image formed in accordance with the information about the annotation image from the client system  240  at the site B in the image recorded by the video camera  118 . The video server  114  may transmit the image information to the client system  240  at the site B. 
         [0031]    After the image projection in step S 105 , or if the drawing server  112  determines that it has not received a drawing command yet in step S 104 , the server system  110  determines whether the user of the site A has issued an instruction to end the operation (S 106 ). If there is an instruction to end the operation, the operation comes to an end. If there is not such an instruction, the procedure for determining whether the drawing server  112  has received a drawing command (S 104 ) and the procedures thereafter are repeated. 
         [0032]      FIG. 4  is a flowchart showing the operation to be performed by the client system  240  at the site B. In accordance with an operation instruction from the user of the site B, the client system  240  generates the drawing command for projecting an annotation image such as an explanatory note on the whiteboard  160  at the site A or the whiteboard  260  at the site B (S 201 ). 
         [0033]    The client system  240  then performs an image drawing process, and generates the information about the annotation image to be projected onto the whiteboard  160  at the site A or the whiteboard  260  at the site B (S 202 ). The client system  240  determines whether the destination of the drawing command is the server system  210  that is also located at the site B (S 203 ). For example, the user of the site B designates the destination through the operation instruction for the generation of the drawing command, and the client system  240  detects the destination from the operation instruction. 
         [0034]    In a case where the destination of the drawing command is not the server system  210  also located at the site B, the client system  240  adds the annotation image information generated in step S 202  to the drawing command, and transmits the drawing command to the drawing server  112  in the server system  110  at the site A (S 204 ) As a result, the server system  110  carries out the procedure of step S 105  of  FIG. 3 . 
         [0035]    In a case where the destination of the drawing command is the server system  210  also located at the site B, the client system  240  adds the annotation image information generated in step S 202  to the drawing command, and transmits the drawing command to the server system  210  (S 205 ). In this case, the drawing server  212  in the server system  210  outputs the annotation image information attached to the received drawing command to the projector  216 , and the projector  216  projects the image in accordance with the annotation image information onto the whiteboard  260 . 
         [0036]    After the transmission of the drawing command in step S 204  or S 205 , the client system  240  determines whether there is an instruction to end the operation from the user of the site B (S 206 ). If there is an instruction to end the operation, the operation comes to an end. If there is not an operation to end the operation, the procedure for generating a drawing command (S 201 ) and the procedures thereafter are repeated. 
         [0037]    The server system  210  at the site B performs the same operation as that illustrated in  FIG. 3 , and the client system  140  at the site A performs as same operation as that illustrated in  FIG. 4 . 
         [0038]    As described above, in the remote supporting system, an image in accordance with annotation image information from the client system  240  at the site B is displayed on the whiteboard  160  at the site A. Likewise, an image in accordance with annotation image information from the client system  140  at the site A is displayed on the whiteboard  260  at the site B. A recorded image of the whiteboard  260  at the site B is displayed on the monitor  142  at the site A, and a recorded image of the whiteboard  160  at the site A is displayed on the monitor  242  at the site B. Accordingly, the whiteboard as the projection region at each site is used as a work area, so that cooperative operations can be performed between the sites, and communications become easier between users. 
         [0039]    The present invention is not limited to the above-described exemplary embodiment, but various changes may be made to it.  FIG. 5  illustrates another example structure of the remote supporting apparatus at the site A. In  FIG. 5 , a server system  110  as a remote supporting apparatus, a client system  170  for monitoring own site, a client system  180  for another site, and a whiteboard  160  are placed at the site A. The same structure may be employed at the site B. 
         [0040]    The client system  180  for another site includes a monitor  182 . The client system  180  for another site receives image information obtained by recording an image of the whiteboard  260 , and displays the image in accordance with the image information on the monitor  182 . 
         [0041]    The user of the site A can issue such an operation instruction as to designate a part of the image displayed on the monitor  182  to be displayed on the whiteboard  160 . If such an instruction is issued, the client system  180  for another site selects the designated part of the image as a copy area  183 , and transmits the image information about the copy area  183  to the client system  170  for monitoring own site. 
         [0042]    Upon receipt of the image information about the copy area  183 , the client system  170  for monitoring own site generates image information about a combined image of an image in accordance with the image information and an image in accordance with already held image information. The client system  170  for monitoring own site also displays the combined image on a monitor  172 , and transmits the image information about the copy area  183  to the server system  110 . 
         [0043]    The drawing server  112  in the server system  110  outputs the received image information about the copy area  183  to the projector  116 . The projector  116  projects the combined image about the image information on the whiteboard  160 . 
         [0044]    In this manner, at the site A, a part of the image in accordance with the image information obtained by recording an image of the whiteboard  260  at the site B is selected in accordance with an operation instruction from the user of the site A, and the selected part of the image is projected on the whiteboard  160 . 
         [0045]    In the above-described exemplary embodiment, remote supporting apparatuses are placed at the two sites A and B in a remote supporting system. However, the present invention may be applied to a remote supporting system having remote supporting apparatuses placed at three or more sites.  FIG. 6  illustrates an example case where remote supporting apparatuses are placed at three sites A, B, and C. At the site A, a server system  110  as a remote supporting apparatus and client systems  140 - 1  and  140 - 2  are placed. At the site B, a server system  210  as a remote supporting apparatus and client systems  240 - 1  and  240 - 2  are placed. At the site C, a server system  310  as a remote supporting apparatus and client systems  340 - 1  and  340 - 2  are placed. 
         [0046]    In this imaging system, the server system  110  at the site A is communicably connected to the client system  240 - 1  at the site B and the client system  340 - 2  at the site C via the Internet  400 . In response to each request from the client systems  240 - 1  and  340 - 2 , the server system  110  at the site A performs an image projecting operation at the site A. Likewise, the server system  210  at the site B is communicably connected to the client system  140 - 1  at the site A and the client system  340 - 1  at the site C via the Internet  400 . In response to each request from the client systems  140 - 1  and  340 - 1 , the server system  210  at the site B performs an image projecting operation at the site B. The server system  310  at the site C is communicably connected to the client system  140 - 2  at the site A and the client system  240 - 2  at the site B via the Internet  400 . In response to each request from the client systems  140 - 2  and  240 - 2 , the server system  310  at the site C performs an image projecting operation at the site C. 
         [0047]    As described so far, the remote supporting apparatuses, the remote supporting systems, and the remote supporting method in accordance with the present invention enable smooth communications between users at different sites, and are suitable for remote supporting apparatuses. 
         [0048]    The foregoing description of the exemplary embodiments of the present invention has been provided for the purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise forms disclosed. Obviously, many modifications and variations will be apparent to practitioners skilled in the art. The exemplary embodiments were chosen and described in order to best explain the principles of the invention and its practical applications, thereby enabling others skilled in the art to understand the invention for various embodiments and with the various modifications as are suited to the particular use contemplated. It is intended that the scope of the invention be defined by the following claims and their equivalents.

Technology Classification (CPC): 8