Abstract:
An image forming apparatus in which a client terminal and a printer are connected to a communication network, includes an image capturing part provided in a vicinity of the printer, a recognizing part to sense a person approaching the printer and to recognize an identity of the person, based on a face image of an image captured by the image capturing part, and a control part to control the printer to perform a print job issued by the person, based on a recognition output from the recognizing part.

Description:
BACKGROUND OF THE INVENTION  
       [0001]     1. Field of the Invention  
         [0002]     The present invention relates to an image forming apparatus in which a printer, a duplicating machine, etc., are connected to a communication network and a method of controlling the same. More particularly, the present invention relates to an image forming apparatus and a method of controlling the same capable of timely performing a print job of a user based on the movement of the user and of improving the convenience of users and security.  
         [0003]     2. Description of the Related Art  
         [0004]     Conventionally, a personal computer designates the number of print copies and a print range to a printer to perform printing. When a printer is connected to a network, the printer performs a printing process in the order of print commands received from a plurality of users requiring printing. Therefore, a paper outlet is piled up high with printed sheets required by the plurality of users. However, in recent years, since it has been strongly demanded to protect the security and privacy of users, user interfaces that require inputting a text ID or password at the time of printing have been widely used. For example, there is a print system having a private print function in which, when a personal computer instructs a print system to perform printing, a text ID, called an ‘access name’, is input through the personal computer, so that, when the ‘access name’ is not input, printing is not performed. However, the user interface has problems in that it is necessary to perform a character input operation on a personal computer side and a printer side and to memorize the ‘access name’.  
         [0005]     Therefore, it is demanded to timely provide a print service with respect to a print job issued by a user by determining that the user having issued the print job approaches a print or a print terminal to get a printout.  
         [0006]     Further, there are JP-A-6-59010 and JP-A-2002-111924 as patent documents related to the present invention.  
         [0007]     JP-A-6-59010 discloses a technique in which a CCD sensor is provided in an image forming apparatus to determine the moving direction or face of a person, thereby controlling a power supply/preheating function.  
         [0008]     Furthermore, in JP-A-2002-111924, a camera captures a face of a person who is willing to use an image forming apparatus, and it is determined whether the image captured by the camera corresponds to a registered user. In this way, JP-A-2002-111924 controls whether to use the image forming apparatus.  
         [0009]     However, JP-A-6-59010 and JP-A-2002-111924 do not disclose a technique of timely performing a print job issued by a user based on the approach of the user to a printer or a print terminal.  
       SUMMARY OF THE INVENTION  
       [0010]     The present invention is designed to solve the above-mentioned problems, and provides an image forming technique capable of timely performing a print job issued by a user without leaving a printout alone for a long time, by determining that the user having issued the print job approaches a print or a print terminal to get the printout.  
         [0011]     The present invention solves the problems by additionally providing an image capturing part such as a camera, a face detecting part, a face recognizing part, and a print permitting part to a printer. In order to supply image data to a face detecting part and a face recognizing part, the image capturing part is preferably arranged at a place suitable for acquiring the face information of a printer operator. In this way, the present invention can realize the following functions.  
         [0012]     (1) A function of performing printing after confirming each person.  
         [0013]     (2) A function of changing the priority of a print queue after confirming each person.  
         [0014]     (3) A function of performing a print control after confirming each person.  
         [0015]     For the function of performing printing after confirming each person, the face detecting part detects the approaching of a person who has issued a print instruction to the printer and then inputs a face image of the person to the face recognizing part, thereby obtaining the face recognition result. When the face recognizing part determines that the input face image corresponds to the person who has issued the print instruction, the print permitting part changes a print enable flag of the print instruction issued by the person from a disable state to an enable state, so that printout is automatically performed. In this case, it is necessary to register face recognition data onto the face recognizing part prior to performing the print instruction using a personal computer. In addition, the print enable flag of each print instruction is initially in the disable state.  
         [0016]     For the function of changing the priority of a print queue after confirming each person, the face detecting part detects the approaching of a person who has issued a print instruction to the printer and then inputs a face image of the person to the face recognizing part, thereby obtaining the face recognition result. When the face recognizing part determines that the input face image corresponds to the person who has issued the print instruction, a previously registered print priority attribute of the person who has issued the print instruction and approached the printer and that of another person whose documents are currently being printed are checked and compared. As a result of the check, when the person who has issued the print instruction and approached the printer has a higher priority, the print enable flag of the print instruction is changed from the disable state to the enable state, and then the printing is automatically performed (the current printing job having a lower priority is changed to a hold state). After the printing having a higher priority is completed, the printing having a lower priority in the hold state returns to the print state.  
         [0017]     For the function of performing a print control after confirming each person, the control is realized in a printer having a plurality of print outlets capable of restricting the carrying-out of documents. The face detecting part detects the approaching of a person who has issued a print instruction to the printer and then inputs a face image of the person to the face recognizing part, thereby obtaining the face recognition result. When the face recognizing part determines that the input face image corresponds to the person who has issued the print instruction, the lock of the print outlet where the printout of the person who has issued the print instruction is held is released, so that the printout can be carried out.  
         [0018]     Further, the present invention will be described below.  
         [0019]     According to an aspect of the present invention, an image forming apparatus in which a client terminal and a printer are connected to a communication network, includes an image capturing part provided in a vicinity of the printer, a recognizing part to sense a person approaching the printer and to recognize an identity of the person, based on a face image of an image captured by the image capturing part, and a control part to control the printer to perform a print job issued by the person, based on a recognition output from the recognizing part.  
         [0020]     In this way, it is possible to appropriately process the print job of the user and the output thereof by detecting a user approaching to the printer and by specifying the user identity, and thus it is possible to provide a timely service and to improve security.  
         [0021]     Further, for example, the control part processes the print job issued by the person based on the recognition output from the recognizing part.  
         [0022]     Alternatively, the control part assigns a higher priority to the print job issued by the person in print queue, based on the recognition output from the recognizing part.  
         [0023]     Furthermore, the control part release a print lock of the person based on the recognition output from the recognizing part.  
         [0024]     Moreover, specifically, the recognizing part determines whether a person approaches the printer based on an arrangement region of the face image in the captured image, the moving direction of the face image, and the direction of the face in the face image.  
         [0025]     Further, the recognizing part may determine whether the person approaches the printer based on the size of the face image.  
         [0026]     Furthermore, the user identity is specified by comparing the registered feature amount of each user face to a feature amount extracted from the face image in the captured image. In addition, it is possible to adopt various well-known recognizing methods.  
         [0027]     Moreover, an attribute indicating whether to perform print control based on the face recognition is added to the print job, and the control part may control the printer only when the attribute indicates the print control to be performed. The print job includes, according to the number of copies or an emergency, a job to be immediately printed as well as a job unnecessary to be immediately printed because it takes a long time to be printed, and so on. The user specifies whether to perform control as the print attribute, so that it is possible to cope with such a demand.  
         [0028]     Further, the user may specify the contents of the print control.  
         [0029]     Furthermore, the present invention can be realized by not only an apparatus or a system but a method. In addition, it goes without saying that a portion of the present invention may be formed in a software. Of course, a software product used for allowing a computer to execute the software is also included in the technical scope of the present invention.  
         [0030]     The above-mentioned aspects and other aspects of the present invention will be explained in detail using the following embodiment described in the appended claims.  
         [0031]     According to the present invention, it is possible to appropriately process the print job of the user and the output thereof by detecting a user approaching the printer and by specifying the user identity, and thus it is possible to provide a timely service and to improve security.  
     
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0032]     Preferred embodiments of the present invention will be described in detail based on the following figures, wherein:  
         [0033]      FIG. 1  is a view illustrating the overall structure of an image forming system according to an embodiment of the present invention;  
         [0034]      FIG. 2  is a block diagram illustrating the structure of the main part of the embodiment of the present invention;  
         [0035]      FIG. 3  is a view illustrating an algorithm for determining the approach of a face image in the embodiment of the present invention;  
         [0036]      FIG. 4  is a view illustrating an example of a user interface screen of a print option in the embodiment of the present invention;  
         [0037]      FIG. 5  is a flow chart illustrating the overall operation of the embodiment of the present invention;  
         [0038]      FIG. 6  is a portion of a flow chart illustrating the operation of an approach determining routine according to the embodiment of the present invention;  
         [0039]      FIG. 7  is another portion of the flow chart illustrating the operation of the approach determining routine according to the embodiment of the present invention; and  
         [0040]      FIG. 8  is a view illustrating a modification of the embodiment of the present invention. 
     
    
     DETAILED DESCRIPTION OF THE INVENTION  
       [0041]     Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings.  
         [0042]      FIG. 1  shows the overall structure of an image forming system according to an embodiment of the present invention. In  FIG. 1 , a client terminal  10 , a print server  20 , a print terminal  30 , and the like are connected to a communication network  60 . The communication network  60  may be a LAN, a wide area network, or the like. A printer  40  and a video camera  50  are connected to the print terminal  30 . The video camera  50  is arranged to determine the approaching of a user to the printer  40 . A plurality of the video cameras  50  may be provided.  
         [0043]     The client terminal  10  is an information terminal such as a personal computer, and functions to specify a print file and to transmit a print job to the print server  20  or the printer  40 . The client terminal  10  may be a fixed terminal, a movable terminal, or a mobile communication apparatus. When a print job is transmitted to the print server  20 , the print job is held in a spool of the print server  20  and is then printed out from the corresponding printer  40  by operating the print terminal  30  or another print terminal (not shown). The print job transmitted to the printer  40  is held in the spool thereof or a spool of the print terminal  30 . In this case, the print jobs held in the spools of the printer  40  and the print terminal  30  can be performed in the waiting order or can be suspended by enable and disable flags. In the present embodiment, the disable flag is set to suspend the printing.  
         [0044]     Further, for example, the user can set various print options using a print option screen shown in  FIG. 4 . Although the present invention is not limited to the example shown in  FIG.4 , in the present embodiment, it is possible to set various printing modes, such as an N up mode (a mode for printing a plurality of pages on a sheet of paper), a both side printing mode, a print time reservation, a face recognition print, etc. When the face recognition print is selected, a face recognition print attribute is turned on, so that the corresponding print job can be controlled based on the face recognition. When the face recognition print attribute is in an off state, control is disabled based on the face recognition. Further, control may be set to on or off state for every user, not every print job, based on the face recognition.  
         [0045]     The print terminal  30  performs a print job in the spool of the printer  40 . For example, the print terminal  30  can output a print job from the printer  40  or can output a print job in the spool of the print server  20  to the printer  40 . Further, the print terminal  30  functions to remove the print job and to change attributes. In general, the print terminal  30  performs certification by a password or a card. Of course, it is also possible to set the print terminal  30  not to perform the certification according to a required security level or user&#39;s convenience.  
         [0046]     The printer  40  is typically a xerography-type image output apparatus, but may have a duplicating function, an image inputting function, and a facsimile function. The printer  40  has a print control mechanism. For example, when the user logging in the print terminal  30  has a carrying-out authority (the owner of a print job), the lock of the print control mechanism of the printer  40  is released. In the present embodiment, the print control mechanism is released based on the face recognition.  
         [0047]      FIG. 2  is a view schematically showing a functional block of the print terminal  30 . In  FIG. 2 , the print terminal  30  includes a face detection processing unit  301 , a face recognition processing unit  302 , an approach determining unit  303 , a printer control unit  304 , etc. Although not shown, the print terminal  30  has an operation/display unit composed of a liquid crystal panel and provides various interfaces for a user. The face detection processing unit  301  determines whether a face image is present in a captured image. For example, after determining the face, the face detection processing unit  301  detects, for example, the midpoint position and movement of the face. Further, the face detection processing unit  301  determines the direction or size of the face. The face recognition processing unit  302  extracts a feature amount from the face image and matches it with the feature amount of each registered user to output a user identity (user ID). The approach determining unit  303  outputs a signal indicating an approached state and the user ID, based on the outputs from the face detection processing unit  301  and the face recognition processing unit  302 . The printer control unit  304  starts to perform a print job of the user based on the user ID.  
         [0048]     The printer starts to perform an image process by detecting a user approaching the printer.  
         [0049]     An approach detecting method is as follows, but is not limited thereto.  
         [0050]     A face detecting process is performed on image data obtained by a camera  50  arranged in a vicinity of the printer  40 . After the face is detected, the position, size, and direction of the face are output to the approach determining unit  303 . Further, the detected face image is output to the face recognition processing unit  302 . Then, the face recognition processing unit  302  determines whose the detected face is. The approach determining unit  303  first detects a moving direction of a specific face in an image frame by the difference between the output time-sequentially transmitted from the face recognition processing unit  302  and the central position of the face input from the face detection processing unit  301 . When the central position of the face is outside a region A ( FIG. 3 ) set in the image frame, a moving direction thereof traverses the region A set in the image frame, and the direction of the face is within a predetermined angle range, it is determined that a specific face is in an approaching state (which is represented by reference numeral ‘1’ in  FIG. 3 ). Further, when the central position of the face is within the region A set in the image frame, the size of the face is smaller than a predetermined size, and the direction of the face is within a predetermined angle range, it is determined that a specific face is in the approaching state (which is represented by reference numeral ‘2’ in  FIG. 3 ). When the approach determining unit  303  determines that a face approaches, it outputs to the printer control unit a signal indicating an approaching state and the previously registered user ID of approaching user.  
         [0051]     When the signal indicating an approaching state and a code number are input, the printer control unit  304  retrieves a text having the same user ID from text data that is transmitted through a network to which a code number is previously given and is held in a storage device (not shown) of the printer control unit  304 , and then outputs it to the printer  40 . In this case, the printer control unit  304  performs a process of decoding the text data and a rasterizing process.  
         [0052]     Hereinafter, the operation of the embodiment according to the present invention will be described in detail.  
         [0053]      FIG. 5  is a flow chart illustrating the overall operation, and the following steps are performed.  
         [0054]     [Step S 10 ]: After determining that a face image is included in a captured image, an approach determining routine starts.  
         [0055]     [Step S 11 ]: The approach determining routine ( FIGS. 6 and 7 ) is performed to output a user ID and an approach signal.  
         [0056]     [Step S 12 ]: A print job list is fetched according to the user ID. Further, jobs in the spool of the print sever  20  as well as the jobs in the spool of the printer  40  or the print terminal  30  may be included in an object to be printed. In this case, the information of the print job is received from the print server  20 .  
         [0057]     [Step S 13 ]: A first job is extracted from the fetched print jobs.  
         [0058]     [Step S 14 ]: It is determined whether a face recognition print attribute is present. When the face recognition print attribute is present, Step S 15  is performed. On the other hand, when it is absent, the process returns to Step S 13 , and Steps S 13  and S 14  are repeatedly performed.  
         [0059]     [Step S 15 ]: A print job is performed using the printer  40 .  
         [0060]     [Step S 16 ]: When the remaining job is present, the process returns to Step S 13 , and Steps S 13  to S 16  are repeatedly performed. On the other hand, when the remaining is absent, the process returns to Step S 10 .  
         [0061]      FIGS. 6 and 7  are flow charts illustrating the operation of the approach determining routine (S 11 ) in detail, and the process thereof is performed as follows. Further, the face image is processed at a predetermined short interval.  
         [0062]     [Step S 20 ]: It is determined whose the previously detected face image is. When there is no user corresponding to the previously detected face image, Step S 21  is performed. When there is a user corresponding to the previously detected face image, Step S 22  is performed.  
         [0063]     [Step S 21 ]: When there is no user corresponding to the previously detected face image, information on the position, size, and angle of the face is held as a new detection entry.  
         [0064]     [Step S 22 ]: The held information on the position, size, and angle of the face is acquired.  
         [0065]     [Step S 23 ]: It is determined whether the present face position is within the region A. When the position is within the region A, Step S 24  is performed. On the other side, when the position is outside the region A, Step S 26  is performed.  
         [0066]     [Step S 24 ]: It is determined whether the moving direction and the direction of the face are within set values. As a result, if within the set values, Step S 25  is performed. On the other hand, if not within the set values, the process is completed.  
         [0067]     [Step S 25 ]: The user ID and the approach signal are output, and then the process is completed.  
         [0068]     [Step S 26 ]: It is determined whether the size and direction of the face are within set values. As a result, if within the set values, Step S 27  is performed. On the other hand, if not within the set values, the process is completed.  
         [0069]     [Step S 27 ]: The user ID and the approach signal are output, and then the process is completed.  
         [0070]     In the above-mentioned operation, control when the print job is executed is described. In addition, a change in the priority of the spool of the print job and the print control mechanism are controlled in the same manner.  
         [0071]     The complete description of the embodiments of the present invention has been described as elaborated above.  
         [0072]     Further, the present invention is not limited to the above-mentioned embodiment, and various modifications and changes can be made without departing from the scope and spirit of the present invention. For example, as described above, the image forming apparatus is described as an example of the present invention, but the present invention can be applied to service providing apparatuses, such as a vending machine and an article storing apparatus. For example, it is possible to directly receive a desired article, by specifying an article which will be delivered and then determining the article approaching to the apparatus.  
         [0073]      FIG. 8  is a view illustrating the structure of such a service providing system. In  FIG. 8 , the same components as those in  FIG. 1  have the same reference numerals. In the service providing system shown in  FIG. 8 , a service providing device  80  is controlled by a service-providing-device control unit  70 , so that each service providing process is controlled based on the face image, similar to the above.