PHOTOGRAPH SHOOTING DEVICE, CONTROL METHOD OF SAME, AND PROGRAM

A photograph shooting device or the like which enables detection of illegal shooting, and implement electronic application using this device. The device performs operation instruction on a subject (operation instruction unit), and determines whether the subject is a living body by shooting a moving image of the movement of the subject responding to operation instruction and analyzing the shot image (living body determination unit). The device shoots an application image that is a photograph used for an application procedure (application image shooting unit). The device compares image cut from moving image with the application image, and determines whether the shot persons match (same person determination unit). When the subject is determined to be the living body, and a subject responding to the operation instruction is determined to match the subject in the application image, the device outputs the application image to a prescribed server (data output unit).

TECHNICAL FIELD OF THE INVENTION

The present invention relates to a photograph shooting device, a control method of the photograph shooting device, and a program, particularly relating to electronic applications for administrative procedures and the like.

BACKGROUND OF THE INVENTION

Electronic application schemes utilizing smartphones and the like are beginning to penetrate into the marketplace. For example, it has been planned to introduce electronic application procedures into applications of new passports or renewal of passports, renewal of individual number cards, driver's license cards, etc., and to realize such the application procedures online. Identity verification using an identity verification medium (an identification card), such as a driver's license card, is essential for the applications. However, without face-to-face verifications and with personal devices being used in electronic applications, there are concerns of illegal acts of impersonation.

Patent Document 1 discloses a technology for detection of forgery or the like of an ID card (an identity verification medium) such as a driver's license card. An authenticity determination device of Patent Document 1 includes a reading device for reading an ID card and a computer for determining authenticity, in which the computer determines authenticity of the ID card by determining whether there are any inconsistencies in contents of character information obtained from a surface image of the ID card or not.

RELATED ART

Patent Document

[Non-Patent Document 1] “Time schedule for digitization of administrative procedures” by Ministry of Foreign Affairs of Japan (online document searched through the Internet on Dec. 15, 2020<http://www.moj.go.jp/content/001318660.pdf>)

SUMMARY OF THE INVENTION

Problems to be Solved by the Invention

Unfortunately, as mentioned above, a risk of data forgery is much higher in electronic applications than in real face-to-face application procedures, and thus it is difficult to perform strict identity verification. It is also difficult to ask users for appropriate photographs if the users use their own smartphones or PCs for the electronic applications. In fact, according to Non-Patent Document 1 (“Time schedule for digitization of administrative procedures”), smartphones or home PCs are not allowed to be used in electronic applications of the next-generation passports. Also, even when shooting a photograph with a photograph shooting device, there may be illegal acts of photo shooting in which a subject wears an elaborate mask or displays an image of a different person on a tablet computer, etc.

The present invention was made in view of such problems. It is an object of the present invention to provide a photograph shooting device that can detect illegal photo shooting and realize electronic applications.

Means for Solving Problems

To solve the above problems, a first aspect of the present invention is a photograph shooting device having a camera that shoots an image of an applicant as a subject. The photograph shooting device includes action instruction means configured to give action instructions to the subject, living-body determination means configured to determine whether the subject is a living body or not by shooting with the camera a moving image of movement of the subject responding to the action instructions and analyzing the shot moving image, application image shooting means configured to shoot with the camera an application image, which is a photograph that is to be used for an application procedure, same-person determination means configured to compare an image cut out from the moving image with the application image and to determine whether shot persons are the same or not, and output means configured to output the application image to a predetermined server if it is determined by the living-body determination means that the subject is a living body and it is determined by the same-person determination means that the images are of the same person.

According to the first aspect of the present invention, the photograph shooting device gives the action instructions to the subject, shoots the moving image of the movement of the subject responding to the action instructions, and analyzes the shot moving image to determine whether the subject is a living body or not. Also, the photograph shooting device shoots the application image that is a photograph to be used for the application procedure. The photograph shooting device compares the image cut out from the moving image with the application image, and determines whether the shot persons are the same or not. If it is determined that the subject is a living body and the subject responding to the action instructions and the subject in the application images are the same, the photograph shooting device outputs the application image to the predetermined server.

In this way, it is possible to check whether the subject of the photograph shooting device is a living body or not and whether the person in the living-body determination is the same person in the application image. Thus, illegal acts in electronic applications can be prevented, thereby realizing electronic applications for administrative procedures and the like. For example, it is possible to detect illegal photo shooting of impersonation by covering a face with a portrait image shown on paper or on a tablet computer, or with a skin film, a mask, etc. Also, by using the image shot by the photograph shooting device, instead of a home PC or a smartphone, for the application, the strictly appropriate application image can be provided.

In the first aspect of the present invention, the action instruction means shows, in a random order, a plurality of prestored action instructions. In this way, it is possible to prevent illegal acts such as playing a prerecorded moving image showing actions that correspond to the action instructions during a living-body determination process.

Also, the photograph shooting device may further include a scanner that reads surface information of an identity verification medium of the applicant, and the same-person determination means may determine whether the person is the same or not by comparing a portrait image contained in the surface information read by the scanner with the image cut out from the moving image and with the application image. This enables to check whether the person of the portrait image on the identity verification medium is the same person in the living-body determination and the person in the application image, and thus stricter identity verification is possible.

It is preferable that the photograph shooting device further includes notification means that notifies an error when it is determined by the living-body determination means that the subject is not a living body or it is determined by the same-person determination means that the persons are not the same.

It is preferable that the application image shooting means is configured to shoot wide-angle photography and the photograph shooting device further includes object detection means configured to detect from the shot image an object covering a face of the subject. In this way, the object covering the face of the subject can be detected, and thus illegal photography can be detected.

Also, it is preferable that the photograph shooting device further includes image determination means configured to determine whether the application image is an appropriate photograph to be used for the application procedure. In this way, the application image that is appropriate for the application procedure can be provided with certainty.

The photograph shooting device may further include eyesight test means configured to display an eyesight test screen and to carry out an eyesight test for the applicant. In this way, the photograph shooting device can be used for applications that require an eyesight test, such as a driver's license application.

It is preferable that the photograph shooting device further includes a document scanner that reads a document that is used for the application procedure, and the output means is configured to output, together with the application image, document data read by the document scanner to the predetermined server. In this way, a set of the result of the identity verification and the image and the document required for the application can be prepared for the electronic application.

It is preferable that the photograph shooting device further includes setting means configured to set password information necessary for the application procedure, and the output means is configured to output the password information set by the setting means together with the application image to the predetermined server. In this way, the password information that is to be set at the time of application, such as a PIN for a driver's license, can be entered through the photograph shooting device.

It is preferable that the photograph shooting device includes reading means that reads information of the identity verification medium of the applicant, registered-information acquiring means configured to acquire registered information registered by the applicant on the server in advance, and identity verification means configured to verify identity of the applicant by comparing the registered information acquired by the registered-information acquiring means with the information read by the reading means. In this way, the application through the photograph shooting device can be efficiently carried out using the information registered by the applicant in advance using his/her home PC or the like. At this time, identity verification is performed, using an identity verification medium, on the person registered the information and on the applicant so as to prevent illegal applications.

A second aspect of the present invention is a control method of a photograph shooting device having a camera that shoots an image of an applicant as a subject. The method includes an action instruction step of giving action instructions to the subject, a living-body determination step of determining whether the subject is a living body or not by shooting with the camera a moving image of movement of the subject responding to the action instructions and analyzing the shot moving image, an application image shooting step of shooting with the camera an application image, which is a photograph that is to be used for an application procedure, a same-person determination step of comparing an image cut out from the moving image with the application image and determining whether the person is the same or not, and an output step of outputting the application image to a predetermined server if it is determined in the living-body determination step that the subject is a living body and it is determined in the same-person determination step that the person is the same.

According to the second aspect of the present invention, it is possible to check whether the subject of the photograph shooting device is a living body or not and to check whether the person in the living body determination is same as the person in the application image or not. Thus, illegal acts in electronic applications can be prevented, thereby realizing electronic applications for administrative procedures and the like. For example, it is possible to detect illegal photo shooting of impersonation by covering a face with a portrait image shown on paper or on a tablet computer, or with a skin film, a mask, etc. Also, by using the image shot by the photograph shooting device, instead of a home PC or a smartphone, for the application, the appropriate application image can be provided.

A third aspect of the present invention is a program that causes a control device of a photograph shooting device, which has a camera that shoots an image of an applicant as a subject, to function as action instruction means configured to give action instructions to the subject, living-body determination means configured to determine whether the subject is a living body or not by shooting with the camera a moving image of movement of the subject responding to the action instructions and analyzing the shot moving image, application image shooting means configured to shoot with the camera an application image, which is a photograph that is to be used for an application procedure, same-person determination means configured to compare an image cut out from the moving image with the application image and to determine whether shot persons are the same or not, and output means configured to output the application image to a predetermined server if it is determined by the living-body determination means that the subject is a living body and it is determined by the same-person determination means that the images are of the same person.

The third aspect of the present invention allows a computer to function as a control device of the photograph shooting device according to the first aspect of the present invention.

Effects of the Invention

The present invention allows a photograph shooting device to detect illegal photo shooting, thereby realizing electronic applications.

DESCRIPTION OF SOME EMBODIMENTS

Hereinafter, preferable embodiments of the present invention will be described in detail with reference to the accompanying drawings. A photograph shooting device1according to the present invention includes a stationary type photograph shooting device for taking identification photos that is installed at a storefront, at a station, on a street corner, or the like, as well as a stationary type photograph shooting device for taking photos for purposes other than identification photos.

FIG.1is a view showing an external appearance of a configuration of the photograph shooting device1, andFIG.2is a view showing a hardware configuration of the photograph shooting device1. As shown inFIG.1, the photograph shooting device1is a box-shaped device or the like having a housing, which is a photo studio, into which a person can enter to be photographed and operate the device. The form of the photograph shooting device1is not limited to the box shape, and may be an open type or a stand type, etc. The photograph shooting device1provides an ID photo menu and an image print menu: the ID photo menu has a function of taking an image of a person inside the photo studio as a subject, creating an identification photo from the image, and printing out the identification photo according to its intended use; and the image print menu has a function of printing out an image received from a user's terminal6such as a smartphone, or downloading and printing out an image that the user has uploaded from the terminal6to a server5in advance. Also, in the present invention, the photograph shooting device1provides an application menu for application procedures for driver's license, individual number cards, resident cards, special permanent resident cards, passports, and so on. The user can select and use any of the ID photo menu, the image print menu, the application menu, and the other menus.

As illustrated inFIG.1, the photograph shooting device1is in a box shape having an opening on a side, being separated into an exterior and an interior (the photo studio) by a curtain28. A front panel inside the photo studio is provided with an operation panel21, a coin mechanism22, a camera23, a lighting unit24, a sound output unit27, a subject sensor29, a reading device31for reading an identity verification medium (hereinafter, referred to as an ID card), a document scanner34, a code reader35, and so on. Also, a control device10, a printer25, and the like are accommodated inside the housing of the photograph shooting device1, and printed matters printed by the printer25are discharged from a print outlet26that is provided on an outer-wall panel of the photograph shooting device1.

FIG.2is a block diagram showing a hardware configuration of the photograph shooting device1. As shown inFIG.2, the control device10of the photograph shooting device1is connected via a bus to the lighting unit24, the camera23, the subject sensor29, the sound output unit27, the operation panel21, the coin mechanism22, the printer25, the reading device31, the document scanner32, the code reader35, and so on, and the control device10controls each unit.

The control device10includes a control unit (CPU: central processing unit)101, an I/O (input/output) control102, an I/F (interface)103, a ROM (read only memory)104, a RAM (random access memory)105, a storage unit106, a communication unit107, and so on. The control device10is connected to the operation panel21and the sound output unit27via the I/O control102. Also, the control device10is connected to the coin mechanism22, the printer25, the reading device31, the document scanner34, and the code reader35via the I/F103.

The control unit101is a CPU that centrally controls each unit of the photograph shooting device1. The control unit101reads out a program stored in the storage unit106, the ROM104, or the like to a work memory region on the RAM105and executes the program. The ROM104is a non-volatile memory that permanently holds programs such as a boot program and programs such as BIOS, data, and the like. The RAM105is a volatile memory that temporary holds the programs and data loaded from the storage unit106and the ROM104etc., and also includes the work memory region to be used for various processes executed by the control unit101.

The storage unit106is any of a hard disk drive, a solid state drive, a flash memory, and the like, and stores programs to be executed by the control unit101for processes described below, data required for the execution of the programs, an OS, and the like. The programs and the data are read out and executed by the control unit101as necessary.

The I/O control102is an interface for input and output of various information from and to the operation panel21, the sound output unit27, and so on.

The I/F103is an interface for input and output of various information from and to the coin mechanism22, the printer25, the reading device31, the document scanner34, and the code reader35.

The communication unit107includes a communication control device that controls communication of the photograph shooting device1and a communication interface.FIG.3is a view illustrating communication connection between the photograph shooting device1and the server5. The photograph shooting device1is provided with a wireless LAN such as Wi-Fi (a registered trademark) or a short-range wireless communication system such as Bluetooth (a registered trademark), and has a communicational connection to the server5via a wide-area network7such as the Internet.

The server5is a server that manages the photograph shooting device1. When a user enters, from the user's terminal6into the server5using a web browser or an application, information that is necessary for identity verification or information required for an application, the server5issues a predetermined code and registers the entered information making a link to the code. The server5includes a storage region that stores various data (photo images and other data) that are uploaded from the photograph shooting device1.

The user can download data of a photo image shot by the photograph shooting device1from the server5via the network7to the terminal6such as a PC or a smartphone. Also, an image that is uploaded from the user's terminal6via the network7to the server5can be downloaded to the photograph shooting device1and printed out (the image print menu).

Going back to descriptions ofFIG.1andFIG.2, the lighting unit24is a surface light source that is located in front, at the back, or the like of the subject. The lighting unit24is connected to a light source (stroboscope)241of which flash timing and an amount of luminescence are controlled according to control signals input by the control unit101.

The camera23is a digital camera including a photographing element such as an optical lens, a CCD (charge coupled device), and a CMOS (complementary metal-oxide semiconductor) with an A/D (analog/digital) conversion unit and the like. The camera23shoots a moving image or a still image according to the control signals input by the control unit101. The camera23inputs the shot image data into the control unit101. The control unit101then outputs and displays the image data input by the camera23to a display of the operation panel21.

The operation panel21is operation means of the photograph shooting device1and has a display for displaying an operation screen to show operation procedures and a preview screen etc., as well as a touch panel for the user to enter operation instructions. The operation panel21executes display processes according to display signals input by the control unit101. The operation panel21also sends control signals to the control unit101according to touch operations on the touch panel.

The subject sensor29is a sensor that detects a person inside the photo studio. The subject sensor29outputs a detection signal to the control unit101when the person approaches a front panel of the photograph shooting device1. For the subject sensor29, an optical sensor including a light transmitter and a photoelectric receiver or the like, is used.

The sound output unit27acquires predetermined voice-sound data (operation guidance voice etc.) from the ROM104or the storage unit106according to the control signals input by the control unit101, and outputs the sound through a speaker.

The coin mechanism22is a cash register provided with a money depositing slot for coins and bills, a return slot for change, and so on. The coin mechanism22executes a payment process for charges of the photograph shooting device1. The coin mechanism22outputs information related to the money received through the money depositing slot to the control unit101, and discharges coins and/or bills for change to the return slot according to the control signals input by the control unit101. Note that, instead of the coin mechanism22, other payment processing units configured for IC card payments or smartphone payments may also be used.

The printer25prints out onto predetermined paper the image data of a photograph shot according to the control signals input by the control unit101. The printer25is a high-resolution color printer and may be of any type, including sublimation type and melting type.

The reading device31is a device for reading information of the ID card. More specifically, the reading device31includes a surface image scanner32for scanning and reading the information on a surface (the front and back faces) of the ID card, and an IC chip reader33(a contactless/contact IC reader etc.) for reading data recorded on an IC chip on the ID card. The surface image scanner32may be a common image scanner with a white-light light source, or may be a UV/IR scanner including an ultraviolet-ray (UV) light source or an infrared-ray (IR) light source. Also, the IC chip on the ID card has a free memory region that is not protected by password information, and a locked memory region that is protected by password information. The reading device31can read data in the free memory region and the locked memory region. The reading device31inputs into the control unit101the image data on the surface of the ID card (hereinafter, refereed to as surface information) read by the surface scanner32. The reading device31also inputs into the control unit101the data read by the IC chip reader33.

The document scanner34is a scanner for optically reading a sheet of paper of approximately A4 size (210 mm by 297 mm). The document scanner34reads a document to be used for the application procedure and inputs the read document data into the control unit101.

The code reader35is a device for reading a two-dimensional code such as a QR code (a registered trademark) and a bar code, and inputs the read code information into the control unit101. The code reader35is used, for example, when reading a QR code (an authentication code) that is to be used in an identity verification process, which will be described below.

Next, a functional configuration of the control unit101of the photograph shooting device1will be described.FIG.4is a block diagram showing the functional configuration of the control unit101of the photograph shooting device1in an application mode. As shown in the drawing, the control unit101includes, as means for realizing the present invention, an identity verification unit401, an authenticity determination unit402, an action instruction unit403, a living-body determination unit404, an application image shooting unit405, an object detection unit406, a same-person determination unit A407, a same-person determination unit B408, an image determination unit409, an eyesight test unit411, a password information input unit412, a document scanning unit413, a payment processing unit414, and a data output unit415.

The identity verification unit401executes the identity verification process using the authentication code. The authentication code is identification information for verification given by the server5to an applicant who is about to proceed to an application procedure using the photograph shooting device1, and the authentication code includes numerical information and character information. The authentication code may be converted into a QR code or a bar code and notified to the applicant. When the applicant accesses the server5from the applicant's own terminal6and enters predetermined information (personal information such as name and date of birth, application detail (which application), and other information necessary for the application, etc.), the server5issues and sends the authentication code to the terminal6. The server5stores the information entered from the terminal6as registered information being linked to the authentication code. When using the photograph shooting device1, the applicant enters the authentication code, or shows the above-mentioned QR code (the authentication code) on the terminal6, which is then held up to the code reader35. Then, the photograph shooting device1acquires a value of the authentication code. The photograph shooting device1accesses the server5, acquires the registered information stored being linked to the authentication code from the server5, and stores the information in a predetermined folder (the storage unit106). Also, the applicant inserts his/her ID card into the photograph shooting device1so that the reading device31reads the information of the ID card. The identity verification unit401compares the registered information acquired from the server5with the information read from the ID card by the reading device31, and determines whether predetermined information such as the name and date of birth coincide one another to verify the identity of the applicant.

The ID card is an identity verification medium, such as a driver's license, an individual number card, a resident card, and a special permanent resident card, that is used for verifying identity at administrations, financial institutions, mobile phone carriers, etc. In a case of a driver's license, the person's name, date of birth, address, date of issuance, expiration date, and the name of the issuer of the ID card, etc. are printed on the surface of the driver's license. Also, the IC chip on the ID card stores a part or all of such information. A memory region of the IC chip includes a locked memory region that is protected by password information such as a PIN, and a free memory region that is not protected by the password information.

The authenticity determination unit402determines authenticity of the ID card inserted into the photograph shooting device1. The authenticity determination unit402determines whether the ID card is forged or altered or not based on the information read by the reading device31. In the present embodiment, the reading device31includes the surface scanner32and the IC chip reader33, and thus it is possible to scan the surface information of the ID card or to read the data recorded on a recording medium (the IC chip) in the ID card. The authenticity determination unit402determines the authenticity of the ID card based on both or either of the surface information read, or the data recorded on the IC chip. The authenticity determination unit402may determine the authenticity of the ID card based on the data stored in the locked memory region of the ID card if it is possible to read the data in the locked memory region. A specific method for determining the authenticity will be described below. The authenticity determination unit402stores a result of the determination in a predetermined folder (the storage unit106).

The action instruction unit403instructs actions to the applicant for the living-body determination unit404, which will be described below, to determine whether the applicant is a living body or not. The action instruction unit403gives instructions of actions, for example, by voice sound from the sound output27, or by displaying written instructions or instruction moving images or animations on the operation panel21. It is preferable that the action instruction unit403shows, in a random order, a plurality of instructions selected from a plurality of action instructions prestored in the storage unit106. In this way, it is possible to prevent illegal acts such as playing a prerecorded moving image showing actions that correspond to the action instructions during a living-body determination process. Examples of actions to be instructed will be shown below from (1) to (5).(1) “Blink slowly”(2) “Slowly turn your head to the right (left)”, and then “Come back slowly to the center and look forward”(3) “Slowly look down (up)”, and then “Come back slowly and look forward”(4) “Tilt your head to the right (left)”, and then “Straighten your head back”(5) “Open your mouth wide”, and then “Shut your mouth”

The living-body determination unit404shoots a moving image, with the camera23, of the applicant who moves in response to the action instructions from the action instruction unit403, and analyzes the shot moving image to determine whether the applicant is a living body or not. The living-body determination unit404performs the living-body determination by checking if there are actions characteristic to a living body, such as swaying of eyes or opening or closing of eyes (blinking), while detecting the applicant's response by using face recognition technology. The living-body determination unit404stores a result of the determination into a predetermined folder (the storage unit106).

The living-body determination unit404is a function pertaining to a main point of the present invention.

The application image shooting unit405shoots with the camera23an application image that is to be used for the application procedure. When shooting, the application image shooting unit405displays operation instructions on the operation panel21and outputs guidance voice sound, and then shoots a portrait image (a still image) of the applicant at a predetermined timing. The application image shooting unit405creates the application image by adding various processing, such as size adjustment, positioning, trimming, and background color setting, to the shot portrait image so as to make the image suit the usage. The application image shooting unit405stores the shot application image into a predetermined folder (the storage unit106).

The object detection unit406examines the application image shot in the application image shooting unit405, and detects if there is an object covering a face of the subject (the applicant). In this way, it is possible to detect illegal acts of impersonation, such as holding up someone else's portrait image (printed out paper or an output matter) in front of the face or holding up a tablet computer or the like displaying someone else's portrait image in front of the face. An example of a method for object detection by the object detection unit406is by an image processing for edge detection, in which the background color is subtracted from the wide-angle shot image before being trimmed to detect any unnatural corners, frames, or edges. The object detection unit406stores a result of the detection into a predetermined folder (the storage unit106).

The same-person determination unit A407compares the applicant's front face image that is cutout from the moving image shot in the living-body determination process by the living-body determination unit404with the application image shot by the application image shooting unit405to determine whether the photographed person is the same person in the moving image or not. This can detect an illegal act of replacing the subject within a moment between the living-body determination process and the shooting of the application image. The same-person determination unit A407stores a result of the determination in a predetermined folder (the storage unit106).

The same-person determination unit A407is a function pertaining to a main point of the present invention.

The same-person determination unit B408compares an image of a person read from the surface of the ID card by the reading device31or, in a case in which the image of the person is stored in the IC chip, the image of the person contained in the information read from the IC chip, with the application image shot by the application image shooting unit405to check whether the person in such the images is the same.

Furthermore, the same-person determination unit B408also checks the applicant's front face image that is cutout from the moving image shot in the living-body determination process by the living-body determination unit404to determine whether the person in all the images is the same. In this way, it can be verified that the image of the person on the surface of or contained in the IC chip on the ID card that is used for identification is an image of the applicant. The same-person determination unit B408stores a result of the determination in a predetermined folder (the storage unit106).

The image determination unit409determines whether the application image shot by the application image shooting unit405is photographed appropriately for the application procedure or not. This is a function for preventing any defects of the photograph. For example, specifications and precautions for passport application photographs are regulated in the international standards, and thus the shot image is determined if the photograph conforms to the standards. If a result of the determination shows that the shot application image is inappropriate, the image determination unit409attempts to shoot a photograph again, or gives some advice about how to behave while being photographed. The image determination unit409stores the result of the determination in a predetermined folder (the storage unit106).

The eyesight test unit411displays an eyesight test screen71on the operation panel21and carries out an eyesight test for the applicant. The eyesight test unit411displays a result of the measurement and stores the result in a predetermined folder (the storage unit106).

The password information input unit412displays an input screen for setting password information necessary for the application procedure (a PIN, etc.). The password information entered by the applicant through the input screen, together with the application image, are stored in a predetermined folder (the storage unit106), and then output to the server5by the data output unit415.

The document scanner unit413reads a document that is to be used for the application procedure by using the document scanner34, and stores the read data in a predetermined folder (the storage unit106). The read data of the document together with the application image and the like are then output to the server5.

The payment processing unit414calculates a charge for using the photograph shooting device1and receives money from the coin mechanism22. The payment processing unit414also calculates change and discharges the change from the coin mechanism22. In a case of an IC card payment or a smartphone payment, the payment processing unit414cooperates with a payment server for execution of the payment.

The data output unit415outputs the application image, the set password information, the scanned document data, and the like to the server5if the identity verification unit401determines that the image of the person of the ID card matches the applicant, the authenticity determination unit402determines that the ID card is authentic, the living-body determination unit404determines that the applicant (the subject) is a living body, the same-person determination unit A407and the same-person determination unit B408determine that the shot moving image, the still image, and the image on the ID card are of the same person, the object detection unit406determines that there is no object covering the face, and the image determination unit409determines that the application image is appropriate. In this way, the information necessary for the application procedure is sent from the photograph shooting device1to the server5. The server5then sends such the information to a computer (a server) of an organization that processes the electronic application.

The data output unit415also outputs, together with the application image and the like, the determination result by the identity verification unit401, the information read from the ID card, the determination result by the authenticity determination unit402, the determination result by the living-body determination unit404, the detection result by the object detection unit406, the determination results by the same-person determination units A407and B408, the determination result of the image determination unit409, and the measurement result by the eyesight test unit411. In this way, it is possible to send to the server5information showing that identity is verified by the ID card, there is no forgery or alteration of the ID card, the applicant is a living body, the person on the ID card matches the person photographed on the application image and the person being determined in the living-body determination, the application image is appropriately photographed, etc.

Next, flow of the processes executed by the photograph shooting device1according to the present embodiment will be described.FIG.5is a flowchart showing a flow of a menu selection process in the photograph shooting device1, andFIG.6is a flowchart showing a flow of an application menu process.

When the subject sensor29detects an entry of a user (the subject/the applicant) into the photo studio (a step S101: YES), the control unit101of the photograph shooting device1displays a top screen on the operation panel21(a step S102). The top screen is provided with buttons for selecting various services, such as an identification photographing service, an application service, and an image printing service.

When the user operates any one of the buttons and selects a service (a menu) to be executed (a step S103: YES), the control unit101transfers the process to the selected service (menu) (a step S104).

If the user already has the authentication code (a QR code or the like) (the step S103: NO), the user may hold up the QR code to the code reader35(a step S105) instead of operating the selection of the service (the menu) on the top screen.

The authentication code (a QR code, a bar code, numerical or character information, etc.) is an authentication code issued by the server5, and is issued if the user has accessed the server5from the user's own terminal6in advance, selecting the menu to be used on the photograph shooting device1, entering information required for identity verification, and entering information required for processing, and such the information has been registered on the server5.

If the user holds up the QR code to the code reader35(the step S105: YES), the code reader35reads the QR code (a step S106). If reading of the QR code fails (a step S107: YES), the control unit101displays an error notification screen (a step S108) and returns the process to the top screen (a step S102). In the process of the step S106, instead of reading the QR code, an input of the authentication code may be accepted. When reading/entering the authentication code is completed (the step S107: NO), the control unit101accesses the server5, acquires from the server5the registered information stored being linked to the read QR code (the authentication code), and transfers the process to a process menu that is designated by the registered information (a step S109).

In the present embodiment, the application menu will be described. When the application menu is selected in the process shown inFIG.5, the control unit101executes an application menu process shown inFIG.6.

FIG.6is a flowchart showing a flow of the application menu process. When the process is transferred to the application menu, firstly, the control unit101displays an application contents selection screen51on the operation panel21and waits for operations.FIG.7is a view showing an example of the application contents selection screen51. As shown inFIG.7, application contents that can be selected are shown on the application contents selection screen51. In the example shown inFIG.7, buttons for “Passport Application”, “Driver's License Application”, and “Individual Number Card Application” are displayed. The application contents are not limited to the above and may include applications for other identity verifying documents (resident cards, special permanent resident cards, etc.). Also, the application may be either a renewal application or a new application. However, only those of which electronic applications are officially permitted may be selectable.

When the user selects any one of the application contents on the application contents selection screen51, the control unit101acquires a value of the authentication code (a step S201). If reading the QR code/entering the authentication code has been completed in the step S106inFIG.5, the control unit101acquires the read or entered code value. If the QR code has not been read or the authentication code has not been entered yet, the control unit101displays a code reader instruction screen52shown inFIG.8or the code input screen53shown inFIG.9on the operation panel21, and reads the QR code or accepts inputs of the value of the authentication code. When the code value is read or entered, the control unit101displays a code confirmation screen54shown inFIG.10. If a “Yes” button54bis selected on the code confirmation screen54, the process is transferred to a step S202inFIG.6. Note that if any of “Back” buttons52a,53aand a cancel button54ais operated in any of the screens shown inFIG.8toFIG.10, the process returns to the application contents selection screen51.

Next, the photograph shooting device1reads the information on the user's ID card. More specifically, the photograph shooting device1scans a surface of the ID card by using the surface scanner32of the reading device31(a step S202), and also reads the data recorded on the IC chip on the ID card by using the IC chip reader33(a step S204). Note that the scanning in the step S202may be common image scanning using white light, or UV/IR image scanning using ultraviolet rays (UV) or infrared rays (IR). Also, in a case of a driver's license, for example, in which the IC chip has a memory region that is protected by password information (a PIN or the like) (the locked memory region), an input of the password information is accepted (a step S203) and the data stored in the free memory region as well as in the locked memory region are read (a step S204). If the IC chip has no memory region that is protected by password information (the locked memory region) or the user enters no password information, then only the data stored in the free memory region is read in the step S204.

If it is unable to read the information from the IC chip (a step S205: YES), there is a possibility that the ID card is forged or altered. Thus, an error is notified (a step S206), and then the process returns to the application contents selection screen51. If the information is read from the IC chip (the step S205: NO), the process proceeds to a step S207.

Note that, in the step S202, the control unit101may display on the operation panel21an ID card insertion instruction screen55shown inFIG.11. Also, in the step S203, the control unit101displays on the operation panel21a password information input screen56shown inFIG.12.

After reading the information on the ID card through the processes from the step S202to the step S204(the step S205: NO), the control unit101executes the identity verification process using the code value acquired in the step S201(the step S207).

FIG.13is a flowchart showing a flow of the identity verification process of the step S207. As shown inFIG.13, the control unit101firstly downloads from the server5the registered information linked to the code value that has been acquired in the step S201(a step S301). If downloading of the registered information has already been completed by the code reading process in the step S106shown inFIG.5and the registered information is stored in the RAM105or the storage unit106of the photograph shooting device1, the control unit101acquires the registered information from the RAM105or the storage unit106.

The control unit101compares the acquired registered information with the information read from the ID card (a step S302), and determines whether there is inconsistency in the contents of the information or not (a step S303). For example, the control unit101executes an OCR (optical character recognition) process on the surface information read from the ID card to acquire the character and numerical information such as name and date of birth. If the name and the date of birth written on the surface of the card match the information of the name and date of birth contained in the registered information, the control unit determines that there is no inconsistency (the step S303: YES) and the verification is a success. If there is inconsistency between any of the name and the date of birth written on the surface of the card and the information of the name and date of birth contained in the registered information, then the control unit determines that there is inconsistency (the step S303: NO) and the verification is a failure. Items that are to be compared in the step S302are not limited to the name and date of birth, and the comparison may be based on the date of issuance and expiration date of the ID card, information stored in the IC chip, and the like. Also, the comparison may be based on the portrait image on the ID card. In such cases, the equivalent information (the date of issuance, expiration date, portrait image, etc.) must be registered in the registered information in advance.

The control unit101acquires a result of the identity verification process, and if the verification is a failure (the step S208inFIG.6: NO), the control unit101displays an error notification screen57as shown inFIG.14(a step S209) and then the process returns to the application contents selection screen51. On the error notification screen57, a description of the error (e.g., “The registered information does not match the information on the ID card”, etc.) is shown. If the verification is a success (the step S208: YES), the control unit101stores the result of the identity verification process (the identity verification process “OK”) into the folder in which the registered information is registered, and proceeds the process to a step S210.

In the step S210, the control unit101executes an authenticity determination process on the ID card (the step S210).

FIG.15is a flowchart showing a flow of the authenticity determination process.

Although there are various methods of the authenticity determination process for the ID card, an example thereof in which the authenticity is determined by detecting obvious inconsistency based on the surface information read by the reading device31and on the IC chip information will be described here.

The control unit101executes an OCR process on the surface information read from the ID card (a step S401), and determines if there is inconsistency in the surface information. The determination of inconsistency of the surface information is disclosed in Patent Document 1, for example. More specifically, if the ID card is a driver's license, for example, the control unit101scans and performs OCR to acquire the name, date of birth, address, date of issuance and expiration date of the ID card, and name of the issuer written on the ID card surface, and determines if there is inconsistency between the contents of certain character information and the contents of the other character information, or if there is inconsistency between the contents of certain character information and attributes of the said character information on the ID card. For example, if it is prescribed by law that the expiration date of the ID card should fall on a date that is exactly one month later than the date of birth, a forgery such as adding a number “2” by using a pen etc. to the date of birth “Jun. 1, 1975” so as to be altered to “Jun. 21, 1975”, for example, is to be determined that there is inconsistency between the date of birth and the expiration date (the day and month of the date of expiration do not fall on the day and month of the following month of the date of birth). Also, if the date of birth written is not possible in any way, e.g., year of Showa 70, or month 13, etc., then it is determined that there is consistency in the content.

If no inconsistency is detected from the surface information, the process is determined as “OK” (a step S403: YES) and the control unit101compares the information stored in the IC chip on the ID card (date of expiration and issuance, for example) with the information acquired from the surface information (date of expiration and issuance) (a step S404). If a comparison result shows a match, it is determined as “OK” (a step S405: YES). Next, the control unit101compares the information stored in the locked memory region of the IC chip on the ID card (name, date of birth, address, etc.) with the information acquired from the surface information (name, date of birth, address, etc.) (a step S406). If a comparison result shows a match, it is determined as “OK” (a step S407: YES). Next, the control unit101determines whether there is an illegality in the ID card or not from a UV/IR scanning result of the ID card surface (a step S408). Sometimes, UV/IR scanning can detect a trace of replacement or the like of the surface of the card. If no illegality is detected from the result of the UV/IR scanning, the ID card is determined as “authentic” (a step S409). If any of the determinations and comparisons in the steps S402, S404, and S406results in failure (“NO” in any of the steps S403, S405, and S407), or if illegality is detected by the UV/IR scanning (the step S408: YES), the control unit101determines that the ID card is “fake” (a step S410).

If the ID card has no locked memory region in the IC chip or it is unable to read the information stored in the locked memory region, the control unit101may skip the processes from the step S406to the step S407. Also, if the reading device31has no UV/IR scanning function, the process in the step S408may be skipped.

Next, the control unit101acquires the portrait image from the surface of the ID card scanned by the reading device31or from the IC chip (a step S411), and stores the acquired portrait image and the result of the authenticity determination from the steps S409and S410(authentic or fake) in a predetermined folder in the storage unit106(a step S412). This completes the authenticity determination process of the ID card.

If the ID card is determined as “fake” as a result of the authenticity determination inFIG.15(the step S410and the step S211inFIG.6: NO), the control unit101displays the error notification screen57as shown inFIG.14(the step S212), and then returns the process to the application contents selection screen51. On the error notification screen57, a description of the error (e.g., “Inconsistency is found in the items mentioned on the ID card”, or “Illegality is found on the ID card”, etc.) is shown. If the ID card is determined as “authentic” (the step S211: YES), the process is transferred to a process in a step S213.

In the step S213, the control unit101executes the living-body determination process to determine whether the subject (the applicant) is a living body or not (the step S213).

FIG.16is a flowchart showing a flow of the living-body determination process. In the living-body determination process, the control unit101shoots a moving image of the subject (the applicant) with the camera23(a step S501). The control unit101also sets a counter n to an initial value “1” (a step S502), and instructs nth action (a step S503).

In the step S503, the control unit101acquires action instruction data that has been prestored in the storage unit106and gives action instructions among a plurality of the action instructions for a predetermined number of times (N times) in a random order. This is to prevent illegal acts, such as playing a prerecorded moving image on a tablet terminal showing actions that correspond to the action instructions, which may happen if the order of the actions is always the same. In the step S503, the control unit101displays on the operation panel21an action instruction screen58as shown inFIG.17to show action instruction contents by means of animations and letters in an instruction description presentation area58aon the screen, or outputs voice sound of the action instructions.

The subject responding to the action instructions is recorded in a moving image, which is simultaneously shown on a moving image displaying area58bon the action instruction screen58. The control unit101examines the response (actions) recorded in the moving image, analyzes the image to examine if the actions correspond to the action instructions and to detect any actions characteristic to a living body (swaying of eyes, blinking, or the like), and determines whether the subject is a living body or not (a step S504). Also, at the time of detecting the response, the control unit101cuts out a front face image from the moving image and saves the image as a still image in the RAM105or in a predetermined folder in the storage unit106(a step S505). The saved front face image is then compared with front face images acquired when the person responds to the next and following action instructions (a step S506).

The control unit101compares the value of the counter n with the predetermined max value N. If n<N (a step S507: YES), the control unit101increments the value of the counter n (n+1) (a step S508), returns the process to the step S503, and instructs the next action. The control unit101repeats the processes in the steps S503to S506for N times. In this way, results of the living-body determinations for N times action instructions and a result of front face image comparison for each time can be obtained. When the N times of processes are completed (the step S507: NO), the control unit101outputs a result of the determination (a step S509).

If, as a result of the living-body determination process, no living-body characteristics are detected in the subject or the front face images acquired from the action instructions do not match one another, then the control unit101determines the process as “NG”. In such the case (a step S214inFIG.6: NO), the process proceeds to a step S215to display the error notification screen57as shown inFIG.14(the step S215), and then returns to the application contents selection screen51. The error notification screen57in the step S215shows a description of the error (e.g., “There are some problems in the actions” etc.).

If, as a result of the living-body determination process, living-body characteristics are detected in the subject and the front face images acquired from the action instructions match one another, then the control unit101determines the process as “OK” (the step S214inFIG.6: YES), and the process proceeds to a step S216shown inFIG.18.

FIG.18is a flowchart showing a flow continued fromFIG.6(the application menu process). The control unit101executes a photographing process of the application image in the step S216. The application image is a photograph that is to be submitted at a time of application of a passport, a driver's license, an individual number card, etc. In the photographing process of the application image, the control unit101displays an application image photographing screen61as shown inFIG.19and operation screens62and63as shown inFIG.20to provide instructions for photographing and positioning of the face or the like. For example, before starting photographing, the control unit101displays the application image photographing screen61as shown inFIG.19on the display panel21and outputs a guidance sound such as “Please look to the front glass” from the sound output unit27. After that, the control unit101shoots a still image of the subject (the applicant) with the camera23and stores the image in the RAM105or in a predetermined folder in the storage unit106. The image is shot at a wide angle with the face in the center, and the size and positioning are adjusted according to the photograph size in the following steps. The photographing may take place a plurality of times.

FIG.20Ais the operation screen62for adjusting a vertical position of the face, andFIG.20Bis the operation screen63for adjusting a horizontal position of the face. On the operation screen62inFIG.20A, a preview image62ais shown with guidelines62band62cfor showing a top position of the head and a tip position of a chin, respectively. Also, up-and-down buttons62dand62efor vertically moving the guidelines62band62c, respectively, are provided. When a set button62fis operated, the operation screen63shown inFIG.20Bis then displayed. On the operation screen63inFIG.20B, a preview image63ais shown with a guideline63bfor showing a center position of the face. Also, left-and-right button63cfor horizontally moving the guideline63bis provided. When a set button63dis operated, the horizontal center position of the face is decided. The control unit101then creates the application image having a size that suits the application content.

Next, the control unit101executes an object detection process (a step S217). The object detection process is a process for determining whether there is an object covering the face of the subject (the applicant).FIG.21is a flowchart showing a flow of the object detection process, andFIG.22(a)toFIG.22(d)are views illustrating illegal acts of face covering and the object detection process. There is a possibility of an illegal act in photographing, such as impersonating someone else by holding up someone else's portrait image shown on a tablet computer or on a printed matter in front of the face as shown inFIG.22(a)andFIG.22(b). To prevent such illegal acts, the photograph shooting device1executes the object detection process.

As shown in the flowchart inFIG.21, the control unit101acquires an image before trimming the application image that has been shot in the step S214(a step S601). The application image is shot at a wide angle as shown inFIG.22(d)at first and then trimmed as shown inFIG.22(c). The control unit101examines the image before trimming acquired in the step S601and detects from the image any unnatural frames or corners (a step S602). An example process in the step S602is an edge detection process, in which the background color is subtracted from the shot image and edges are detected in the subtracted image. If unnatural frames, corners, or edges are detected, it means that an object in front of the face is detected, and thus it is determined as “NG”. If unnatural frames, corners, or edges are not detected, it means that an object in front of the face is not detected, and thus it is determined as “OK”. The control unit101stores a result of the detection into the RAM105or in a predetermined folder in the storage unit106(a step S603).

Next, the control unit101executes a same-person determination process A (a step S218inFIG.18).FIG.23is a flowchart showing a flow of the same-person determination process A. In the same-person determination process A, the control unit101compares the front face image that has been cutout from the moving image shot in the living-body determination process in the step S211with the application image shot in the step S216, so as to determine whether the person being photographed is the same person or not (a step S701). The control unit101executes a face recognition process, and determines that it is “OK” if the person in the front face image matches the person in the application image. The control unit101determines that it is “NG” if the person in the front face image does not match the person in the application image. The control unit101stores a result of the determination into the RAM105or in a predetermined folder in the storage unit106(a step S702).

Next, the control unit101executes a same-person determination process B (a step S219inFIG.18).FIG.24is a flowchart showing a flow of the same-person determination process B. In the same-person determination process B, the control unit101compares the portrait image that is on the surface and contained in the IC chip on the ID card that has been acquired in the steps S202to S204(hereinafter, referred to as an ID card face image) with the application image shot in the step S214, so as to determine whether the person being photographed is the same person or not (a step S801). Furthermore, the control unit101compares the front face image that has been cutout from the moving image shot in the living-body determination process in the step S211with the ID card face image and the application image, so as to determine whether the person being photographed is the same person or not (a step S802). The control unit101executes a face recognition process, and determines that it is “OK” if the person in the ID card face image matches the person in the application image as well as the person in the front face image. It is determined as “NG” if any of the persons in the images do not match one another. The control unit101stores a result of the determination into the RAM105or in a predetermined folder in the storage unit106(a step S803).

After completing the processes in the steps S217to S219(the object detection process, the same-person determination process A, and the same-person determination process B), the control unit101acquires the results of the processes (“OK” or “NG”) stored in the RAM105or the predetermined folders in the storage unit106. If there is an “NG” in any of the processes (a step S220: NO), the control unit101displays the error notification screen57as shown inFIG.14(a step S221), and then returns the process to the application contents selection screen51. The error notification screen57in the step S221shows a description of the error (e.g., “There are some problems in the shot image” etc.).

If the results of the object detection process, the same-person determination process A, and the same-person determination process B are all “OK” (the step S220: YES), the process proceeds to a step S222.

The control unit101shows a preview of the application image shot and created in the step S216(the step S222), and executes an image determination process (a step S223). In the image determination process in the step S223, the control unit101determines whether the application image is an appropriate image for the application procedure. A method for determination is, for example, to set a plurality of items to be determined in advance in the storage unit106and to make a determination of “OK” or “NG” on each of the items. The items to be determined may include, for example, if the background includes any patterns or not, if the background color is appropriate or not, if the boundary between the person and the background is clear or not, if the pupils of the eyes are photographed as red or not, if the lighting is reflected by the glasses or not, if there is a shade on the face or not, if there is a shade on the background or not, if there are any blurs or out of focus in the photo or not, and if there are any reflections or uneven parts in the photo or not. The followings may be also detected: the head is tilted to the left or to the right, the person is facing to the side, the position is not at the center, the head is covered by a hat or a wide headband, accessories or the like cover any of the organs of the face or the head, the outline of the face is hidden, the hair hangs over the eyes, color contacts are worn, colored glasses, sunglasses or frames of the glasses hang over the eyes, and the like. If any of the items is determined as “NG” (a step S224: NO), the error notification screen57is displayed (a step S225), and then the process is returned to the living-body determination process (shooting the moving image) in the step S213(FIG.6). The error notification screen57in the step S225shows a description of the error (e.g., “There are some problems in the shot image” etc.) or descriptions of the items that have been determined as “NG”, giving some advice about how to behave while being photographed. If all the items to be determined in the image determination process are “OK” (the step S224: YES), the process proceeds to a step S226shown inFIG.25.

FIG.25is a flowchart showing a flow of a process continued fromFIG.18. The control unit101executes an eyesight test process (the step S226). In the step S226, the control unit101displays on the operation panel21an eyesight test screen71as shown inFIG.26(a). On the eyesight test screen71, an indicator71afor an eyesight test, such as a Landolt C, and operation buttons, such as an upward button71b, a downward button71c, a left button71d, and a right button71e, are shown to accept response operations from the applicant. For example, in the case ofFIG.26(a), there is a gap at the top of the Landolt C, so it is correct if the applicant operates the upward button71b. If a correct response at a predetermined standard value or above can be obtained (a step S227: YES), the control unit101displays a test result screen72as shown inFIG.26(b)(a step S231), and stores the test result in the RAM105or in a predetermined folder in the storage unit106. If the correct response at the predetermined standard value or above cannot be obtained (the step S227: NO), for the first test (a step S228: NO), an instruction for a re-test is shown on the screen (a step S229) and then the re-test is carried out (a step S226). If test results of the predetermined standard value or above cannot be obtained after repeating the test for several times (the step S228: YES), an error is notified (a step S230) and the process returns to the application contents selection screen51(the step S201inFIG.6).

If the eyesight test confirms that the person's eyesight is at the predetermined standard value or above (the step S227: YES), the control unit101then accepts an entry of a password information setting value (a step S232). The value to be entered here is a value for the password information (e.g., a PIN) for the ID card to be applied. The control unit101displays a password information setting screen73as shown inFIG.27on the operation panel21. When the applicant enters a value (the password information) on the password information setting screen73, the control unit101displays a confirmation screen for the entered contents (a step S233). If a response of a confirmation “OK” is obtained (a step S234: YES), the control unit101stores the entered value in a predetermined folder in the storage unit106or into the RAM105and precedes the process to a step S235. If the response of the confirmation “OK” is not obtained on the confirmation screen in the step S233(the step S234: NO), the control unit101returns the process to the step S232and accept a re-entry of the password information.

When the password information is entered, the control unit101next accepts scanning of an application document (the step S235). In the step S235, the control unit101shows a reading instruction screen74as shown inFIG.28(a). The reading instruction screen74displays an instruction message, such as “Scan your application document”, with illustration of a position of a document insertion slot of the document scanner34. When a document is set into the document scanner34, the control unit101starts a reading operation by the document scanner34. When the reading of the document is completed, the control unit101displays a checking screen75as shown inFIG.28(b), showing a result of the scan (a step S236). The checking screen75shows the content75athat has been read by the document scanner34, with an “OK” button75band a “Back” button75c. If the “OK” button75bis operated by the applicant (a step S237: YES), the control unit101stores the read content as document data into the RAM105or a predetermined folder in the storage unit106, and proceeds to a process in a step S239. If the “Back” button75cis operated (the step S237: NO), the control unit101provides an instruction for rescan by screen display or sound output (a step S238), returns the process to the step S235, and rescans the document.

When scanning of the application document is completed, the control unit101next displays an application confirmation screen76on the operation panel21(a step S239). An example shown inFIG.29is the application confirmation screen76for an individual number card. The application confirmation screen76shows a date of application76a, an application ID76b, an application image76c, a request button76dfor an electronic certification or a braille transcription, a change button76efor the electronic certification or the braille transcription, an application button76f, a cancel button76g, and the like.

The control unit101issues the date of application76aand the application ID76b. The application image76cis the application image photographed and created in the above-mentioned processes in the step S216. The request button76dfor the electronic certification or the braille transcription is in the information contained in the registered information that the applicant has registered in advance in the server5or in the scanned document data. If the change button76eis operated, the screen transitions to a setting screen for the electronic certification or the braille transcription to accept the change of the request.

When the applicant confirms the application content on the application confirmation screen76and operates the application button76f(a step S240: YES), the process proceeds to a payment process in a step S241. If the cancel button76gis operated (the step S240: NO), the process returns to the application contents selection screen51(the step S201).

In the payment process in the step S241, the control unit101calculates the charge and accepts deposition of money into the coin mechanism22. The control unit101also calculates the change and discharges the change from the coin mechanism22. In a case of an IC card payment by such as an IC card for transportation or a smartphone payment, the control unit101cooperates with a payment server for execution of the payment of the charge (the step S241).

When the payment process is completed, the control unit101links the information required for the application with the application ID76bto create data, and outputs the data to the server5(a step S242). The information required for the application is set in advance according to the application content, and, for example, for a driver's license application, the required information includes, in addition to the application image, attached information such as the eyesight test result, the password information set value, and the scanned document data. Also, the identity verification information acquired in the steps S202to S204(the information read from the ID card), the authenticity determination result of the ID card, and the like are output to the server5as the attached information together with the information required for the application. After that, the control unit101displays a termination screen77as shown inFIG.30and terminates the application menu process.

The contents of the application menu process are not limited to the above and can be appropriately changed according to the application contents. For example, while the application for driver's license includes the eyesight test and the password information setting, the eyesight test is not required for the application for the individual number card. Also, in the passport application, the eyesight test and the password information setting are unnecessary, and the items for determination in the image determination process are to be set so that the image suits the passport photograph.

As described above, the photograph shooting device1of the present invention gives the action instructions to the subject (the action instruction unit403), shoots with the camera23the moving image of the movement of the subject responding to the action instructions, and analyzes the shot moving image to determine whether the subject is a living body or not (the living-body determination unit404). Also, the photograph shooting device1shoots the application image that is a photograph to be used for the application procedure (the application image shooting unit405). The photograph shooting device1compares the image cut out from the moving image with the application image, and determines whether the shot persons are the same or not (the same-person determination unit A407). If it is determined that the subject is a living body and the subject responding to the action instructions and the subject in the application images are the same, the photograph shooting device1outputs the application image to the predetermined server5(the data output unit415). In this way, it is possible to check whether the subject of the photograph shooting device1is a living body or not and whether the person in the living-body determination is the same person in the application image or not. Thus, illegal acts in electronic applications can be prevented, thereby realizing electronic applications for administrative procedures and the like. For example, it is possible to detect illegal photo shooting of impersonation by covering a face with a portrait image shown on paper or on a tablet computer, or with a skin film, a mask, etc. Also, by using the image shot by the photograph shooting device1, instead of a home PC or a smartphone, for the application, the appropriate application image can be provided.

Although the embodiments of the present invention have been described referring to the attached drawings, the technical scope of the present invention is not limited to the embodiments described above. It is obvious that persons skilled in the art can think out various examples of changes or modifications within the scope of the technical idea disclosed in the claims, and it will be understood that they naturally belong to the technical scope of the present invention.

DESCRIPTION OF NOTATIONS