Patent Description:
When opening a bank account, creating a credit card, and the like, personal identification using an identification card is performed. Then, when opening an account and the like via the Internet, and the like, an image acquired by capturing an identification card by a camera instead of an original of the identification card may be used for personal identification.

In a case where personal identification is performed by using an image of an identification card, spoofing needs to be prevented. PTL <NUM> discloses a system for confirming that a personal identification document is a user's by comparing capturing data about a face photograph of the personal identification document with capturing data about the user.

Further, in the system in PTL <NUM>, while an instruction such as "capture a front surface of a personal identification document" and "capture a back surface of a personal identification document" is provided in a user terminal in order to acquire an image of a plurality of surfaces of the personal identification document (identification card), a video in which the personal identification document is captured is generated. Then, the video is transmitted to an authentication server.

Herein, a timing of the instruction described above is predetermined by a relative time from start time of capturing a video. The authentication server uses, as an image of the front surface and the back surface of the personal identification document, an image associated with a timing of each instruction (i.e., an image at a predetermined timing starting from a point in start time of capturing a video) from the received video.

The document <CIT> discloses methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for obtaining, in real-time from an image capture device, a video stream comprising images of a document by a computing device. The computing device provides, for display in an image preview window, the video stream overlaid with a graphical capture guide. In response to detecting a lighting artifact in at least one image of the video stream, the computing device modifies the graphical capture guide within the image preview window. The computing device captures one or more of the images of the document from the video stream.

The document <CIT> discloses a program causing an information processing apparatus including a display unit to execute a step of displaying a moving image of a shooting area output to the display unit by a shooting device, a step of displaying a shooting guide indicating a shooting method of the side surface of a personal identification document on the moving image of the shooting area according to information indicating a position and an inclination in the shooting area determined on the basis of a predetermined logic, and a step of causing the shooting device to shoot the moving image of the shooting area.

The document <CIT> discloses first, obtaining an identity certificate image of an object to be verified; then, verifying whether the identity certificate image comes from the same shooting scene as a first background image that is collected on site in advance and/or whether the identity certificate image comes from a physical certificate to obtain a first verification result; simultaneously verifying the authenticity of identity information in the identity certificate image to obtain a second verification result; and finally, determining an identity verification result of the object to be verified, based on the first verification result and the second verification result.

In the system in PTL <NUM>, whether an image of a personal identification document (for example, an image of a front surface or an image of a back surface) captured in a desired state is included in a video transmitted from a user terminal to an authentication server is determined in the authentication server. Thus, the image of the personal identification document captured in the desired state may not be included in the video received by the authentication server. In this case, an authentication error is transmitted to the user terminal, and a video needs to be captured again in the user terminal.

The present invention has been made in view of the problem described above, and one of objects of the present invention is to provide a technique for increasing a probability that an image of an identification card captured in a desired state is provided.

The invention is defined by a program causing a computer to execute the processing according to claim <NUM>, an image analysis apparatus according to claim <NUM> and a control method according to claim <NUM>.

The dependent claims define further advantageous details of the invention.

The present invention provides a technique for increasing a probability that an image of an identification card captured in a desired state is provided.

Hereinafter, example embodiments of the present invention will be described with reference to the drawings. Note that, in all of the drawings, a similar component has a similar reference sign, and description thereof will be appropriately omitted. Further, in each block diagram, each block represents a configuration of a functional unit instead of a configuration of a hardware unit unless otherwise described.

<FIG> is a diagram for describing an outline of an image analysis apparatus <NUM> according to the present example embodiment. Note that, <FIG> is exemplification for facilitating understanding of the image analysis apparatus <NUM>, and a function of the image analysis apparatus <NUM> is not limited to that represented in <FIG>.

The image analysis apparatus <NUM> performs an analysis of a plurality of captured images <NUM> including an identification card <NUM> of a user <NUM>. A camera <NUM> is a camera that generates the captured image <NUM>. The camera <NUM> generates a time-series of the captured images <NUM> by repeatedly capturing the identification card <NUM> of the user <NUM>. For example, a time-series of the captured images <NUM> constitute one video. The identification card <NUM> is any certificate usable for proving person's identity. For example, the identification card <NUM> is a driver's license, another license, a passport, various certificates, a student's identification card, an identification card of a company, a health insurance card, or the like.

The captured image <NUM> is used for proving identity of the user <NUM>. For example, there is a case where an image including an identification card instead of an original of the identification card is required to be provided. As an example, there is a case where a procedure of opening a bank account, creating a credit card, and the like via the Internet is performed. In such a case, it is difficult to provide an original of an identification card. Thus, personal identification of the user <NUM> is performed by using image data (such as the captured image <NUM> described above) acquired by capturing an identification card.

When personal identification is performed by using image data about an identification card in such a manner, a method of specifying, by a user, image data about a main surface (a surface on which main information is described) of the identification card or image data about a surface (hereinafter, a back surface) on a back side of the main surface, and performing personal identification by using the image data specified by the user is conceivable. However, it is difficult for this method to prevent unauthorized use of an identification card. For example, once a user can acquire copies of a main surface and a back surface of an identification card of another person by some sort of method, the user can spoof to be the another person by providing image data in which the copies are captured by a camera.

In order to handle such a problem, the image analysis apparatus <NUM> confirms that the identification card <NUM> is captured at n kinds (n is an integer equal to or greater than two) of angles, and then outputs the captured image <NUM> including the identification card <NUM>. Specifically, the image analysis apparatus <NUM> detects, for each of n predetermined conditions, the captured image <NUM> that satisfies the predetermined condition. In other words, the image analysis apparatus <NUM> detects each of the captured image <NUM> that satisfies a first predetermined condition, the captured image <NUM> that satisfies a second predetermined condition,. , and the captured image <NUM> that satisfies an n-th predetermined condition. Hereinafter, processing of detecting the captured image <NUM> that satisfies an i-th predetermined condition (i is an integer that satisfies <NUM> ≤ i ≤ n) is referred to as i-th detection processing.

The i-th predetermined condition includes a condition that the "identification card <NUM> captured at an i-th predetermined angle is included in the captured image <NUM>". Thus, in first detection processing to n-th detection processing, the captured image <NUM> including the identification card <NUM> captured at a first predetermined angle, the captured image <NUM> including the identification card <NUM> captured at a second predetermined angle,. , and the captured image <NUM> including the identification card <NUM> captured at an n-th predetermined angle are each detected. Note that, it is assumed that <NUM>° ≤ first predetermined angle < second predetermined angle <. < n-th predetermined angle < <NUM>°. In this way, by detecting the captured image <NUM> that satisfies each of the n predetermined conditions, it can be confirmed that the identification card <NUM> is captured at the n kinds of angles.

<FIG> is a plan view representing a scene in which the identification card <NUM> is captured at an angle X by using the camera <NUM>. As illustrated in <FIG>, the "identification card <NUM> is captured at the angle X" means that the "main surface of the identification card <NUM> is rotated by the angle X from a state where the main surface of the identification card <NUM> faces the front of the camera <NUM>, and the identification card <NUM> is captured in that state". Thus, in a case of <NUM>°, the main surface of the identification card <NUM> is captured, in a case of <NUM>°, the back surface of the identification card <NUM> is captured, and, in cases of <NUM>° and <NUM>°, a side surface of the identification card <NUM> is captured.

When all of the captured image <NUM> that satisfies the first predetermined condition to the captured image <NUM> that satisfies the n-th predetermined condition are detected, the image analysis apparatus <NUM> outputs one or more of the n captured images <NUM> being detected. In other words, one or more of the captured image <NUM> including the identification card <NUM> captured at the first predetermined angle, the captured image <NUM> including the identification card <NUM> captured at the second predetermined angle,. , and the captured image <NUM> including the identification card <NUM> captured at the n-th predetermined angle are output.

The image analysis apparatus <NUM> according to the present example embodiment confirms that the identification card <NUM> is captured at n kinds of angles, and then outputs the captured image <NUM> including the identification card <NUM>. Thus, as compared to a case where an image of the identification card <NUM> is output without performing such confirmation, a probability that an image of the identification card <NUM> captured in a desired state is included in the captured image <NUM> to be output can be increased. Thus, when the captured image <NUM> output from the image analysis apparatus <NUM> is used for personal identification of the user <NUM>, the captured image <NUM> needed for the personal identification can be more reliably acquired from the image analysis apparatus <NUM>.

Hereinafter, the present example embodiment will be described in more detail.

<FIG> is a diagram illustrating a functional configuration of the image analysis apparatus <NUM> according to an example embodiment <NUM>. The image analysis apparatus <NUM> includes a detection unit <NUM> and an image output unit <NUM>. The detection unit <NUM> performs each i-th detection processing described above. Herein, as described above, since <NUM> ≤ i ≤ n and n ≥ <NUM>, the detection unit <NUM> performs at least the first detection processing of detecting the captured image <NUM> that satisfies the first predetermined condition and the second detection processing of detecting the captured image <NUM> that satisfies the second predetermined condition. The image output unit <NUM> outputs one or more of the plurality of captured images <NUM> detected by the detection unit <NUM>.

Each functional component unit of the image analysis apparatus <NUM> may be achieved by hardware (for example, a hard-wired electronic circuit, and the like) that achieves each functional component unit, and may be achieved by a combination of hardware and software (for example, a combination of an electronic circuit and a program that controls the electronic circuit, and the like). Hereinafter, a case where each functional component unit of the image analysis apparatus <NUM> is achieved by the combination of hardware and software will be further described.

<FIG> is a diagram illustrating a computer <NUM> for achieving the image analysis apparatus <NUM>. The computer <NUM> is any computer. For example, the computer <NUM> is a portable computer such as a smartphone and a tablet terminal. In addition, for example, the computer <NUM> may be a stationary computer such as a personal computer (PC) and a server machine.

The computer <NUM> may be a dedicated computer designed for achieving the image analysis apparatus <NUM>, and may be a general-purpose computer. In the latter case, for example, a function of the image analysis apparatus <NUM> is achieved in the computer <NUM> by installing a predetermined application into the computer <NUM>. The application described above is formed of a program for achieving each functional component unit of the image analysis apparatus <NUM>. In other words, the program causes the computer <NUM> to execute each of processing performed by the detection unit <NUM> and processing performed by the image output unit <NUM>.

The computer <NUM> includes a bus <NUM>, a processor <NUM>, a memory <NUM>, a storage device <NUM>, an input/output interface <NUM>, and a network interface <NUM>. The bus <NUM> is a data transmission path for allowing the processor <NUM>, the memory <NUM>, the storage device <NUM>, the input/output interface <NUM>, and the network interface <NUM> to transmit and receive data with one another. However, a method of connecting the processor <NUM> and the like to each other is not limited to bus connection.

The processor <NUM> is various types of processors such as a central processing unit (CPU), a graphics processing unit (GPU), and a field-programmable gate array (FPGA). The memory <NUM> is a main storage apparatus achieved by using a random access memory (RAM) and the like. The storage device <NUM> is an auxiliary storage apparatus achieved by using a hard disk, a solid state drive (SSD), a memory card, a read only memory (ROM), or the like.

The input/output interface <NUM> is an interface for connecting the computer <NUM> and an input/output device. For example, an input apparatus such as a keyboard and an output apparatus such as a display apparatus are connected to the input/output interface <NUM>.

In addition, for example, the camera <NUM> is connected to the input/output interface <NUM>. In this way, each captured image <NUM> generated by the camera <NUM> is input to the computer <NUM>. The captured image <NUM> is stored in the memory <NUM> and the storage device <NUM>.

The network interface <NUM> is an interface for connecting the computer <NUM> to a communication network. The communication network is, for example, a local area network (LAN) and a wide area network (WAN).

The storage device <NUM> stores a program module (a program module that achieves the application described above) that achieves each functional component unit of the image analysis apparatus <NUM>. The processor <NUM> achieves a function associated with each program module by reading each of the program modules to the memory <NUM> and executing the program module.

The camera <NUM> is any camera that generates image data (the captured image <NUM>) representing a result of capturing by performing capturing. For example, the camera <NUM> is a camera mounted on a smartphone, a tablet terminal, a notebook PC, or the like. However, the camera <NUM> may be a camera externally attached to the image analysis apparatus <NUM>.

<FIG> is a flowchart illustrating a flow of processing performed by the image analysis apparatus <NUM> according to the example embodiment <NUM>. S102 to S108 are loop processing of performing the first detection processing to the n-th detection processing. In S102, the detection unit <NUM> determines whether i ≤ n is satisfied. Note that, an initial value of i is <NUM>.

When i ≤ n is satisfied, the processing in <FIG> proceeds to S104. On the other hand, when i ≤ n is not satisfied, the processing in <FIG> proceeds to S110.

In S104, the image output unit <NUM> detects the captured image <NUM> that satisfies the i-th predetermined condition (performs the i-th detection processing). The detection unit <NUM> adds <NUM> to i (S106). Since S108 is an end of the loop processing A, the processing in <FIG> proceeds to S102.

When the processing in <FIG> reaches S110, the image output unit <NUM> outputs one or more of the n captured images <NUM> being detected.

Herein, the flowchart in <FIG> does not illustrate processing when the captured image <NUM> that satisfies the i-th predetermined condition is not detected in the i-th detection processing. Processing performed by the image analysis apparatus <NUM> when the captured image <NUM> that satisfies the i-th predetermined condition is not detected in the i-th detection processing is optional. For example, the image analysis apparatus <NUM> may end the processing illustrated in <FIG>. In other words, in this case, an output of the captured image <NUM> is not performed. Herein, before the image analysis apparatus <NUM> ends the processing illustrated in <FIG>, the image analysis apparatus <NUM> may output a warning message indicating that the captured image <NUM> that satisfies a predetermined condition is not detected (i.e., that capturing of the identification card <NUM> is not properly performed), and the like.

In addition, for example, when the captured image <NUM> that satisfies the i-th predetermined condition is not detected in the i-th detection processing, the image analysis apparatus <NUM> may output a warning message and the like that capturing of the identification card <NUM> needs to be properly performed in such a way as to satisfy the i-th predetermined condition (for example, that the identification card <NUM> needs to be captured at the i-th predetermined angle), and then may perform the i-th detection processing again.

Note that, a condition for ending the i-th detection processing in a situation where the captured image <NUM> that satisfies the i-th predetermined condition is not detected is optional. For example, the detection unit <NUM> ends the i-th detection processing when a predetermined period of time has elapsed since the i-th detection processing starts, or when the i-th detection processing is performed on, as a target, a predetermined number or more of the captured images <NUM>.

In order to make the following description clear, a more specific usage environment of the image analysis apparatus <NUM> will be illustrated. However, a usage environment of the image analysis apparatus <NUM> is not limited to the example described herein.

<FIG> is a diagram illustrating the usage environment of the image analysis apparatus <NUM>. In this example, the image analysis apparatus <NUM> is achieved in a user terminal <NUM>. The user terminal <NUM> is, for example, a smartphone provided with the camera <NUM>.

The user <NUM> provides an image of the identification card <NUM> to a server apparatus <NUM> by using the user terminal <NUM>. For example, an application for causing the user terminal <NUM> to function as the image analysis apparatus <NUM> is installed in the user terminal <NUM>. The user <NUM> activates and operates this application. As a result, a message that prompts capturing of the identification card <NUM> to be performed is displayed on a display apparatus of the user terminal <NUM>, and the camera <NUM> is also activated. The user <NUM> performs capturing of the identification card <NUM> by using the camera <NUM>. For example, the user <NUM> causes the camera <NUM> to capture the identification card <NUM> while rotating the identification card <NUM>.

The user terminal <NUM> analyzes, in order, a time-series of the captured images <NUM> generated by the camera <NUM> through the operation described above. For example, the user terminal <NUM> performs the first detection processing on, as a target, each of the captured images <NUM> in order from a first captured image <NUM> in time series. When the captured image <NUM> that satisfies the first predetermined condition is detected in the first detection processing, the user terminal <NUM> performs the second detection processing on, as a target, each of the captured images <NUM> being generated after the detected captured image <NUM>. Furthermore, when the captured image <NUM> that satisfies the second predetermined condition is detected in the second detection processing, the user terminal <NUM> performs a third detection processing on, as a target, each of the captured images <NUM> being generated after the detected captured image <NUM>. Hereinafter, the user terminal <NUM> similarly performs the processing in order up to the n-th detection processing.

For example, in the example in <FIG>, the first detection processing to a fourth detection processing are performed. A first predetermined angle to a fourth predetermined angle are each <NUM>°, <NUM>°, <NUM>°, and <NUM>°. In other words, four captured images <NUM> being the captured image <NUM> in which the main surface of the identification card <NUM> is captured from the front, the captured image <NUM> in which the main surface of the identification card <NUM> is captured obliquely from <NUM>°, the captured image <NUM> in which the back surface of the identification card <NUM> is captured obliquely from <NUM>°, and the captured image <NUM> in which the back surface of the identification card <NUM> is captured from the front are detected by the detection unit <NUM>.

The user terminal <NUM> provides, to the server apparatus <NUM>, at least one or more of the captured images <NUM> detected in each detection processing. For example, all of the four captured images <NUM> described above are transmitted to the server apparatus <NUM>. These captured images <NUM> are used for personal identification of the user <NUM>. Note that, any method can be used as a specific method of performing personal identification of a user by using an image in which the identification card <NUM> is captured.

Herein, by performing each detection processing described above, the captured image <NUM> including the identification card <NUM> captured at a predetermined angle is transmitted from the user terminal <NUM> to the server apparatus <NUM>. Thus, when the identification card <NUM> captured at the predetermined angle is needed for personal identification, a situation where "the server apparatus <NUM> requires the user terminal <NUM> to provide the captured image <NUM> again for a reason that the identification card <NUM> is not captured at the predetermined angle" can be prevented from occurring. In this way, personal identification of the user <NUM> can be performed more smoothly.

Note that, as described above, a usage environment of the image analysis apparatus <NUM> is not limited to the example described herein. For example, the image analysis apparatus <NUM> is not limited to a portable terminal such as a smartphone. For example, a desktop PC may be used as the image analysis apparatus <NUM>, and a camera connected to the desktop PC may be used as the camera <NUM>.

The detection unit <NUM> acquires the captured image <NUM>, and performs each detection processing. Various methods of acquiring the captured image <NUM> by the detection unit <NUM> can be used. For example, the detection unit <NUM> receives the captured image <NUM> transmitted from the camera <NUM>. In addition, for example, the detection unit <NUM> accesses the camera <NUM>, and acquires the captured image <NUM> stored in the camera <NUM>.

Note that, the camera <NUM> may store the captured image <NUM> in a storage apparatus (for example, the storage device <NUM>) provided outside the camera <NUM>. In this case, the detection unit <NUM> acquires the captured image <NUM> by accessing the storage apparatus.

A timing at which the detection unit <NUM> acquires the captured image <NUM> is optional. For example, each time the captured image <NUM> is generated by the camera <NUM>, the detection unit <NUM> acquires the newly generated captured image <NUM>. In addition, for example, the detection unit <NUM> may regularly acquire the captured image <NUM> that has not yet been acquired. For example, when the detection unit <NUM> acquires the captured image <NUM> once in a second, the detection unit <NUM> collectively acquires one or more of the captured images <NUM> being generated in one second (for example, <NUM> captured images <NUM> when the camera <NUM> is a video camera having a frame rate of <NUM> frames/second (fps)).

<Performing of Detection Processing: S104>.

The detection unit <NUM> performs the first detection processing to the n-th detection processing (S104). There are various types of specific methods of achieving the detection processing. For example, the detection unit <NUM> is provided in advance with a discriminator (hereinafter, an i-th discriminator) that has performed learning in such a way as to discriminate whether the captured image <NUM> satisfies the i-th predetermined condition. The i-th discriminator outputs a discrimination result of whether the captured image <NUM> satisfies the i-th predetermined condition in response to an input of the captured image <NUM>. For example, a first discriminator outputs a discrimination result of whether the captured image <NUM> satisfies the first predetermined condition in response to an input of the captured image <NUM>. Similarly, a second discriminator outputs a discrimination result of whether the captured image <NUM> satisfies the second predetermined condition in response to an input of the captured image <NUM>. For example, the discrimination result is a flag indicating <NUM> when the captured image <NUM> satisfies the i-th predetermined condition and indicating <NUM> when the captured image <NUM> does not satisfy the i-th predetermined condition. Herein, various models such as a neural network and a support vector machine (SVM) can be used as a model of the discriminator.

The discriminator is provided for each kind of an identification card, for example. For example, when a driver's license and a passport can be used as an identification card, both of a discriminator that targets the captured image <NUM> including a driver's license and a discriminator that targets the captured image <NUM> including a passport are provided in advance. Note that, when any of a plurality of kinds of an identification card can be used for personal identification, which kind of identification card is included in the captured image <NUM> provided from the user <NUM> (i.e., which kind of identification card is provided for personal identification by a user) needs to be specified in advance by the user.

The discriminator has performed learning in advance in such a way as to be able to achieve the processing described above. Specifically, learning of the i-th discriminator is performed by using, as learning data, positive example data that an "image that satisfies the i-th predetermined condition, a discrimination result = <NUM>" and negative example data that an "image that does not satisfy the i-th predetermined condition, a discrimination result = <NUM>". Herein, an existing technique can be used as a technique for performing learning of the discriminator by using the positive example data and the negative example data.

For example, it is assumed that the i-th predetermined condition is that the "identification card <NUM> captured at the i-th predetermined angle is included". In this case, an image included in the positive example data is an image including the identification card <NUM> captured at the i-th predetermined angle. Further, an image included in the negative example data is an image that does not include the identification card <NUM> captured at the i-th predetermined angle.

Herein, the discriminator is not used for only an identification card of a specific individual as a target, and is used for an identification card of various users as a target. Thus, an identification card included in an image used for learning does not need to completely coincide with the identification card <NUM> to be detected, and may have, to some extent, a feature of an identification card of the same kind as that of the identification card <NUM> to be detected. For example, when a driver's license is handled as the identification card <NUM>, the positive example data used for learning of the i-th discriminator may include an image to a degree that a feature of the driver's license viewed from the i-th predetermined angle is clear.

In this way, an image used for learning may not be necessarily an image in which a formal identification card (for example, an original of an identification card issued by government and municipal offices) is captured. For example, an image used for learning can be generated by capturing a sample and the like of the identification card <NUM>. In addition, for example, an image used for learning may be artificially generated by using a technique such as generative adversarial networks (GAN).

Further, in a condition that the "identification card <NUM> captured at the i-th predetermined angle is included", a slight difference in angle of the identification card <NUM> may be permitted. For example, when a predetermined condition that the "identification card <NUM> captured obliquely at <NUM>° is included" is used, the captured image <NUM> in which the identification card <NUM> is captured obliquely at <NUM>° or <NUM>° may also be handled as an image that satisfies the predetermined condition. For example, such an i-th discriminator that can permit a slight error can be constructed by using, as an image of the positive example data used for learning, not only an image in which the identification card <NUM> is captured at the i-th predetermined angle but also an image in which the identification card <NUM> is captured at an angle deviated from the i-th predetermined angle within a range of permitted errors.

Note that, a method of achieving each detection processing is not limited to a method of using the discriminator. For example, it is assumed that the i-th predetermined condition is that the "identification card <NUM> captured at the i-th predetermined angle is included". In this case, an image feature (hereinafter, an i-th image feature) of an image region representing the identification card <NUM> captured at the i-th predetermined angle is prepared for each i-th predetermined angle, and is stored in advance in a storage apparatus that can be accessed from the detection unit <NUM>. The detection unit <NUM> uses the image feature stored in the storage apparatus.

For example, the detection unit <NUM> determines whether an image feature having a high degree of similarity with the i-th image feature (having a degree of similarity equal to or more than a predetermined threshold value) is included in the captured image <NUM>. When the image feature having a high degree of similarity with the i-th image feature is included in the captured image <NUM>, the detection unit <NUM> determines that the identification card <NUM> captured at the i-th predetermined angle is included in the captured image <NUM>. On the other hand, when the image feature having a high degree of similarity with the i-th image feature is not included in the captured image <NUM>, the detection unit <NUM> determines that the identification card <NUM> captured at the i-th predetermined angle is not included in the captured image <NUM>.

Herein, similarly to learning of the discriminator described above, an image used for generation of the i-th image feature does not necessarily need to be an image in which a formal identification card is captured. For example, an image feature extracted from an image generated by capturing a replica of the identification card <NUM> or an image feature extracted from an image artificially generated by using a technique such as GAN may be used.

The first detection processing to the n-th detection processing may be simultaneously performed, may be performed in any order, or may be performed in a predetermined order. When the first detection processing to the n-th detection processing are performed in a predetermined order, for example, the detection unit <NUM> performs each one of the first detection processing to the n-th detection processing in this order as illustrated in the flowchart in <FIG>. In other words, when the captured image <NUM> that satisfies the i-th predetermined condition is detected in the i-th detection processing, the detection unit <NUM> performs (i+<NUM>)-th detection processing. In other words, detection of the captured image <NUM> that satisfies an (i+<NUM>)-th predetermined condition is not performed until the captured image <NUM> that satisfies the i-th predetermined condition is detected.

The i-th predetermined condition may include another condition in addition to the condition that an "image region representing the identification card <NUM> captured at the i-th predetermined angle is included". For example, it is assumed that the identification card <NUM> includes a face photograph of a person himself/herself. In this case, by performing capturing in such a way as to include, in the captured image <NUM>, not only the identification card <NUM> but also a face of a provider of the identification card <NUM> (see the captured image <NUM> in <FIG>), whether the provider of the identification card <NUM> is a rightful owner of the identification card <NUM> (a person having his/her identity proved by the identification card <NUM>) can be determined by determining a degree of coincidence between an image of the face of the provider of the identification card <NUM> included in the captured image <NUM> and the face image of the identification card <NUM> included in the captured image <NUM>.

Thus, for example, a condition that a "degree of coincidence between an image of a face of a provider of the identification card <NUM> included in the captured image <NUM> and a face image of the identification card <NUM> included in the captured image <NUM> satisfies a reference (degree of coincidence is equal to or more than a threshold value)" may be included in the predetermined condition. In this case, for example, the detection unit <NUM> extracts, from the captured image <NUM>, a face of a provider of the identification card <NUM> and a face image of the identification card <NUM>, and computes a degree of coincidence between the face of the provider and the face image. Note that, an existing technique can be used as a technique for computing a degree of coincidence between face images.

Note that, depending on an angle of the captured identification card <NUM>, a face image of the identification card <NUM> may not be included in the captured image <NUM>, or a feature of a face cannot be accurately extracted from a face image included in the captured image <NUM>. Thus, a condition related to a degree of coincidence between a face of a provider of the identification card <NUM> and a face image of the identification card <NUM> is suitably included only in a predetermined condition that the i-th predetermined angle is an angle at which the identification card <NUM> is captured in a state where a feature of a face can be sufficiently extracted from a face image included in the identification card <NUM>. For example, when a face image is included in the main surface of the identification card <NUM> and a first predetermined angle is <NUM>° (an angle at which the main surface of the identification card <NUM> is captured from the front), a condition that a "degree of coincidence between an image of a face of a provider of the identification card <NUM> included in the captured image <NUM> and a face image of the identification card <NUM> included in the captured image <NUM> satisfies a reference" may be included only in a first predetermined condition.

For example, a condition related to a background (hereinafter, a background of the captured image <NUM>) of the identification card <NUM> in the captured image <NUM> is included in the i-th predetermined condition. Specifically, a condition that a degree of coincidence between a background of the captured image <NUM> to be determined and a background of the captured image <NUM> detected in (i-<NUM>)-th detection processing or previous detection processing satisfies a reference (for example, the degree of coincidence is equal to or more than a reference value) can be included in the i-th predetermined condition. By using such a predetermined condition, the captured images <NUM> detected in each of the detection processing have a higher degree of coincidence between backgrounds. Thus, fraud such as falsification added to a series of the captured images <NUM> can be prevented. Note that, when the first detection processing to the n-th detection processing are performed in this order, a condition of a background may not be included in the first predetermined condition.

For example, a condition that a "degree of coincidence between a background of the captured image <NUM> detected in the i-th detection processing and a background of the captured image <NUM> detected in the (i-<NUM>)-th detection processing satisfies a reference" is used as the i-th predetermined condition. In other words, a degree of coincidence between backgrounds is set in such a way as to satisfy the reference in the captured image <NUM> detected in each of two continuous detection processing.

In addition, for example, a condition that a "degree of coincidence between a background of the captured image <NUM> detected in the i-th detection processing and a background of the captured image <NUM> detected in the first detection processing satisfies a reference" is used as the i-th predetermined condition. In other words, a degree of coincidence between a background of each of the captured images <NUM> detected in the second detection processing to the n-th detection processing and a background of the captured image <NUM> detected first (detected in the first detection processing) is set in such a way as to satisfy the reference.

Note that, whether a degree of coincidence between backgrounds in two captured images <NUM> different from each other satisfies the reference can be determined by various methods. Herein, in order to make description clear, two compared captured images <NUM> are referred to as captured images A and B. For example, the detection unit <NUM> computes an image feature of a background for each of the captured images A and B. Herein, a background of the captured image <NUM> is a portion (the captured image <NUM> in which an image region representing the identification card <NUM> is masked) acquired by excluding an image region representing the identification card <NUM> from the captured image <NUM>. Then, the detection unit <NUM> determines whether a degree of coincidence between the image features of the backgrounds is equal to or more than a reference value. When the degree of coincidence between the image features of the backgrounds is equal to or more than the reference value, the detection unit <NUM> determines that the degree of coincidence between the backgrounds of the captured images A and B satisfies the reference. On the other hand, when the degree of coincidence between the image features of the backgrounds is not equal to or more than the reference value, the detection unit <NUM> determines that the degree of coincidence between the backgrounds of the captured images A and B does not satisfy the reference.

The detection unit <NUM> may divide a comparison between backgrounds into two that are <NUM>) a comparison between faces of the user <NUM> and <NUM>) a comparison between backgrounds other than the faces. For example, the comparison is performed as follows. First, the detection unit <NUM> computes an image feature of an image region representing a face for each of the captured images A and B. Then, the detection unit <NUM> computes a degree of coincidence between the image feature of the face in the captured image A and the image feature of the face in the captured image B.

Furthermore, the detection unit <NUM> computes an image feature of a background other than the face (the captured image <NUM> acquired by excluding an image region of the identification card <NUM> and the image region of the face) for each of the captured images A and B. Then, the detection unit <NUM> computes a degree of coincidence between the image feature of the background other than the face in the captured image A and the image feature of the background other than the face in the captured image B.

When both of the degree of coincidence between the image features of the faces and the degree of coincidence between the image features of the backgrounds other than the faces are equal to or more than a threshold value, the detection unit <NUM> determines that the degree of coincidence between the backgrounds of the captured images A and B satisfies the reference. On the other hand, when at least one of the degree of coincidence between the image features of the faces and the degree of coincidence between the image features of the backgrounds other than the faces is not equal to or more than the threshold value, the detection unit <NUM> determines that the degree of coincidence between the backgrounds of the captured images A and B does not satisfy the reference.

The image output unit <NUM> outputs information including one or more of the n captured images <NUM> detected in the first detection processing to the n-th detection processing. Hereinafter, the information is referred to as output information. For example, the output information is information in which identification information about the user <NUM> and one or more of the captured images <NUM> are associated with each other. By associating the captured image <NUM> included in the output information with the identification information about the user <NUM>, whose identity is confirmed by using the captured image <NUM> can be determined. In other words, the captured image <NUM> included in the output information is used for personal identification of the user <NUM> determined by the identification information included in the output information.

However, when the captured image <NUM> and the identification information about the user <NUM> can be associated with each other in an apparatus (such as the server apparatus <NUM> in <FIG>) that receives the output information, the identification information about the user <NUM> may not be included in the output information. For example, a predetermined connection is established between the server apparatus <NUM> and the image analysis apparatus <NUM>, and the identification information about the user <NUM> and the output information are transmitted from the image analysis apparatus <NUM> to the server apparatus <NUM> via the connection. With this method, even when the identification information about the user <NUM> and the output information are transmitted at timings different from each other, the server apparatus <NUM> can associate the identification information about the user <NUM> with the output information (i.e., the captured image <NUM> included in the output information). Note that, in order to for simplify description below, it is assumed that the identification information about the user <NUM> is included in the output information unless otherwise specified.

The output information may include all of the captured images <NUM> detected by the detection unit <NUM>, or may include only a part of the captured images <NUM>. In the latter case, for example, the output information includes only the captured image <NUM> detected in a predetermined number of detection processing less than n.

Herein, it is conceivable that the captured image <NUM> in which the main surface of the identification card <NUM> is captured is highly useful in personal identification of the user <NUM>. Thus, it is preferable that at least the captured image <NUM> in which the main surface of the identification card <NUM> is captured is included in the output information. Further, information described on the back surface of the identification card <NUM> may also be important. In this case, it is conceivable that the captured image <NUM> in which the back surface of the identification card <NUM> is captured is also highly useful in personal identification of the user <NUM>. Thus, in this case, it is preferable that the captured image <NUM> in which the back surface of the identification card <NUM> is captured is also included in the output information.

Note that, which detection processing each of the main surface and the back surface of the identification card <NUM> is detected can be recognized in advance. For example, it is assumed that the first predetermined angle is <NUM>° (a state where the main surface of the identification card <NUM> faces the camera <NUM>), and the n-th predetermined angle is <NUM>° (a state where the back surface of the identification card <NUM> faces the camera <NUM>). In this case, the captured image <NUM> including the main surface of the identification card <NUM> is the captured image <NUM> detected in the first detection processing, and the captured image <NUM> including the back surface of the identification card <NUM> is the captured image <NUM> detected in the n-th detection processing. Then, the image output unit <NUM> includes, in the output information, at least the captured image <NUM> detected in the first detection processing and the captured image <NUM> detected in the n-th detection processing.

Note that, not only the captured image <NUM> detected by the detection unit <NUM> but also the other captured image <NUM> may be included in the output information. For example, the image output unit <NUM> may include, in the output information, all of the captured images <NUM> generated by the camera <NUM>. In addition, for example, the image output unit <NUM> may include, in the output information, all of a time-series of the captured images <NUM> from the captured image <NUM> detected in the first detection processing to the captured image <NUM> detected in the n-th detection processing. For example, in this case, when it is assumed that the first predetermined angle = <NUM>° and the n-th predetermined angle = <NUM>°, the output information includes a time-series of the captured images <NUM> (video) in which a series of flows from the state where the main surface of the identification card <NUM> faces the camera <NUM> until the back surface of the identification card <NUM> faces the camera <NUM> is captured.

The processing of confirming identity of the user <NUM> by using the captured image <NUM> may be manually performed, or may be automatically performed by an apparatus. Note that, any method can be used as a method of performing personal identification of a user by using an image including an identification card of the user.

<FIG> is a block diagram illustrating a functional configuration of an image analysis apparatus <NUM> according to an example embodiment <NUM>. The image analysis apparatus <NUM> according to the example embodiment <NUM> has a function similar to that of the image analysis apparatus <NUM> according to the example embodiment <NUM> except for a point described below.

In the image analysis apparatus <NUM> according to the example embodiment <NUM>, as a premise, each one of first detection processing to n-th detection processing is performed in this order. In other words, when a captured image <NUM> is detected in i-th detection processing, a detection unit <NUM> performs (i+<NUM>)-th detection processing.

The image analysis apparatus <NUM> according to the example embodiment <NUM> includes a guide output unit <NUM>. The guide output unit <NUM> outputs a guide to a user <NUM> in such a way as to increase a probability that the captured image <NUM> that satisfies each predetermined condition is acquired. Hereinafter, the guide to be output in such a way as to increase a probability that the captured image <NUM> that satisfies an i-th predetermined condition is acquired is referred to as an i-th guide.

The image analysis apparatus <NUM> performs the i-th detection processing after outputting the i-th guide. Further, when the captured image <NUM> that satisfies the i-th predetermined condition is detected in the i-th detection processing, the image analysis apparatus <NUM> outputs an (i+<NUM>)-th guide. Subsequently, the image analysis apparatus <NUM> outputs the (i+<NUM>)-th guide.

The image analysis apparatus <NUM> according to the present example embodiment increases a probability that an identification card <NUM> is captured in such a way as to satisfy a predetermined condition. Thus, the image analysis apparatus <NUM> can increase a probability that the captured image <NUM> that satisfies the predetermined condition can be detected in each detection processing.

Further, the user <NUM> can provide a correct image of the identification card <NUM> by performing capturing of the identification card <NUM> according to a guide output from the image analysis apparatus <NUM>. Thus, usability of the image analysis apparatus <NUM> improves for the user <NUM>.

Hereinafter, the image analysis apparatus <NUM> according to the present example embodiment will be described in more detail.

<FIG> is a flowchart illustrating a flow of processing performed by the image analysis apparatus <NUM> according to the example embodiment <NUM>. The flowchart in <FIG> is the same as the flowchart in <FIG> except for a point that an output of the i-th guide (S202) is added before the i-th detection processing (S104).

The guide output unit <NUM> outputs a guide that increases a probability that the captured image <NUM> that satisfies the predetermined condition is captured. For example, when the i-th predetermined condition is a condition that the "identification card <NUM> captured at an i-th predetermined angle is included", the i-th guide is a guide that prompts capturing of the identification card <NUM> at the i-th predetermined angle.

<FIG> is a diagram illustrating a guide output from the guide output unit <NUM>. In this example, the image analysis apparatus <NUM> performs the first detection processing to a fourth detection processing. In other words, n = <NUM>. Then, a first predetermined angle to a fourth predetermined angle are each <NUM>°, <NUM>°, <NUM>°, and <NUM>°.

In this case, first, the guide output unit <NUM> outputs a first guide <NUM> having a content that "face main surface toward camera and hold still" and the like to a display apparatus (for example, a display apparatus provided on a user terminal <NUM> in <FIG>) that can be viewed by the user <NUM>. By viewing the first guide <NUM>, the user <NUM> can recognize that the main surface of the identification card <NUM> should face a camera <NUM>.

When the captured image <NUM> that satisfies a first predetermined condition is detected, the guide output unit <NUM> outputs a second guide <NUM> having a content that "slowly rotate to <NUM>° and hold still" and the like. When the captured image <NUM> that satisfies a second predetermined condition is detected, the guide output unit <NUM> outputs a third guide <NUM> having a content that "slowly rotate to <NUM>° and hold still" and the like. When the captured image <NUM> that satisfies a third predetermined condition is detected, the guide output unit <NUM> outputs a fourth guide <NUM> having a content that "slowly rotate until back surface faces camera and hold still" and the like.

Note that, an output method of a guide is not limited to a method of outputting a message to a display apparatus. For example, the guide output unit <NUM> may output the guide described above by sound.

In addition to the guide described above, a message for causing the user <NUM> to recognize that the captured image <NUM> that satisfies the predetermined condition is detected may be further displayed. For example, in the example in <FIG>, when the captured image <NUM> that satisfies the first predetermined condition is detected after the first guide <NUM> is output, a message, such as "OK" and "capturing successful", indicating that the captured image <NUM> that satisfies the first predetermined condition is detected is output. This message may be output simultaneously with the second guide <NUM>, or may be output before the second guide <NUM> is output. The same also applies when the captured image <NUM> that satisfies the other predetermined condition is detected. By outputting a message representing that the captured image <NUM> that satisfies the predetermined condition is detected in such a manner, usability of the image analysis apparatus <NUM> further improves for the user <NUM>.

Further, as described above, a condition related to coincidence between backgrounds may be included in the predetermined condition. Thus, for example, a message, such as "do not change background", "do not change capturing place", and "do not move", that prompts capturing to be performed in a situation where a background does not change may be further included in the guide.

While the example embodiments of the present invention have been described with reference to the drawings, the example embodiments are only exemplification of the present invention, and combination of each of the above-described example embodiments or various configurations other than the above-described example embodiments can also be employed as long as they fall into the scope of the appended claims.

Claim 1:
A program causing a computer to execute:
first detection processing of detecting (S104), from one or more captured images generated by a camera for personal identification of a user, a first captured image that satisfies a first predetermined condition;
second detection processing of detecting (S104), from the one or more captured images generated by the camera, a second captured image that satisfies a second predetermined condition; and
image output processing of outputting (S110), upon detecting the first captured image and the second captured image, at least either one of the first and second captured images,
characterized in that:
the first predetermined condition includes a condition that a first image region included in the first captured image represents one surface of a target identification card captured at a first predetermined angle and has a first degree of similarity with a first image feature equal to or more than a predetermined threshold value, the first image feature being of the target identification card or of an identification card of the same kind as the target identification card, and being stored in advance,
the second predetermined condition includes a condition that a second image region included in the second captured image represents the one surface of the target identification card captured at a second predetermined angle and has a second degree of similarity with a second image feature equal to or more than the predetermined threshold value, the second image feature being of the target identification card or of said identification card of the same kind as the target identification card, and being stored in advance, and
the at least one of the first and second captured images outputted in image output processing is used for personal identification of the user.