Patent Description:
Emigration and immigration examination is performed at airports. An officer in charge of the emigration and immigration examination compares a photograph of a face attached to a passport and the face of a person in front of the officer. If the face image in the passport matches the face of the person in front of the officer, the emigration and immigration of the person is permitted.

In addition, in recent years, apparatuses that automatically perform the above emigration and immigration examination have been introduced. For example, a gate apparatus disclosed in PTL <NUM> performs matching between a face image attached to a passport and a face image of a person standing in front of the gate apparatus and examines emigration and immigration of the person based on the result of the matching.

[PTL <NUM>] <CIT>. Further emigration and immigration examination systems have been disclosed by <CIT>, <CIT> and <CIT>.

When the gate apparatus disclosed in PTL <NUM> is used, a user opens his or her passport to a page including a face image and places the page on a passport reader (a scanner) of the gate apparatus. This gate apparatus acquires data necessary for the above matching (<NUM>-to-<NUM> matching) from the passport reader (for example, see paragraph [<NUM>] in PTL <NUM>).

To use the gate apparatus disclosed in PTL <NUM>, a user needs to stand in front of the gate apparatus, open his or her passport, and have the gate apparatus to read information on the opened page via the passport reader. However, if the user is made to perform this operation, the significance of automatically performing the emigration and immigration examination is lost. That is, the gate apparatus disclosed in PTL <NUM> cannot allow its users to smoothly complete the emigration and immigration examination without stress.

The present invention contributes to realization of smooth emigration and immigration examination, and it is a principal object of the present invention to provide a gate apparatus, a server apparatus, an emigration and immigration examination system, a control method of a gate apparatus, and a control method of a server apparatus.

According to a first aspect of the present invention, there is provided an emigration and immigration examination system according to claim <NUM>.

The individual aspect of the present invention provides an emigration and immigration examination system that contribute to realization of smooth emigration and immigration examination. The advantageous effects of the present invention are not limited to the above advantageous effect. The present invention may provide other advantageous effects, instead of or in addition to the above advantageous effect.

First, an outline of an example embodiment will be described. In the following outline, various components are denoted by reference characters for the sake of convenience. That is, the following reference characters are used as examples to facilitate the understanding of the present invention. Thus, the description of the outline is not intended to impose any limitations. In addition, unless otherwise specified, an individual block illustrated in the drawings represents a configuration of a functional unit, not a hardware unit. An individual connection line between blocks in the drawings signifies both one-way and two-way directions. An arrow schematically illustrates a principal signal (data) flow and does not exclude bidirectionality. In the present description and drawings, elements that can be described in a like way will be denoted by a like reference character, and redundant description thereof will be omitted as needed.

A gate apparatus <NUM> according to an example embodiment includes an acquisition unit <NUM>, a matching request unit <NUM>, and a control unit <NUM> (see <FIG>). The acquisition unit <NUM> acquires biological information about an examination target user. The matching request unit <NUM> requests a server apparatus that stores biological information about users and MRZ (Machine Readable Zone) information written in machine readable zones in passports issued to the users in association with each other to perform matching on the biological information about the examination target user. The control unit <NUM> controls a gate so that the examination target user can pass through the gate if information is successfully read out from an IC (Integrated Circuit) chip in a passport issued to the examination target user by using MRZ information determined by the matching.

In an emigration and immigration system including the above gate apparatus <NUM>, users who wish to use the system previously register their biological information in the server apparatus. In this way, the users can complete the emigration and immigration examination by bringing their passports into contact with the gate apparatus <NUM> without opening the passports. As a result, convenient and smooth emigration and immigration examination can be realized for the users.

Hereinafter, specific example embodiments will be described in more detail with reference to the accompanying drawings.

A first example embodiment will be described in more detail with reference to drawings.

<FIG> is a diagram illustrating an example of a schematic configuration of an emigration and immigration examination system according to the first example embodiment. As illustrated in <FIG>, the emigration and immigration examination system includes a plurality of gate apparatuses <NUM>-<NUM> to <NUM>-<NUM> and a server apparatus <NUM>. In the following description, unless there is a particular reason to distinguish the gate apparatuses <NUM>-<NUM> to <NUM>-<NUM> from each other, any one of these gate apparatuses <NUM>-<NUM> to <NUM>-<NUM> will simply be referred to as a "gate apparatus <NUM>". In addition, while three gate apparatuses <NUM> are illustrated in <FIG>, the number of gate apparatuses <NUM> included in the system is not of course limited to any particular number. The emigration and immigration examination system includes at least one gate apparatus <NUM>.

The individual gate apparatus <NUM> is an apparatus that automatically performs an emigration and immigration examination procedure for users. The gate apparatus <NUM> includes a gate that can be opened and closed. If the gate apparatus <NUM> determines that a person located in front of the gate apparatus <NUM> possesses a correct passport, the gate apparatus <NUM> opens the gate and permits the user to pass through the gate. In this way, the gate apparatus <NUM> controls the gate based on the result of the emigration and immigration examination of the user.

The server apparatus <NUM> is an apparatus that realizes the emigration and immigration examination based on the above gate apparatus <NUM>. For example, the server apparatus <NUM> stores information about users who can use the gate apparatus <NUM> (the information will hereinafter be referred to as gate user information). Specifically, the server apparatus <NUM> stores biological information about users and information written in Machine Readable Zones (MRZs) in passports issued to the users (the information will hereinafter be referred to as MRZ information) in association with each other.

For example, the biological information about a user is data (feature values) calculated from physical features unique to this individual user. The physical features are about the face, a fingerprint, a voiceprint, a vein, a retina, or an iris pattern of an eye of the user. According to the invention, the biological information about a user are image data of a face image of the user. Any information including physical features of a user may be used as the biological information about the user.

The information (MRZ information) written in an MRZ in a passport includes the name, the nationality, the gender, the date of birth of the user, the passport number, the validity period, etc..

The gate apparatuses <NUM> and the server apparatus <NUM> can communicate with each other via wired or wireless communication means. The server apparatus <NUM> may be placed in the same airport where the gate apparatuses <NUM> are placed. Alternatively, the server apparatus <NUM> may be placed on a network (cloud).

Next, a schematic operation in the emigration and immigration examination system according to the first example embodiment will be described with reference to <FIG>.

The gate user information about a user of the emigration and immigration examination system is registered in the server apparatus <NUM>, for example, before departure of the user. That is, the gate user information about a user is registered in the server apparatus <NUM> before his or her departure date or before departure on his or her departure date. The server apparatus <NUM> includes a database for storing the gate user information, and after a user registers his or her gate user information, an entry corresponding to this user is added in the database. Hereinafter, the database for storing the gate user information will be referred to as a "gate user database". In the gate user database, biological information about at least one user and MRZ information written in a machine readable zone in the passport issued to the at least one user are stored in association with each other.

A user visits the airport on the departure data (the airport where the gate apparatuses <NUM> are installed). After the user completes boarding procedures such as a check-in procedure and a security check procedure, the user moves to an area (an emigration and immigration examination area) where the gate apparatuses <NUM> are installed.

When the user arrives at the emigration and immigration examination area, the user moves to a gate apparatus <NUM>.

When the user comes close to the gate apparatus <NUM> (when the distance between the user and the gate apparatus <NUM> reaches a predetermined distance or less), the gate apparatus <NUM> acquires (generates) biological information from the user. The biological information acquired by the gate apparatus <NUM> is the same kind of information as the biological information registered as the gate user information in the server apparatus <NUM>.

The gate apparatus <NUM> transmits the acquired biological information to the server apparatus <NUM>. More specifically, the gate apparatus <NUM> transmits a "matching request" including the acquired biological information to the server apparatus <NUM>.

The server apparatus <NUM> searches the gate user database by using the received biological information as a key and determines the MRZ information corresponding to the received biological information. The server apparatus <NUM> transmits a reply (a reply to the matching request) including the determined MRZ information to the gate apparatus <NUM>.

When the user arrives at the gate apparatus <NUM>, the user holds his or her passport on a reader (a reader <NUM> to be described below) of the gate apparatus <NUM> (brings his or her passport into contact with a reader). That is, the user brings his or her passport into contact with a reader of the gate apparatus <NUM> in the same way as people bring their transportation IC cards into contact with card readers at ticket gates in stations.

The gate apparatus <NUM> attempts to read out information written in an IC chip in the passport via the reader. If the gate apparatus <NUM> succeeds in reading out the information written in the IC chip by using the MRZ information acquired from the server apparatus <NUM>, the gate apparatus <NUM> determines that the user possesses a correct passport and opens the gate.

The user passes through the gate, and the emigration and immigration examination ends.

As described above, in the emigration and immigration examination system according to the first example embodiment, the gate user information is previously registered in the server apparatus <NUM>. In this way, the users can pass through the gate by bringing their passports into contact with the reader without opening their passports.

Next, the individual apparatuses included in the emigration and immigration examination system according to the first example embodiment will be described in detail. The following description will be made based on an example in which information about faces of people are used as the biological information registered in the server apparatus <NUM>.

<FIG> is a diagram for illustrating an example of a processing configuration (processing modules) of the server apparatus <NUM> according to the first example embodiment. As illustrated in <FIG>, the server apparatus <NUM> includes a communication control unit <NUM>, a user information registration unit <NUM>, a first matching unit <NUM>, and a storage unit <NUM>.

The communication control unit <NUM> is means for controlling communication with other apparatuses. Specifically, the communication control unit <NUM> receives data (packets) from the gate apparatuses <NUM>. In addition, the communication control unit <NUM> transmits data to the gate apparatuses <NUM>.

The user information registration unit <NUM> is means for acquiring gate user information and registering the acquired gate user information in the gate user database established in the storage unit <NUM>. The user information registration unit <NUM> can use any method to acquire the gate user information.

For example, an officer in charge at a passport center may register gate user information in the server apparatus <NUM>. Specifically, an officer in charge operates a scanner and reads out a photograph of a face and a page including an MRZ in a passport. The officer in charge operates a terminal (a computer installed at the passport center) and transmits the read data (image data) to the server apparatus <NUM>. Alternatively, the above data may be input to the server apparatus <NUM> via an external storage device, such as a USB (Universal Serial Bus) memory.

The above data (image data including the photograph of the face and the MRZ) may be input to the server apparatus <NUM> during a check-in procedure at the airport. Specifically, when an airline company staff member checks the passport of the user, the staff member reads out the photograph of the face and the page including the MRZ by using a scanner. The airline company staff member may transmit the read data from a terminal to the server apparatus <NUM>.

Alternatively, the user may use a so-called automatic check-in machine, to read the photograph of the face and the page including the MRZ by himself or herself. In this case, the data read from the automatic check-in machine is transmitted to the server apparatus <NUM>.

The user may capture the photograph of the face and the page including the MRZ in the passport, to acquire image data including the photograph of the face and the MRZ. Specifically, the user uses a smartphone or the like to capture the photograph of the face and the page including the MRZ in the passport. Next, the captured image data is transmitted from the smartphone or the like to the server apparatus <NUM>.

Alternatively, the image data including the photograph of the face and the MRZ may be acquired from a database in which records of the users who have been abroad are stored. Specifically, if the photograph of the face and the MRZ have previously been acquired at a staffed emigration and immigration booth and registered in a database, the corresponding image data may be transmitted from the database to the server apparatus <NUM>.

The user information registration unit <NUM> may use any one of the above-described methods or a different method, to acquire the image data including the photograph of the face and the MRZ included in the passport of the user.

The user information registration unit <NUM> extracts feature points from the face image included in the acquired image data. Since an existing technique can be used to extract these feature points, detailed description of the extraction will be omitted. For example, the user information registration unit <NUM> extracts the eyes, nose, mouth, etc. as feature points from the face image. Next, the user information registration unit <NUM> calculates, as feature values, the location of the individual feature point and the distance between feature points and generates a feature vector formed by a plurality of feature values (vector information that features the face image).

The user information registration unit <NUM> extracts the MRZ information from the image data including the MRZ information. Specifically, the user information registration unit <NUM> uses an optical character recognition (OCR) technique to extract the MRZ information from the image data.

The user information registration unit <NUM> associates the generated feature vector and the extracted MRZ information with each other and registers the associated information in the gate user database.

<FIG> is a diagram illustrating an example of the gate user database. As illustrated in <FIG>, the feature vectors generated from face images are registered as biological information. In addition, MRZ information corresponding to the individual biological information (feature vectors) is registered in the gate user database.

The first matching unit <NUM> is means for processing matching requests transmitted by the gate apparatuses <NUM>. Specifically, the first matching unit <NUM> sets the biological information (the feature vector) included in a matching request as the matching target and performs matching processing between this biological information and the biological information registered in the gate user database.

More specifically, the first matching unit <NUM> sets a feature vector extracted from a matching request as the matching target and performs <NUM>-to-N matching (N will hereinafter represents a positive integer) between this feature vector and the plurality of feature vectors registered in the gate user database.

The first matching unit <NUM> calculates a similarity between the feature vector as the matching target and each of the plurality of feature vectors registered. For this similarity, the chi-squared distance, the Euclidean distance, or the like may be used. A large distance represents a lower similarity, and a smaller distance represents a higher similarity.

The first matching unit <NUM> searches the plurality of feature vectors registered in the gate user database for a feature vector whose similarity to the matching target feature vector is more than or equal to a predetermined value and whose similarity is the highest.

The first matching unit <NUM> reads out the MRZ information corresponding to the feature vector (biological information) obtained as a result of the <NUM>-to-N matching from the gate user database. The first matching unit <NUM> transmits the read MRZ information to the gate apparatus <NUM> that has transmitted the matching request (replies to the matching request).

For example, in the example in <FIG>, if the similarity between a feature vector FV acquired from a gate apparatus <NUM> and a feature vector FV1 stored in the gate user database is the highest and if the similarity is more than or equal to a predetermined value, MRZ01 corresponding to this feature vector FV1 is transmitted to the gate apparatus <NUM>.

If, as a result of the above <NUM>-to-N matching, the biological information that matches (substantially matches or is substantially similar to) the biological information included in the matching request is not registered in the gate user database, the first matching unit <NUM> notifies the gate apparatus <NUM> to that effect. For example, the first matching unit <NUM> sets "None" in the feature vector and transmits a reply to the gate apparatus <NUM>.

Alternatively, the first matching unit <NUM> may notify the gate apparatus <NUM> that the biological information included in the matching request is not included in the gate user database by not transmitting any reply to the gate apparatus <NUM>. In this case, the gate apparatus <NUM> that has transmitted the matching request recognizes that the biological information included in the matching request is not included in the gate user database by not receiving any reply from the server apparatus <NUM> within a predetermined period.

The storage unit <NUM> stores various kinds of information necessary for operations of the server apparatus <NUM>. In addition, as described above, the gate user database is established in the storage unit <NUM>.

<FIG> is a diagram illustrating an example of a hardware configuration of the server apparatus <NUM> according to the first example embodiment. The server apparatus <NUM> can be configured by an information processing apparatus (a so-called computer) and has a configuration illustrated as an example in <FIG>. For example, the server apparatus <NUM> includes a processor <NUM>, a memory <NUM>, an input-output interface <NUM>, a communication interface <NUM>, etc. The components such as the processor <NUM> are connected to an internal bus, etc. so that these components can communicate with each other.

The hardware configuration of the server apparatus <NUM> is not limited to the configuration illustrated in <FIG>. The server apparatus <NUM> may include hardware not illustrated or may be configured without the input-output interface <NUM> if desired. In addition, the number of components, such as the number of processors <NUM>, included in the server apparatus <NUM> is not limited to the example illustrated in <FIG>. For example, a plurality of processors <NUM> may be included in the server apparatus <NUM>.

For example, the processor <NUM> is a programmable device such as a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or a DSP (Digital Signal Processor). Alternatively, the processor <NUM> may be a device such as an FPGA (Field Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit). The processor <NUM> executes various kinds of programs including an operating system (OS).

The memory <NUM> is a RAM (Random Access Memory), a ROM (Read-Only Memory), an HDD (Hard Disk Drive), an SSD (Solid State Drive), or the like. The memory <NUM> stores an OS program, an application program, and various kinds of data.

The input-output interface <NUM> is an interface for a display apparatus and an input apparatus not illustrated. For example, the display apparatus is a liquid crystal display or the like. For example, the input apparatus is an apparatus, such as a keyboard or a mouse, which receives user operations.

The communication interface <NUM> is a circuit, a module, or the like for performing communication with other apparatuses. For example, the communication interface <NUM> includes a NIC (Network Interface Card) or the like.

The functions of the server apparatus <NUM> are realized by various kinds of processing modules. The processing modules are realized, for example, by causing the processor <NUM> to execute a program stored in the memory <NUM>. In addition, this program can be recorded in a computer-readable storage medium. The storage medium may be a non-transient (non-transitory) storage medium, such as a semiconductor memory, a hard disk, a magnetic recording medium, or an optical recording medium. That is, the present invention can be embodied as a computer program product. In addition, the above program may be updated by downloading a program via a network or by using a storage medium in which a program is stored. In addition, the above processing modules may be realized by semiconductor chips.

Next, the gate apparatus <NUM> according to the first example embodiment will be described.

<FIG> is a diagram illustrating an example of a hardware configuration of the gate apparatus <NUM> according to the first example embodiment. As illustrated in <FIG>, the gate apparatus <NUM> includes a processor <NUM>, a memory <NUM>, an input-output device <NUM>, a communication interface <NUM>, a camera device <NUM>, a reader <NUM>, and a gate <NUM>.

Since the processor <NUM>, the memory <NUM>, and the communication interface <NUM> may be equivalent to those of the server apparatus <NUM> described with reference to <FIG>, detailed description thereof will be omitted.

The input-output device <NUM> serves as a device (for example, a touch panel) for inputting information to the gate apparatus <NUM> and a device (for example, a liquid crystal monitor) for outputting information.

For example, the camera device <NUM> is a digital camera installed to capture an image of a person in front of (in the forward direction of) the gate apparatus <NUM>. The camera device <NUM> may be installed at any location. For example, the camera device <NUM> may be installed on the main body of the gate apparatus <NUM> or away from the gate apparatus <NUM>. As long as the camera device <NUM> can capture an image of a user approaching the gate apparatus <NUM> (in particular, the face of the user), the camera device <NUM> may be installed at any location.

The reader <NUM> is a device that reads out information from passports. The reader <NUM> has a function of accessing IC chips in passports and a function of scanning pages of passports placed. That is, the reader <NUM> has a function as a card reader that exchanges information (data) with IC chips in passports in a non-contact state and has a function as a passport reader (a scanner) that acquires images from passports. The reader <NUM> may be installed at any location. However, it is preferable that the reader <NUM> be installed at a location where users can easily bring their passports into contact with this reader <NUM>. While the present application will be described assuming that the reader <NUM> has a function as a card reader and a function as a passport reader, these two functions may be of course realized by different devices.

When a user passes the emigration and immigration examination, the gate <NUM> shifts from a closed state in which the gate <NUM> blocks passage of the user to an opened state in which passage of the user is permitted. The mechanism of the gate <NUM> is not limited to any particular mechanism. For example, the gate <NUM> is a flap gate that opens and closes a flap installed on one side or flaps installed on both sides of the passage or is a turnstile gate that rotate three bars.

<FIG> is a diagram illustrating an example of a processing configuration (processing modules) of the gate apparatus <NUM> according to the first example embodiment. As illustrated in <FIG>, the gate apparatus <NUM> includes a communication control unit <NUM>, a biological information acquisition unit <NUM>, a matching request unit <NUM>, a passport possession determination unit <NUM>, a gate control unit <NUM>, and a storage unit <NUM>.

As with the communication control unit <NUM> of the server apparatus <NUM>, the communication control unit <NUM> is means for controlling communication with other apparatuses.

The biological information acquisition unit <NUM> is means for acquiring biological information about users who wish to pass through the gate apparatus <NUM>. For example, the biological information acquisition unit <NUM> controls the camera device <NUM> and acquires a face image of a person walking toward the camera device <NUM> (the gate apparatus <NUM>). For example, when the biological information acquisition unit <NUM> detects a user's face in an image being constantly or regularly captured, the biological information acquisition unit <NUM> captures the user's face and acquires the image of the face.

If the size of the face area included in the captured image is more than or equal to a predetermined value, the biological information acquisition unit <NUM> may extract the face area from the image. In this case, by appropriately adjusting the predetermined value (a threshold), the biological information acquisition unit <NUM> can acquire the face image of the user at a location a predetermined distance away from the gate apparatus <NUM>. That is, it is possible to prevent capturing an image of the user at a location far away from the gate apparatus <NUM> and prevent transmitting biological information inappropriate for face authentication (<NUM>-to-<NUM> matching by the server apparatus <NUM>) to the server apparatus <NUM>.

Alternatively, the biological information acquisition unit <NUM> may measure the distance between the gate terminal <NUM> and a user by using a distance sensor or the like and may acquire a face image of a user at a predetermined location.

The biological information acquisition unit <NUM> gives the acquired face image to the matching request unit <NUM>.

The matching request unit <NUM> is means for requesting the server apparatus <NUM> to perform matching to determine whether the biological information about the person corresponding to the acquired face image is registered in the gate user database. Specifically, the matching request unit <NUM> calculates feature values from the acquire face image and generates a feature vector from the plurality of feature values (generates a feature vector corresponding to the biological information registered in the server apparatus <NUM>).

The matching request unit <NUM> generates a matching request including the generated feature vector (biological information) and transmits the matching request to the server apparatus <NUM>.

For example, the matching request unit <NUM> generates a matching request including an identifier of the corresponding gate apparatus <NUM> (hereinafter referred to as a gate identifier), a feature vector, etc. (see <FIG>). A MAC (Media Access Control) address or an IP (Internet Protocol) address of the gate apparatus <NUM> may be used as the gate identifier.

The matching request unit <NUM> receives a reply to the matching request from the server apparatus <NUM>. The matching request unit <NUM> gives the reply (MRZ information corresponding to the feature vector determined by the <NUM>-to-N matching) from the server apparatus <NUM> to the passport possession determination unit <NUM>.

The passport possession determination unit <NUM> is means for determining whether the user who wishes to pass through the gate apparatus <NUM> possesses a correct passport. Specifically, the passport possession determination unit <NUM> attempts to read out information from an IC chip in the passport held over the reader <NUM> by using the MRZ information acquired from the matching request unit <NUM>.

As described in "Measures for Safety of IC Passports" in the following reference document <NUM>, information stored in an IC chip in a passport is encrypted (converted) by using MRZ information written in the same passport as a password.

https://www. jp/mofaj/toko/passport/ic_faq. html#<NUM>.

If the passport possession determination unit <NUM> succeeds in reading out information from the IC chip by using the MRZ information acquired from the server apparatus <NUM> (succeeds in decrypting the information), the passport possession determination unit <NUM> determines that the user who wishes to pass through the gate apparatus <NUM> possesses a correct passport.

In contrast, if the passport possession determination unit <NUM> fails to read out information from the IC chip by using the acquired MRZ information from the server apparatus <NUM>, the passport possession determination unit <NUM> determines that the user who wishes to pass through the gate apparatus <NUM> does not possess a correct passport.

In this way, if the passport possession determination unit <NUM> succeeds in reading out information from the IC chip by using the MRZ information (succeeds in decrypting the encrypted information), the passport possession determination unit <NUM> determines that the user possesses a correct passport. Thus, the passport possession determination unit <NUM> may check or may not check the content of the information read out from the IC chip.

If the passport possession determination unit <NUM> determines that the user who wishes to pass through the gate apparatus <NUM> possesses a correct passport, the passport possession determination unit <NUM> instructs the gate control unit <NUM> to "open the gate".

If the passport possession determination unit <NUM> determines that the user who wishes to pass through the gate apparatus <NUM> does not possess a correct passport, the passport possession determination unit <NUM> does not perform any particular operation. That is, if the passport possession determination unit <NUM> determines that the user who wishes to pass through the gate apparatus <NUM> does not possess a correct passport, the gate <NUM> does not open the gate.

The gate control unit <NUM> is means for controlling the gate <NUM> of the gate apparatus <NUM>. Specifically, the gate control unit <NUM> controls opening and closing of the gate <NUM> in accordance with an instruction from the passport possession determination unit <NUM>. If the gate control unit <NUM> determines, by using a distance sensor or the like, that a user who possesses a correct passport and who is permitted to pass through the gate <NUM> has passed through the gate <NUM>, the gate control unit <NUM> closes the gate <NUM>.

Next, an operation in the emigration and immigration examination system according to the first example embodiment will be described.

<FIG> is a sequence diagram illustrating an example of an operation in an emigration and immigration system according to the first example embodiment. <FIG> is a sequence diagram illustrating an example of a system operation on the departure date of a user. The following description assumes that, prior to this operation in <FIG>, the "gate user information" about the user is previously registered in the server apparatus <NUM>.

When a user approaches the gate apparatus <NUM>, the gate apparatus <NUM> acquires biological information from the user (step S01). For example, the gate apparatus <NUM> acquires a face image of the user.

When the gate apparatus <NUM> successfully acquires a face image, the gate apparatus <NUM> transmits a matching request including the biological information about the user to the server apparatus <NUM> (step S02). Specifically, the gate apparatus <NUM> calculates a feature vector from the face image and transmits a matching request including the calculated feature vector to the server apparatus <NUM>.

Upon acquiring the matching request, the server apparatus <NUM> extracts the biological information (the feature vector) from the matching request and performs <NUM>-to-N matching between the feature vector and the biological information registered in the gate user database (step S03). The server apparatus <NUM> searches the plurality of items of biological information registered in the gate user database for biological information whose similarity to the biological information acquired from the gate apparatus <NUM> is the highest and whose similarity is more than or equal to a predetermined value.

The server apparatus <NUM> transmits a reply including the MRZ information determined as a result of the above <NUM>-to-N matching to the gate apparatus <NUM> (the gate apparatus <NUM> that has transmitted the matching request) (step S04).

The gate apparatus <NUM> attempts to read out information from an IC chip in the passport held over the reader <NUM> by using the received MRZ information (step S05). If the gate apparatus <NUM> fails to read out information from the IC chip (No in step S05), the gate apparatus <NUM> performs no particular processing. In this case, the gate apparatus <NUM> maintains the gate <NUM> in its closed state.

If the gate apparatus <NUM> succeeds in reading out information from the IC chip (Yes in step S05), the gate apparatus <NUM> opens the gate <NUM> (step S06).

As described above, the gate apparatus <NUM> acquires biological information about an examination target user and transmits the biological information about the examination target user to the server apparatus <NUM>. The server apparatus <NUM> searches the gate user information for MRZ information corresponding to the received biological information and transmits the determined MRZ information to the gate apparatus <NUM>. If the gate apparatus <NUM> succeeds in reading out information from an IC chip in the passport by using the received MRZ information, the gate apparatus <NUM> controls the gate so that the examination target user can pass through the gate.

The gate apparatus <NUM> opens the gate <NUM> only when a user who has previously registered gate user information in the server apparatus <NUM> presents his or her correct passport to the gate apparatus <NUM>. In other words, in the following case, the gate apparatus <NUM> does not open the gate <NUM>, and the emigration and immigration examination by the gate apparatus <NUM> does not end successfully.

For example, if a user who has not registered gate user information in the server apparatus <NUM> presents his or her own passport to the gate apparatus <NUM>, since corresponding MRZ information cannot be acquired from the server apparatus <NUM>, the gate <NUM> does not open. In this case, the gate apparatus <NUM> may display a relevant message (a message indicating that necessary information is not registered in the server apparatus), requesting the user to move to a manned emigration and immigration examination place.

Even if a user has registered gate user information in the server apparatus <NUM>, if the user tries to pass through the gate <NUM> without bringing his or her passport into contact with the reader <NUM> of the gate apparatus <NUM>, the gate <NUM> does not open. This is because the gate apparatus <NUM> is an apparatus that automatically performs the emigration and immigration examination and has an important function of checking whether the users possess correct passports. That is, even if a user has previously registered gate user information in the server apparatus <NUM>, unless the user's possession of a passport in which the gate user information is registered is not confirmed, the gate <NUM> does not open. In this case, for example, the gate apparatus <NUM> may display a message or output an audio guidance for requesting the user to bring his or her correct passport (his or her own passport) into contact with the reader <NUM>.

In addition, even if a user brings a passport into contact with the reader <NUM>, there are cases in which the gate apparatus <NUM> cannot read out MRZ information. For example, the gate apparatus <NUM> may not be able to read out MRZ information when the passport is stored in a case or the like. If this happens, the gate apparatus <NUM> may display a message requesting the user to extract the passport from the case or the like and to bring the passport into contact with the reader <NUM>. Alternatively, the gate apparatus <NUM> may display a message requesting the user to open the passport and to bring the passport into contact with the reader <NUM>.

In addition, if a user presents someone else's passport, instead of his or her own passport, to the gate apparatus <NUM>, the gate <NUM> does not open, either. For example, even if a parent and his or her child have previously registered their gate user information in the server apparatus <NUM>, if the parent presents his or her child's passport to the gate apparatus <NUM>, the gate <NUM> does not open. In this case, for example, the gate apparatus <NUM> may display a message or output an audio guidance requesting the user to bring his or her correct passport (his or her own passport) into contact with the reader <NUM>.

As described above, in the emigration and immigration examination system according to the first example embodiment, a user who wishes to use the gate apparatus <NUM> registers biological information in the server apparatus <NUM> in advance. After the registration, when the user appears in front of the gate apparatus <NUM> on the departure date, the gate apparatus <NUM> acquires biological information about the user and queries the server apparatus <NUM> about whether the acquired biological information is registered in the server apparatus <NUM> (requests the server apparatus <NUM> to perform <NUM>-to-N matching by using the biological information). As a result of the matching, if the gate apparatus <NUM> determines that the biological information about the user is registered in the server apparatus <NUM>, the gate apparatus <NUM> determines whether the passport presented by the user is truly the passport of the user. Specifically, the gate apparatus <NUM> determines whether information can be read out from an IC chip in the passport of the user by using the MRZ information determined as a result of the face authentication by the server apparatus <NUM>. As described above, to extract information from the IC chip, the MRZ information written in the same passport is needed. Thus, the fact that the gate apparatus <NUM> succeeds in reading information from the IC chip by using the MRZ information indicates that the MRZ information acquired from the server apparatus <NUM> matches the MRZ information in the passport presented by the user. Based on this matching, the gate apparatus <NUM> determines that the user in front of the gate apparatus <NUM> possesses a correct passport and opens the gate (the user passes the emigration and immigration examination). In this way, according to the first example embodiment, if users have previously registered information in the database, these users can complete the emigration and immigration examination by bringing their passports into contact with the gate apparatus <NUM> without opening their passports. As a result, it is possible to realize convenient and smooth emigration and immigration examination for users.

Next, a second example embodiment will be described in detail with reference to drawings.

The first example embodiment assumes that there is no inconsistency (incoherence) between biological information and MRZ information in gate user information registered in the server apparatus <NUM>. When an officer of a public agency registers gate user information, it is hard to imagine that the above inconsistency occurs. However, when a user registers gate user information in the server apparatus <NUM>, there is a slight possibility that the above inconsistency occurs.

If the above inconsistency occurs, correct emigration and immigration examination cannot be performed, thereby causing a problem. According to the second example embodiment, more reliable emigration and immigration examination can be realized by causing a gate apparatus <NUM> to perform matching processing, too.

According to the second example embodiment, the emigration and immigration system may have the same schematic configuration and the server apparatus <NUM> may have the same processing configuration as those according to the first example embodiment. Thus, description corresponding to <FIG>, <FIG>, etc. will be omitted. The following description will be made with a focus on the difference between the first and second example embodiments.

<FIG> is a diagram for illustrating an example of a processing configuration (processing modules) of a gate apparatus <NUM> according to the second example embodiment. As illustrated in <FIG>, the gate apparatus <NUM> includes a second matching unit <NUM> in addition to the components illustrated in <FIG>.

The second matching unit <NUM> is means for performing matching (<NUM>-to-<NUM> matching) to determine whether biological information about a user matches biological information read out from an IC chip in a passport that the user has brought into contact with the reader <NUM>.

The biological information acquisition unit <NUM> stores acquired biological information (for example, a user's face image captured by the camera device <NUM>) in the storage unit <NUM>.

If the passport possession determination unit <NUM> succeeds in reading out information from an IC chip by using MRZ information acquired from the server apparatus <NUM>, the passport possession determination unit <NUM> gives a face image included in the read information to the second matching unit <NUM>.

The second matching unit <NUM> performs matching (<NUM>-to-<NUM> matching) between the face image acquired from the passport possession determination unit <NUM> and the face image stored in the storage unit <NUM>. Specifically, the second matching unit <NUM> calculates a feature vector from each of the two images. Next, the second matching unit <NUM> calculates the similarity (for example, a Euclidean distance) between these two images and determines whether the two images indicate a face image of the same person, based on a result of threshold processing on the calculated similarity. For example, if the similarity is over a predetermined value (if the distance is shorter than a predetermined value), the second matching unit <NUM> determines that these two face images indicate the same person.

If the second matching unit <NUM> succeeds in the <NUM>-to-<NUM> matching (if the face image of the user in front of this gate apparatus <NUM> matches the face image read out from the IC chip in the presented passport), the gate apparatus <NUM> instructs the gate control unit <NUM> to open the gate.

As described above, if the gate apparatus <NUM> succeeds in the matching between the biological information acquired from the user and the biological information read out from the IC chip presented by the user, the gate apparatus <NUM> opens the gate <NUM> so that the user can pass through the gate <NUM>. That is, if the gate apparatus <NUM> determines that the distance between the feature vectors obtained from the above two items of biological information is shorter than a predetermined distance and that the two items of biological information are substantially the same, the gate apparatus <NUM> controls the gate <NUM> so that the user can pass through the gate <NUM>.

As described above, according to the second example embodiment, <NUM>-to-<NUM> matching between a face image of a user obtained from the camera device <NUM> attached to the gate apparatus <NUM> and a face image read out from an IC chip in a passport presented by the user is performed. As a result of this <NUM>-to-<NUM> matching, if the gate apparatus <NUM> succeeds in authenticating the user, the gate apparatus <NUM> opens the gate <NUM>. That is, according to the first example embodiment, a problem is caused if there is an inconsistency in gate user information registered in the server apparatus <NUM>. However, according to the second example embodiment, since the gate apparatus <NUM> performs final authentication processing on users, users who are not supposed to pass through the gate apparatus <NUM> are prevented from passing through the gate apparatus <NUM>. According to the second example embodiment, after the server apparatus <NUM> performs <NUM>-to-N matching, the gate apparatus <NUM> performs <NUM>-to-<NUM> matching. In this way, it is guaranteed that the users who pass through the gate apparatus <NUM> possess their correct passports (their own passports).

The configurations, operations, etc. of the emigration and immigration examination systems according to the above example embodiments have been described as examples, and the above description is not intended to limit the system configurations, etc..

For example, the functions of the server apparatus <NUM> may entirely or partially be realized by the gate apparatus <NUM>. Alternatively, the gate user database of the server apparatus <NUM> may be established in a different database server. The emigration and immigration examination system may be configured in different ways as long as the emigration and immigration examination system includes, for example, storage means (the storage unit <NUM>) for storing the gate user information, matching means (the first matching unit <NUM>) for performing <NUM>-to-N matching by using the gate user information, and determination means (the passport possession determination unit <NUM>) for determining whether the users possess their passports.

In the above example embodiments, transmission and reception of a feature vector calculated from a face image is performed between the gate apparatus <NUM> and the server apparatus <NUM>. However, a face image may be transmitted from the gate apparatus <NUM> to the server apparatus <NUM>. In this case, a matching request including a face image is transmitted from the gate apparatus <NUM> to the server apparatus <NUM>, and the server apparatus <NUM> calculates a feature vector from the received face image and performs <NUM>-to-N matching. Alternatively, a face image, not a feature vector, may be registered in the user information database of the server apparatus <NUM>, and feature values may be calculated each time matching is performed.

The above example embodiments assume that the gate apparatuses <NUM> are placed away from each other and do not assume that a user moving to one gate apparatus <NUM> presents a passport to another gate apparatus <NUM> different from the one gate apparatus <NUM>. However, as illustrated in <FIG>, there are cases where a plurality of users wait in a line and a user walks to a gate apparatus <NUM> that has become available upon completion of the examination on a previous user. In this case, although a gate apparatus <NUM> captures and acquires a face image of a user in a line, this user could finally present his or her passport to a different gate apparatus <NUM>. To solve this problem, the server apparatus <NUM> transmits a reply to a matching request (a reply including MRZ information) not only to the gate apparatus <NUM>, which has transmitted the matching request, but also to the other gate apparatuses <NUM>. For example, <FIG> assumes a case in which a gate apparatus <NUM>-<NUM> captures an image of a user in a line and transmits a matching request including biological information about the user. In this case, the server apparatus <NUM> transmits MRZ information obtained as a result of the <NUM>-to-N matching not only to the gate apparatus <NUM>-<NUM>, which has transmitted the matching request, but also to a gate apparatus <NUM>-<NUM> and a gate apparatus <NUM>-<NUM>. As a result, as illustrated in <FIG>, even when the user walks to the gate apparatus <NUM>-<NUM>, not the gate apparatus <NUM>-<NUM>, the gate apparatus <NUM>-<NUM> performs the emigration and immigration examination on the user (can determine whether reading of information from an IC chip by using the MRZ information has succeeded). This is because the gate apparatus <NUM>-<NUM> possesses the MRZ information corresponding to the passport of the user.

The example embodiments assume that a single image includes a single face area. However, depending on the situation of the examination place, a single image could include a plurality of face areas. In this case, the server apparatus <NUM> may extract a face area closest thereto and may transmit the corresponding biological information to the server apparatus <NUM>. For example, the server apparatus <NUM> may calculate the area of the individual face area and extract a face area having the largest area. Alternatively, for example, a depth camera that can measure the distance in the depth direction may be adopted as the camera device <NUM>. In this case, the camera device <NUM> may measure the distance therefrom and may extract a face image of a person closest thereto.

In addition, while the present application discloses a technique for smoothly completing emigration and immigration examination by allowing users to bring their passports into contact with the card reader <NUM> without opening the passports, the present application does not exclude opening of the passports by the users. For example, if a predetermined condition is met, a user may be requested to open his or her passport, and the examination may be performed on the opened passport. For example, when a result of the matching by the server apparatus <NUM> is relatively low (when the similarity is lower than a predetermined threshold), the server apparatus <NUM> notifies the gate apparatus <NUM> to that effect (that the matching result is relatively low) in addition to the MRZ information. Upon receiving the notification, the gate apparatus <NUM> may instruct the user to open the passport and place the page including a face photograph and MRZ on the reader <NUM>. The gate apparatus <NUM> acquires biological information (face image) about the user (the user in front of the gate apparatus <NUM>) and performs <NUM>-to-<NUM> matching by using this biological information and the face image obtained from an IC chip in the passport. As a result of the matching, if the gate apparatus <NUM> determines that the user possesses a correct passport, the gate apparatus <NUM> may open the gate <NUM>.

Alternatively, if the gate apparatus <NUM> cannot read out information from an IC chip in a passport by using MRZ information received from the server apparatus <NUM>, the gate apparatus <NUM> may request the examination target user to open and place the passport on the reader <NUM>. In this case, as a result of the matching, if the gate apparatus <NUM> determines that the user possesses a correct passport, the gate apparatus <NUM> opens the gate <NUM>.

The gate apparatus <NUM> may change its operation, depending on the attributes of the individual user. For example, in the emigration and immigration examination, while fingerprints do not need to be collected from people who have Japanese nationality, fingerprints need to be collected from people who do not have Japanese nationality. Thus, the gate apparatus <NUM> may determine the nationality of the individual user (Japanese, non-Japanese) based on the MRZ information acquired from the server apparatus <NUM> and may change the control processing including the gate opening/closing operation based on the determination result. For example, regarding the emigration and immigration examination for the non-Japanese, the gate apparatus <NUM> may open the gate <NUM> after collecting fingerprints and succeeding in the examination on the collected fingerprint.

In the above example embodiments, while the gate apparatuses <NUM> are installed at a single airport (a departure airport or an arrival airport), the gate apparatuses <NUM> may be installed at a departure airport and an arrival airport (see <FIG>). Users whose gate user information has already been registered in the server apparatus <NUM> do not need to register their gate user information again and can enjoy smooth emigration and immigration examination based on the gate apparatuses <NUM> according to the present application.

Alternatively, as illustrated in <FIG>, when the gate apparatuses <NUM> according to the present application are installed at a departure airport and an arrival airport, the gate user information about users registered in the server apparatus <NUM> may be deleted after their flight. The biological information and the MRZ information including the name, etc. registered in the server apparatus <NUM> are personal information, and it is not desirable to hold such personal information for a long time. In this case, when a gate apparatus <NUM> at the arrival airport (a gate apparatus <NUM>-<NUM> in the example in <FIG>) notifies the server apparatus <NUM> that the immigration examination has successfully been completed on a user, the gate user information about this user is deleted. Alternatively, if a gate apparatus <NUM> at a departure airport (a gate apparatus <NUM>-<NUM> in the example in <FIG>) has successfully completed the emigration examination on a user, the gate apparatus <NUM> may notify the gate apparatus <NUM>-<NUM> at the arrival airport to that effect. In this case, when the gate apparatus <NUM>-<NUM> completes the immigration examination on the user, the gate apparatus <NUM>-<NUM> may request the server apparatus <NUM> to delete the corresponding gate user information.

In the above example embodiment, operations of the gate apparatuses <NUM> and the server apparatus <NUM> have been described assuming that "faces" are mainly used as the biological information. However, the biological information that can be used in the present application is not limited to "faces". Other biological information such as irises may be used.

The data transmission and reception modes among the gate apparatuses <NUM> and the server apparatus <NUM> are not limited to any particular modes, and the data transmitted and received among these apparatuses may be encrypted. The face images and the feature values calculated from these face images are personal information, and to appropriately protect the personal information, it is desirable that encrypted data be transmitted and received.

The above example embodiments have been described based on a case where information is acquired from IC chips in passports by using MRZ information. However, the technique according to the present application is also applicable to other methods. That is, information about a password or a key corresponding to MRZ information may be used to read out information from an IC chip mounted on a medium such as a card similar to a passport.

In the flowcharts and sequence diagrams used in the above description, a plurality of steps (processes) are sequentially described. However, the order of the execution of the steps performed in the individual example embodiment is not limited to the described order. In the individual example embodiment, the order of the illustrated steps may be changed to the extent that a problem is not caused on the content of the individual example embodiment. For example, individual processes may be executed in parallel.

The above example embodiments have been described in detail to facilitate the understanding of the present application disclosed and not to mean that all the configurations described above are needed. In addition, if a plurality of example embodiments have been described, each of the example embodiments may be used individually or a plurality of example embodiments may be used in combination. For example, part of a configuration according to one example embodiment may be replaced by a configuration according to another example embodiment. For example, a configuration according to one example embodiment may be added to a configuration according to another example embodiment. In addition, addition, deletion, or replacement is possible between part of a configuration according to one example embodiment and another configuration.

Claim 1:
An emigration and immigration examination system comprising:
a server apparatus (<NUM>) configured to store, as gate user information, biological information comprising face images about users and Machine Readable Zone, MRZ, information written in machine readable zones in passports issued to the users in association with each other; and
a plurality of gate apparatuses (<NUM>-<NUM>, <NUM>-<NUM>, <NUM>-<NUM>) each configured to control its gate based on an emigration and immigration examination result of a user,
wherein each of the plurality of gate apparatuses (<NUM>-<NUM>, <NUM>-<NUM>, <NUM>-<NUM>) comprises a reader (<NUM>) configured to read out information from an Integrated Circuit, IC chip in a passport and wherein each of the plurality of gate apparatuses is further configured to capture and acquire biological information comprising face images of a user in a line,
wherein a first gate apparatus (<NUM>-<NUM>) of the plurality of the gate apparatuses (<NUM>-<NUM>, <NUM>-<NUM>, <NUM>-<NUM>) is configured to acquire biological information comprising face images about an examination target user in a line and to transmit the biological information about the examination target user to the server apparatus (<NUM>),
wherein the server apparatus (<NUM>) is configured to search the gate user information to determine the MRZ information corresponding to the received biological information and to transmit the determined MRZ information to each of the plurality of the gate apparatuses (<NUM>-<NUM>, <NUM>-<NUM>, <NUM>-<NUM>), and
wherein, a second gate apparatus (<NUM>-<NUM>) of the plurality of the gate apparatuses (<NUM>-<NUM>, <NUM>-<NUM>, <NUM>-<NUM>) is configured to control its gate so that the examination target user can pass through the gate if information is successfully read out by its reader (<NUM>) from an IC chip in a passport issued to the examination target user by using the received MRZ information.