Patent Description:
The present disclosure relates to a determination apparatus, an imaging apparatus, a driver confirmation system, a moveable body, and a determination method.

A driver confirmation system that can unlock a vehicle door, start the engine, and the like only for a legitimate driver has been proposed for apparatuses such as vehicles. For example, see patent literature (PTL) <NUM>. <CIT> discloses a vehicular anti-theft system comprising data acquisition units provided on a vehicle, a data generation unit for generating the driver data according to the outputs of the data acquisition units, a data maintaining unit for maintaining the data corresponding the driver data regarding an owner of the vehicle, an information processing unit for determining permission or non-permission of the drive of the vehicle by comparing the driver data with the data in the data maintaining unit, a drive restriction unit for restricting the drive of the vehicle in an impossible state based on the determination of the information processing unit, a transmission unit for transmitting the signal to the owner of the vehicle or the like when non-permission is determined by the information processing unit, and a control unit for releasing the drive restriction unit if the information processing unit determines permission or according to the predetermined input operation by the owner of the vehicle or the like. <CIT> discloses a similarity degree calculating unit calculating a similarity degree of a feature quantity of a face image of an imaged checking target, and a feature quantity of a face image of a registrant or a non-registrant registered in a registrant information DB or a non-registrant information DB. A check determination unit notifies a service determination unit when the similarity degree is higher than a threshold value. A related information acquiring unit acquires related information including inter-registrant information, and provides to a time-series information DB generation unit and the service determination unit. The time-series information DB generation unit updates the time-series information DB. A statistical information DB generation unit generates the statistical information DB from the time-series information DB. The service determination unit provides services based on the related information, the time-series information DB, and the statistical information DB.

The present invention provides a determination apparatus according to claim <NUM>, an imaging apparatus according to claim <NUM>, a driver confirmation system according to claim <NUM>, a moveable body according to claim <NUM>, and a determination method according to claim <NUM>.

An example driver confirmation system for suspending use of a vehicle other than by a legitimate user stores an image of the driver's license of the legitimate driver in a storage apparatus. At the time of driving, the driver confirmation system compares an image of a driver's license in the driver's possession with the image of the driver's license stored in the storage means and permits the user to operate the vehicle when the image of the driver's license in the driver's possession matches the registered image of the driver's license. In contrast with this example, embodiments of the present disclosure are described below.

Embodiments of the present disclosure are described below with reference to the drawings. The drawings referred to below are schematic illustrations. The dimensional ratios and the like in the drawings do not necessarily match the actual dimensions.

With reference to <FIG>, the configuration of a driver confirmation system according to an embodiment is described. The driver confirmation system <NUM> authenticates whether the driver of a particular apparatus is a legitimate driver permitted to drive the apparatus. The particular apparatus may, for example, be a moveable body, a construction machine, an apparatus for operating a factory, a robot controlled by the driver, or the like. Examples of moveable bodies include vehicles, such as automobiles, buses, and trucks; aircraft, helicopters, airships, marine vessels, submarines, personal watercraft, snowmobiles, and the like. The driver confirmation system <NUM> includes a determination apparatus <NUM>, a reading apparatus <NUM>, and a camera <NUM>.

The reading apparatus <NUM> reads a facial photograph image as a first facial image from a medium <NUM> that holds the facial photograph image. The medium <NUM> is a license, a permit, a certification, a qualification certificate, a membership card, an ID card, an electronic key with an internal memory, or the like that indicates permission for a particular individual to drive a particular apparatus. The medium <NUM> may be an individual identification medium for identifying an individual or an individual authentication medium for authenticating an individual. The medium <NUM> may include a memory and hold the facial photograph image in the memory as electronic data. The medium <NUM> may hold the facial photograph image as a photograph displayed on the surface of the medium <NUM>. The medium <NUM> may both display the facial photograph image on the surface of the medium <NUM> and store the facial photograph image in a memory. The medium <NUM> may have any of various shapes: a card shape, a prismatic shape, a cylindrical shape, a disk shape, or the like.

The reading apparatus <NUM> may, for example, be a reading apparatus that reads, as electronic data, the facial photograph image stored in the memory of the medium <NUM>. The reading apparatus <NUM> may include an input unit when authentication by a PIN, a password, or the like is necessary when reading the facial photograph image stored in the memory of the medium <NUM>. The input unit may, for example, be a keyboard, a touch panel, or the like for input of the PIN or password. The reading apparatus <NUM> may also include a scanner that optically reads the content displayed on the surface of the medium <NUM>, including the facial photograph image. The reading apparatus <NUM> may extract the facial photograph image from the optically read image and convert the facial photograph image to electronic data.

The camera <NUM> can capture an image of an imaging area, in which the face of the driver of the apparatus to be driven is assumed to be located, at the start of driving and/or during driving. The imaging area in which the face of the driver is assumed to be located is a wider area than the face of a standard driver. The imaging area in which the face of the driver is assumed to be located can be set to include the position of the face when a driver of average build operates the apparatus to be driven while adopting standard posture. The "start of driving" in the present disclosure refers to when the apparatus to be driven is activated and/or operated in preparation for activation. "Preparation for activation" may, for example, include any of adjusting the seat position, adjusting the steering position, adjusting the mirror angle, operating a brake pedal, and operating a clutch pedal. "During driving" refers to when the apparatus is being driven continuously.

The camera <NUM> can include an optical system <NUM> including a lens and the like, an image sensor <NUM>, and a signal processing circuit <NUM>. The optical system <NUM> forms an image of a subject located in the imaging area on an imaging surface of the image sensor <NUM>. The optical system <NUM> may include one or more lenses. The image sensor <NUM> may, for example, be a charge-coupled device (CCD) image sensor or a complementary MOS (CMOS) image sensor. The signal processing circuit <NUM> generates an image signal representing the image formed by the image sensor <NUM>. The signal processing circuit <NUM> may perform any process on the image, such as distortion correction, brightness adjustment, contrast adjustment, and/or gamma correction.

The determination apparatus <NUM> acquires the first facial image from the reading apparatus <NUM>. The determination apparatus <NUM> extracts, from the captured image acquired from the camera <NUM>, an image of the face of the driver as a second facial image. The determination apparatus <NUM> compares the first facial image and the second facial image and determines whether the driver is the person in the facial photograph image held by the medium <NUM>. The determination apparatus <NUM> includes a first acquisition unit <NUM>, a second acquisition unit <NUM>, and a controller <NUM>. The determination apparatus <NUM> can include one or more of a memory <NUM>, an output unit <NUM>, an input unit <NUM>, and a communication unit <NUM> as necessary. The memory <NUM>, output unit <NUM>, input unit <NUM>, and communication unit <NUM> are not, however, essential components. The functions of the determination apparatus <NUM> are controlled by the controller <NUM>. Operations of the determination apparatus <NUM> may be synonymous with operations of the controller <NUM>.

The first acquisition unit <NUM> acquires the facial photograph image included in the medium <NUM> from the reading apparatus <NUM>. The second acquisition unit <NUM> receives an image signal from the camera <NUM> and acquires a captured image that includes the face of the driver and was captured at the start of driving and/or during driving of the apparatus to be driven. The first acquisition unit <NUM> can be embodied as an input interface corresponding to the transmission format of signals output by the reading apparatus <NUM>, and the second acquisition unit <NUM> can be embodied as an input interface corresponding to the transmission format of signals output by the camera <NUM>.

The controller <NUM> controls the determination apparatus <NUM> overall. The controller <NUM> includes one or more processors. The controller <NUM> or the processor may include one or more memories that store programs for various processing and store information during calculations. The term "memory" encompasses volatile memory and non-volatile memory. The memory may be configured as a memory independent from the processor and/or a memory built into the processor. The term "processor" encompasses universal processors that execute particular functions by reading particular programs and dedicated processors that are specialized for particular processing. Dedicated processors include application specific integrated circuits (ASICs) for specific applications. Processors also include programmable logic devices (PLDs). PLDs include field-programmable gate arrays (FPGAs). The controller <NUM> may be either a system-on-a-chip (SoC) or a system in a package (SiP) with one processor or a plurality of processors that work together.

The controller <NUM> may include the following functional blocks: a processing initiation unit <NUM>, a face extraction unit <NUM>, an identity determination unit <NUM>, and a driving restriction unit <NUM>. Each functional block may be a hardware module or a software module. The controller <NUM> can execute the operations performable by each functional block. The controller <NUM> may execute all of the operations of the functional blocks. The operations performed by each functional block may be synonymous with operations performed by the controller <NUM>. The controller <NUM> itself may execute the processing that the controller <NUM> causes any of the functional blocks to perform.

The processing initiation unit <NUM> initiates determination processing to determine whether the driver photographed by the camera <NUM> is the person in the facial photograph image held by the medium <NUM>. The processing initiation unit <NUM> initiates the determination processing before the driver starts driving the apparatus to be driven. The processing initiation unit <NUM> can initiate the determination processing on the basis of the signal from the reading apparatus <NUM> acquired by the first acquisition unit <NUM> and/or a signal from a detection apparatus <NUM> acquired by the input unit <NUM>. For example, the processing initiation unit <NUM> may initiate the determination processing when the reading apparatus <NUM> newly acquires a facial photograph image held by the medium <NUM>. When the apparatus to be driven is a vehicle, the processing initiation unit <NUM> may initiate the determination processing when the key of the vehicle is inserted in an insertion slot. The processing initiation unit <NUM> may initiate the determination processing when, for example, a possible switch in drivers during driving of the apparatus to be driven is detected from the detection apparatus <NUM> via the input unit <NUM>.

The face extraction unit <NUM> acquires a captured image captured by the camera <NUM> from the second acquisition unit <NUM> and extracts an image of the face of the driver as a second facial image. The face extraction unit <NUM> can use a variety of algorithms for face extraction. For example, an image of a face can be extracted by detecting facial components, such as the eyes, nose, and mouth, from the captured image. An image of a face can also be detected on the basis of the silhouette of body parts such as the head, neck, and shoulders.

The identity determination unit <NUM> compares the first facial image, which is the facial photograph image included in the medium <NUM> and acquired from the first acquisition unit <NUM>, and the second facial image extracted from the captured image acquired from the second acquisition unit <NUM>. By this comparison, the identity determination unit <NUM> determines whether the driver is the person in the facial photograph image held by the medium <NUM>. The determination can be made using various methods. For example, the identity determination unit <NUM> can calculate feature values related to the arrangement of the eyes, nose, and/or mouth from the images of the faces included in the first facial image and the second facial image and compare the feature values to determine whether they belong to the same person.

When it is not determined that the driver is the person in the facial photograph image held by the medium <NUM>, the driving restriction unit <NUM> can restrict driving of the apparatus. The driving restriction unit <NUM> outputs a signal, through the output unit <NUM>, to a control apparatus <NUM> of the apparatus to be driven so as to restrict driving of the apparatus. In the present disclosure, to "restrict" driving of the apparatus refers to limiting receipt of operations for driving, limiting uses of functions for driving, limiting performance when driving, and the like. The performance includes, for example, the maximum speed at which the apparatus can be driven. The driving restriction unit <NUM> can also output a signal, through the output unit <NUM>, to the control apparatus <NUM> of the apparatus to be driven so as to suspend driving. In the present disclosure, to "suspend" driving refers to making driving impossible. In the present disclosure, "restrict" encompasses "suspend". The driving restriction unit <NUM> may also display a warning on a display apparatus <NUM> through the output unit <NUM>.

The memory <NUM> can store the first facial image acquired from the first acquisition unit <NUM>. The first facial image is stored for comparison with the second facial image extracted from the captured image acquired from the second acquisition unit <NUM>. The memory <NUM> may, for example, include a semiconductor memory, a magnetic memory, an optical memory, or the like. The semiconductor memory may include volatile memory and non-volatile memory. When a memory built into the controller <NUM> is used to store the first facial image, the memory <NUM> need not be provided in the determination apparatus <NUM>. The memory <NUM> may store information other than the first facial image. For example, when the apparatus to be driven is a vehicle used for car sharing, the memory <NUM> may store identification information such as the name and address of the user of the vehicle.

The output unit <NUM> is an interface for output to the display apparatus <NUM>, the control apparatus <NUM>, or the like provided in the apparatus to be driven. A physical connector or a wireless communication device can be used in the output unit <NUM>. In an embodiment, the output unit <NUM> may be connected to a vehicle network, such as a control area network (CAN), when the apparatus to be driven is a vehicle. The determination apparatus <NUM> may be connected to the control apparatus <NUM> through the CAN.

When it cannot be determined that the driver is the person in the facial photograph image held by the medium <NUM>, the controller <NUM> can instruct the control apparatus <NUM> through the output unit <NUM> to restrict driving operations by the driver.

When the apparatus to be driven is a vehicle, restriction of driving operations includes suspension of driving operations. Suspension of driving operations may encompass disabling performance of a target operation, disabling a process for receiving a target operation, and disabling a process for operation execution after receipt of a target operation. Driving operations can be suspended by one or more methods such as not starting the engine, disabling operation of the shift knob, and disabling release of the parking brake. When the apparatus to be driven is a vehicle, not starting the engine encompasses disabling rotation of the ignition key to the start position and not starting the engine in conjunction with the immobilizer. Disabling operation of the shift knob encompasses not allowing the shift knob to move from the parking position, not outputting an operation signal of the shift knob when operation of the shift knob is received, and not shifting gears after operation of the shift knob is received. Disabling release of the parking brake encompasses fixing the operation lever of the parking brake to prevent release, deactivating or suspending activation of a parking brake release device, and not outputting a parking brake release signal after a parking brake release operation is received.

Restriction of driving operations is not limited to suspension of the aforementioned driving operations. For example, after starting the engine, the control apparatus <NUM> can restrict driving operations by not increasing the engine speed above a predetermined speed even when the accelerator is depressed. The control apparatus <NUM> can also restrict driving operations by disabling operation of an electric mirror.

Driving operations are restricted by cooperation between the controller <NUM> and the control apparatus <NUM>. Therefore, the present disclosure is not limited to the case of the controller <NUM> instructing the control apparatus <NUM> to restrict driving operations by the driver. In an embodiment, the control apparatus <NUM> may transition to a state capable of restricting driving operations when receiving, from the controller <NUM>, a signal indicating that it cannot be determined that the driver is the person in the facial photograph image held by the medium <NUM>.

When it cannot be determined that the driver is the person in the facial photograph image held by the medium <NUM>, the controller <NUM> can also display a warning on the display apparatus <NUM> through the output unit <NUM>. Specifically, the controller <NUM> can display a warning on the display apparatus <NUM> indicating that driving is not permitted, such as "Authentication error. You cannot drive". A variety of displays may be used as the display apparatus <NUM>, such as a liquid crystal display (LCD), an organic electro-luminescence (EL) display, an inorganic EL display, or a plasma display panel (PDP).

The input unit <NUM> is an interface for receiving input from various detection apparatuses <NUM> provided in the apparatus to be driven. A physical connector or a wireless communication device can be used in the input unit <NUM>. Physical connectors include an electrical connector corresponding to transmission by an electric signal, an optical connector corresponding to transmission by an optical signal, and an electromagnetic connector corresponding to transmission by an electromagnetic wave. Electrical connectors include connectors conforming to IEC <NUM>, connectors conforming to the USB standard, connectors comprising RCA terminals, connectors comprising S terminals prescribed by EIAJ CP-1211A, connectors comprising D terminals prescribed by EIAJ RC-<NUM>, connectors conforming to the HDMI° standard (HDMI is a registered trademark in Japan, other countries, or both), and connectors comprising a coaxial cable including a BNC connector. Optical connectors include a variety of connectors conforming to IEC <NUM>. Wireless communication devices include wireless communication devices conforming to standards that include Bluetooth® (Bluetooth is a registered trademark in Japan, other countries, or both) and IEEE802. The wireless communication device includes at least one antenna. When the apparatus to be driven is a vehicle, the input unit <NUM> may connect to a vehicle network such as a CAN. In an embodiment, the input unit <NUM> is a separate constituent element from the first acquisition unit <NUM> and the second acquisition unit <NUM>, but this configuration is not limiting. In an embodiment, the first acquisition unit <NUM>, the second acquisition unit <NUM>, and the input unit <NUM> may be implemented as one input unit. At least a portion of the first acquisition unit <NUM>, the second acquisition unit <NUM>, and the input unit <NUM> may be implemented along with the output unit <NUM> as one input/output unit.

The input unit <NUM> transmits the output of the detection apparatus <NUM> to the controller <NUM>. When the apparatus to be driven is a vehicle, the detection apparatus <NUM> may, for example, be a sensor for detecting operation of the ignition key, operation of the parking brake, operation of the shift knob, or the like. The processing initiation unit <NUM> of the controller <NUM> may execute the above-described determination processing when the driver performs at least one of these operations at the start of driving.

When the apparatus to be driven is a vehicle, the detection apparatus <NUM> may, for example, be a sensor for detecting that the driver is sitting in the driver's seat or a sensor for detecting attachment or detachment of the seatbelt. The detection apparatus <NUM> may also function as the camera <NUM>. In this case, the camera <NUM> may monitor the presence of the driver's head in the imaging area and output a signal when the head is not detected. The processing initiation unit <NUM> of the controller <NUM> may start the determination processing when the detection apparatus <NUM> detects a predetermined event. Detection of a predetermined event includes the case of the detection apparatus <NUM> detecting attachment or detachment of the driver's seat seatbelt, the case of the driver not being detected in images captured by the camera <NUM> for a predetermined time or longer, and the case of the detection apparatus <NUM> being a vehicle seat sensor and not detecting sitting by the driver for a predetermined time or longer.

The communication unit <NUM> is an interface for communicating with a destination external to the apparatus to be driven. When it cannot be determined that the driver is the person in the facial photograph image held by the medium <NUM>, the controller <NUM> can notify an external system <NUM> through the communication unit <NUM>. When the apparatus to be driven is a vehicle, the external system <NUM> can, for example, be a system of a security company or a rental car company. When it cannot be determined that the driver photographed by the camera <NUM> is the person in the facial photograph image held by the medium <NUM>, the controller <NUM> notifies the external system <NUM> that someone other than the legitimate user is attempting to drive the vehicle. In this case, the controller <NUM> may also transmit the captured image of the driver's face photographed by the camera <NUM> to the external system <NUM>.

The driver confirmation system <NUM> according to an embodiment can be mounted in a moveable body, in particular a vehicle, that can be operated by a driver. The term "vehicle" in the present disclosure includes, but is not limited to, automobiles, railway vehicles, industrial vehicles, and vehicles for daily life. For example, the term "vehicle" may include airplanes that travel down a runway. Automobiles include, but are not limited to, passenger vehicles, trucks, buses, motorcycles, and trolley buses, and may include other vehicles that travel on the road. Railway vehicles include, but are not limited to, locomotives, streetcars, guided railway vehicles, linear motor cars, and monorails, and may include other vehicles that travel along a track. Industrial vehicles include industrial vehicles for agriculture and for construction. Industrial vehicles include, but are not limited to, forklifts and golf carts. Industrial vehicles for agriculture include, but are not limited to, tractors, cultivators, transplanters, binders, combines, and lawnmowers. Industrial vehicles for construction include, but are not limited to, bulldozers, scrapers, backhoes, cranes, dump cars, and road rollers. Vehicles for daily life include, but are not limited to, bicycles and motorized, two-wheeled standing vehicles. Power engines for the vehicle include, but are not limited to, internal-combustion engines including diesel engines, gasoline engines, and hydrogen engines, and electrical engines including motors. The vehicle is not limited to the above-listed types. For example, automobiles may include industrial vehicles that can drive on the road, and the same vehicle may be included in multiple categories.

With reference to <FIG>, the configuration of a driver confirmation system <NUM> mounted in a vehicle according to an embodiment is described. <FIG> illustrates a portion of the constituent elements in <FIG>. However, the constituent elements that are illustrated in <FIG> but not in <FIG> can also be mounted in the vehicle. Each constituent element in <FIG> can also be mounted in the driver confirmation system <NUM> that is mounted in the vehicle.

In <FIG>, the determination apparatus <NUM> is disposed inside the dashboard of the vehicle. The determination apparatus <NUM> is not limited to being disposed at this position and may be disposed at another location in the vehicle. The reading apparatus <NUM> can be installed inside or on top of the dashboard. Alternatively, the reading apparatus <NUM> can be disposed at any other position inside the vehicle cabin near the driver's seat.

A driver's license can be used as an example of the medium <NUM> used in the driver confirmation system <NUM> mounted in the vehicle. A facial photograph image of the license holder is displayed on the driver's license. A contactless integrated circuit (IC) chip is embedded in Japanese driver's licenses. Electronic data of the facial photograph image, and of information such as the license holder's name and the type of license, is recorded on this IC chip. This electronic data can be read by using a reading apparatus <NUM> compatible with driver's licenses. Apart from contactless media, contact media such as a contact IC card can also be used as the medium <NUM> that holds electronic data.

<FIG> illustrates an example of a medium 13a assumed to be the surface of a driver's license. A facial photograph <NUM> and driver information <NUM> are displayed on the surface of the medium <NUM>. The driver information <NUM> is information such as the name and address of the driver and the type of driver's license. The reading apparatus <NUM> may optically scan the facial photograph <NUM> on the medium <NUM> to acquire the facial photograph image. Along with the facial photograph image, the reading apparatus <NUM> may also acquire the driver information <NUM> using optical character recognition.

<FIG> illustrates an example of a medium 13b assumed to be a driver's license in which a contactless IC chip is embedded. The medium 13b includes a central processing unit (CPU) <NUM>, a memory <NUM>, a communication interface <NUM>, and a power supply unit <NUM>. The IC chip includes the CPU <NUM> and the memory <NUM>. The memory <NUM> stores electronic data of the facial photograph image along with the driver information <NUM>, such as the name of the license holder and the type of license. The CPU <NUM> can receive a request to read electronic data from the reading apparatus <NUM> through the communication interface <NUM> and transmit the electronic data. The communication interface <NUM> includes an antenna coil connected to the IC chip and can communicate with the reading apparatus <NUM> by methods such as electromagnetic induction. In this case, the reading apparatus <NUM> can be a reader/writer of a contactless IC card. The power supply unit <NUM> may receive electromagnetic waves, transmitted by the reading apparatus <NUM>, through the antenna of the communication interface <NUM>, convert the electromagnetic waves to electric power, and supply the electric power to the CPU <NUM> and memory <NUM>. The power supply unit <NUM> may also have a built-in battery to supply electric power to the CPU <NUM> and the memory <NUM>.

The driver information <NUM> acquired from the medium 13a and/or 13b may be transmitted to the determination apparatus <NUM> and used by the controller <NUM>. For example, the controller <NUM> can determine whether the type of driver's license conforms to the vehicle to be driven. When determining that the type of driver's license does not conform, the controller <NUM> may suspend driving even when determining that the driver is the same as the person in the facial photograph image held by the medium <NUM>. In a vehicle for car sharing, the controller <NUM> may suspend driving when the name and address do not match member information registered in the memory <NUM>.

In <FIG>, the camera <NUM> can be disposed on the dashboard of the vehicle. The camera <NUM> is not limited to being installed at this position and may be located at any position allowing imaging of an imaging area Ai in which the face of the seated driver is assumed to be located. The camera <NUM> may, for example, be installed in a meter hood, on top of the steering column, or at the upper portion of the windshield. The camera <NUM> can also function as a camera for monitoring the state of the driver to prevent the driver from falling asleep at the wheel.

The display apparatus <NUM> can be attached at any position visible to the driver, such as in or on the dashboard. A liquid crystal display for a car navigation system mounted in the vehicle can, for example, be used as the display apparatus <NUM>. A dedicated display for the driver confirmation system <NUM> may also be provided as the display apparatus <NUM>.

The control apparatus <NUM> can be a control device or an electronic control unit that controls the different parts of the vehicle. When the determination apparatus <NUM> determines that the driver is not the person in the facial photograph image held by the medium <NUM>, the determination apparatus <NUM> transmits a signal to various control apparatuses <NUM> to restrict or suspend driving operations. The various control apparatuses <NUM> may include any of an engine start control device that enables starting of the engine, an accelerator control device that enables the accelerator, a shifting control device that enables movement of the shift knob, a parking brake control device that enables movement of the parking brake, and the like.

The example detection apparatus <NUM> illustrated in <FIG> is a seat sensor for detecting that the driver is sitting in the seat. The seat sensor can be a pressure sensor provided inside the driver's seat. When the seat sensor does not detect sitting by the driver for a predetermined time or longer and subsequently detects sitting by the driver, the driver may have switched. Hence, even during driving, the determination apparatus <NUM> initiates the determination processing for determining whether the driver is the person in the facial photograph image held by the medium <NUM>. The driver may switch in this way during driving when the vehicle is driving autonomously.

The detection apparatus <NUM> can use a variety of sensors provided in the vehicle other than the seat sensor to detect the possible switch in drivers. For example, a pressure sensor for detecting that a foot has been placed on a foot rest, a sensor for detecting that the driver is gripping the steering wheel, or the like may be included as the detection apparatus <NUM>. A grip sensor for detecting pressure on the steering wheel, for example, can be used to recognize whether the driver is gripping the steering wheel. It can also be recognized whether the driver is gripping the steering wheel by detecting counter torque acting on the steering wheel. As described above, the camera <NUM> can function as the detection apparatus <NUM>.

Next, the detection processing executed by the controller <NUM> is described with reference to <FIG>.

In response to the various conditions described above, the processing initiation unit <NUM> of the controller <NUM> initiates the determination processing. When the determination processing is initiated by the processing initiation unit <NUM>, the controller <NUM> judges whether the reading apparatus <NUM> can read the facial photograph image of the medium <NUM> (step S01). The controller <NUM> can make this judgment on the basis of whether the facial photograph image can be acquired from the first acquisition unit <NUM>. When the facial photograph image cannot be acquired, it may be that the medium <NUM> is not inserted readably in the reading apparatus <NUM>. In this case, the controller <NUM> proceeds to the processing from step S07 on, described below. When the facial photograph image can be acquired from the first acquisition unit <NUM>, the controller <NUM> acquires and stores the facial photograph image as a first facial image in the memory <NUM> (step S02).

The controller <NUM> acquires a captured image captured by the camera <NUM> from the second acquisition unit <NUM> (step S03).

Next, the controller <NUM> uses the face extraction unit <NUM> to extract an image of a face as a second facial image from the captured image acquired in step S03 (step S04).

The controller <NUM> uses the identity determination unit <NUM> to determine whether the first facial image acquired in step S02 and the second facial image extracted in step S04 are the same (step S05).

The processing by the controller <NUM> branches on the basis of the result of the determination of identity (step S06). When it is determined that the person in the first facial image and the person in the second facial image are the same, the controller <NUM> ends the processing without restricting driving in any way. Before ending the processing, the controller <NUM> may store the facial photograph <NUM> and driver information <NUM> of the current driver and/or information such as the time of determination in the memory <NUM>. When the controller <NUM> cannot judge that the person in the first facial image and the person in the second facial image are the same, the processing proceeds to step S07.

In step S07, the controller <NUM> issues a warning via the output unit <NUM>. This warning includes displaying a warning that the driver is not a legitimate driver on the display apparatus <NUM>. The controller <NUM> may cause a speaker inside the vehicle to emit a warning sound.

Furthermore, the controller <NUM> transmits a signal to the control apparatus <NUM> through the output unit <NUM> for the control apparatus <NUM> to restrict driving by the driver (step S08). The method by which the control apparatus <NUM> restricts driving may differ in accordance with the conditions when the driver confirmation system <NUM> confirms the driver by the determination processing. For example, when the driver is confirmed at the start of driving, the control apparatus <NUM> can disable engine activation and/or shift knob operation or the like to suspend the start of driving. When the driver is confirmed during driving, the controller <NUM> may first issue a warning to the driver. In conjunction with the control apparatus <NUM>, the controller <NUM> may then slow the vehicle down, move the vehicle to the shoulder of the road being driven on, and stop the vehicle. Alternatively, before restricting driving, the controller <NUM> may prompt the driver to insert a proper medium <NUM>, i.e. license, in the reading apparatus <NUM> in step S07.

As described above, the driver confirmation system <NUM> of the present embodiment compares a first facial image included in the medium <NUM>, which holds a facial photograph image, and a second facial image of the driver obtained by capturing an image of the driver at the start of driving and/or during driving and determines whether the facial images are of the same person. In this way, driving can be limited to legitimate drivers only. When the medium <NUM> is a driver's license, driving by a non-legitimate driver that does not possess a driver's license can be deterred. At this time, the driver need not perform any cumbersome operations such as registering a facial photograph image of the driver or the like in advance.

The driver confirmation system <NUM> according to the present embodiment can judge whether the person in the facial photograph image of the medium <NUM> and the driver included in the captured image are the same not only at the start of driving but also during driving. It can therefore be detected that the driver has switched during driving of the apparatus to be driven. A warning can then be issued, and driving can be restricted or suspended.

<FIG> illustrates an imaging apparatus <NUM> according to an embodiment along with the above-described reading apparatus <NUM>, display apparatus <NUM>, control apparatus <NUM>, detection apparatus <NUM>, and external system <NUM>. Constituent elements that are identical or correspond to the constituent elements in <FIG> and constituent elements that have identical or corresponding operations are labeled with the same reference signs, and a description thereof is omitted. In the embodiment in <FIG>, the controller <NUM> controls the imaging apparatus <NUM> overall.

Along with an imaging unit <NUM> provided with the optical system <NUM>, the image sensor <NUM>, and the signal processing circuit <NUM>, the imaging apparatus <NUM> includes all of the constituent elements of the determination apparatus <NUM> in <FIG>. In this way, the imaging apparatus <NUM> extracts the second facial image from the captured image of the driver. Furthermore, the imaging apparatus <NUM> can compare the second facial image with the first facial image held by the medium <NUM> and acquired from the reading apparatus <NUM> to determine whether the driver is a legitimate driver. Accordingly, the imaging apparatus <NUM> can obtain the same effects as the driver confirmation system <NUM>.

The present disclosure is not limited to the above embodiments, and a variety of modifications and changes are possible. For example, the functions and the like included in the various components and steps may be reordered in any logically consistent way. Components, steps, or the like may also be combined into one or divided. For example, the reading apparatus and/or display apparatus may be provided in the same housing as the determination apparatus or the imaging apparatus. While embodiments of the present disclosure have been described focusing on apparatuses, the present disclosure may also be embodied as a method that includes steps performed by the components of an apparatus.

When the facial photograph image is displayed on the surface of the medium in the above embodiment, the reading apparatus has been described as optically reading the content displayed on the surface of the medium, including the facial photograph image, and extracting the facial photograph image from the read image. The determination apparatus, however, may extract the facial photograph image instead of the reading apparatus. In this case, the facial photograph image acquired by the first acquisition unit may be considered an image that includes a facial photograph and other content.

Claim 1:
A determination apparatus (<NUM>) comprising:
a first acquisition unit (<NUM>) configured to acquire a facial photograph image (<NUM>), included in a medium (<NUM>) holding the facial photograph image (<NUM>), as a first facial image;
a second acquisition unit (<NUM>) configured to acquire a captured image obtained by capturing an image of an imaging area (Ai), in which a face of a driver of a moveable body is assumed to be located, at a start of driving and/or during driving; and
a controller (<NUM>) configured to extract an image of the face of the driver as a second facial image from the captured image acquired from the second acquisition unit (<NUM>), compare the first facial image and the second facial image, and determine during driving whether the driver is a person in the facial photograph image (<NUM>) held by the medium (<NUM>),
wherein the controller (<NUM>) is configured to measure a driving time of the driver and issue a warning when the driving time of the driver exceeds a predetermined time.