Physique determination apparatus and physique determination method

According to the present invention, a physique determination apparatus includes an image acquisition unit configured to acquire an image of an occupant, a facial feature point estimation unit configured to estimate facial feature points of the occupant based on the image, a head position estimation unit configured to estimate a relative position of a head of the occupant with respect to the camera based on the facial feature points, a body part estimation unit configured to estimate sizes of a plurality of parts of the occupant in the image, a skeleton estimation unit configured to estimate a skeleton size indicating an actual size of each part of the occupant based on the relative position and a size of each of the parts, and a physical determination unit configured to determine a physique of the occupant based on the skeleton size of each part of the occupant.

TECHNICAL FIELD

The present invention relates to a physique determination apparatus and a physique determination method for determining a physique of an occupant in a vehicle.

BACKGROUND ART

Conventionally, a technique has been disclosed in which a plurality of sensors are provided in a seat in a vehicle, the load applied to the seating surface when the occupant is seated is detected by each sensor, and whether the occupant is an adult or a child is determined (see, for example, Patent Document 1). In addition, the technology is disclosed in which a grid pattern and a CCD camera are provided on the seating part of the seat in the vehicle, and the grid pattern deformed by the load applied to the seating part when the occupant is seated is photographed by the CCD camera to determine the physique of the occupant (see, for example, Patent Document 2). Patent Documents 1 and 2 refer to controlling the deployment of airbags according to the physique of a seated occupant.

PRIOR ART DOCUMENTS

Patent Documents

SUMMARY

Problem to be Solved by the Invention

In recent years, vehicles equipped with airbags for all seats for the purpose of protecting all occupants are increased. In such a vehicle, the determination is required for the physiques of all occupants to be seated in the seats in order to control the deployment of airbags. Further, the determination result of the physique of an occupant can be used not only for controlling the deployment of the airbag but also for controlling to automatically adjust the seat position, for example. In this manner, there is an increasing demand for determining the physiques of all occupants.

In Patent Documents 1 and 2, in order to determine the physiques of all occupants of the seats, equipping sensors and the like for all seats is required, causing a problem of high total system cost.

The present invention has been made to solve such a problem, and an object of the present invention is to provide a physique determination apparatus and a physique determination method capable of inexpensively determining the physique of an occupant in a vehicle.

Means to Solve the Problem

In order to solve the above problem, according to the present invention, a physique determination apparatus includes an image acquisition unit configured to acquire an image of an occupant in a vehicle taken by a camera, a facial feature point estimation unit configured to estimate facial feature points of the occupant based on the image acquired by the image acquisition unit, a head position estimation unit configured to estimate a relative position of a head of the occupant with respect to the camera based on the facial feature points estimated by the facial feature point estimation unit, a body part estimation unit configured to estimate sizes of a plurality of parts of the occupant in the image based on the image acquired by the image acquisition unit, a skeleton estimation unit configured to estimate a skeleton size indicating an actual size of each part of the occupant based on the relative position of the head of the occupant estimated by the head position estimation unit and a size of each of the parts estimated by the body part estimation unit, and a physical determination unit configured to determine a physique of the occupant based on the skeleton size of each part of the occupant estimated by the skeleton estimation unit.

Effects of the Invention

According to the present invention, the physique determination apparatus includes the body part estimation unit that estimates the sizes of a plurality of parts of the occupant in the image based on the image acquired by the image acquisition unit, the skeleton estimation unit that estimates a skeleton size indicating the actual size of each part of the occupant based on the relative position of the head of the occupant estimated by the head position estimation unit and the size of each part estimated by the body part estimation unit, and the physique determination unit that determines the physique of the occupant based on the skeleton size of each part of the occupant estimated by the skeleton estimation unit; therefore, the physique of the occupant in the vehicle can be inexpensively determined.

The objects, features, aspects, and advantages of the present invention will become more apparent from the following detailed description and the accompanying drawings.

DESCRIPTION OF EMBODIMENT(S)

Embodiment of the present invention will be described below with reference to the drawings.

Embodiment

Configuration

FIG.1is a block diagram illustrating a configuration example of the physique determination apparatus1according to Embodiment. Note thatFIG.1illustrates the minimum necessary configuration for configuring the physique determination apparatus according to Embodiment. It is assumed that the physique determination apparatus1is mounted in a vehicle.

As illustrated inFIG.1, the physique determination apparatus1includes an image acquisition unit2, a facial feature point estimation unit3, a head position estimation unit4, a body part estimation unit5, a skeleton estimation unit6, and a physique determination unit7. Further, the image acquisition unit2is connected to a camera8. The camera8is mounted in the vehicle.

The image acquisition unit2acquires an image of an occupant in the vehicle taken by the camera8. The facial feature point estimation unit3acquires facial feature points of the occupant based on the image acquired by the image acquisition unit2. The head position estimation unit4estimates the relative position of the head of the occupant with respect to the camera8based on the facial feature points estimated by the facial feature point estimation unit3. The body part estimation unit5estimates the sizes of a plurality of parts of the occupant in the image based on the image acquired by the image acquisition unit2. The skeleton estimation unit6estimates a skeleton size indicating the actual size of each part of the occupant based on the relative position of the head of the occupant estimated by the head position estimation unit4and the size of each part estimated by the body part estimation unit5. The physique determination unit7determines the physique of the occupant based on the skeleton size of each part of the occupant estimated by the skeleton estimation unit6.

Next, another configuration of the physique determination apparatus including the physique determination apparatus1illustrated inFIG.1will be described.

FIG.2is a block diagram illustrating a configuration example of the physique determination apparatus9according to another configuration. It is assumed that the physique determination apparatus9is mounted in a vehicle.

As illustrated inFIG.2, the physique determination apparatus9includes the image acquisition unit2, the facial feature point estimation unit3, the head position estimation unit4, the body part estimation unit5, the skeleton estimation unit6, the physique determination unit7, and a determination result output unit10. Further, the image acquisition unit2is connected to the camera8. The determination result output unit10is connected to an airbag control device11. The airbag control device11is mounted in the vehicle.

The image acquisition unit2acquires an image of an occupant in the vehicle taken by the camera8. The camera8photographs occupants seated in the driver's seat, the passenger seat, and the rear seat, respectively.FIG.3is a diagram illustrating an image example of an occupant seated in the passenger seat. Although the example ofFIG.3illustrates a case where the occupant is seated in the passenger seat, Embodiment is not limited thereto. The image taken by the camera8may include an occupant seated in the driver's seat and an occupant seated in the rear seat.

The facial feature point estimation unit3estimates facial feature points of the occupant based on the image acquired by the image acquisition unit2. Specifically, the facial feature point estimation unit3estimates facial feature points indicating positions such as, a face, eyes, a nose and the like of the occupant included in the image acquired by the image acquisition unit2by collating the image acquired by the image acquisition unit2with a learning dictionary (not illustrated) prepared in advance. In the learning dictionary, the feature points of the faces of a plurality of people are recorded. When the image includes a plurality of occupants, the facial feature point estimation unit3estimates the facial feature points of each occupant.

FIG.4is a diagram illustrating an example of the facial feature points of the occupant estimated by the facial feature point estimation unit3. The occupant illustrated inFIG.4is seated in the passenger seat. InFIG.4, the line squarely surrounding the face of the occupant indicates the position of the face of the occupant. The dashed circles in the centers of the eyes of the occupant indicate the position and size of the pupils of the occupant. A plurality of dots illustrated on the face of the occupant indicate the positions of both ends of the eyebrows, the positions of both ends of the eyes, the position of the tip of the nose, and the positions of both ends of the mouth. The eye distance indicates the distance between both eyes of the occupant, specifically indicates the distance between the centers of each pupil. What the facial feature point estimation unit3estimates is not limited to the facial feature points illustrated inFIG.4, and may estimate other facial feature points.

The head position estimation unit4estimates the relative position of the head of the occupant with respect to the camera8based on the facial feature points estimated by the facial feature point estimation unit3. Specifically, the head position estimation unit4estimates the physical relative position of the head of the occupant with respect to the camera8based on the distance between the eyes or the size of the pupils of the occupant estimated by the facial feature point estimation unit3. That is, the head position estimation unit4estimates the distance between the camera8and the head of the occupant.

In Embodiment, the relative position of the head of the occupant with respect to the camera8is estimated based on the distance between the eyes or the size of the pupils of the occupant, which is less affected by any size of physique. Accordingly, regardless of the physique of the occupant, the relative position of the head of the occupant with respect to the camera8is estimated with a great accuracy. Also, even when the seat position in which the occupant is seated changes, the relative position of the head of the occupant with respect to the camera8is estimated with a great accuracy in accordance with the change.

The body part estimation unit5estimates the sizes of a plurality of parts of the occupant in the image based on the image acquired by the image acquisition unit2. Specifically, the body part estimation unit5estimates a size and position of each part such as the neck, arms, and upper body of the occupant in the image included in the image acquired by the image acquisition unit2by collating the image acquired by the image acquisition unit2with a learning dictionary (not illustrated) prepared in advance. In the learning dictionary, the size and position of each part of a plurality of people in the image are recorded. When the image includes a plurality of occupants, the body part estimation unit5estimates a plurality of parts of each occupant.

FIGS.5to8are diagrams illustrating an example of the size and position of a plurality of parts of the occupant in the image acquired by the image acquisition unit2.FIGS.5to7illustrate the size and position of each part of the occupants with different physiques. InFIGS.5to8, the straight line segments connecting each point correspond to the parts of the occupant, and the unit of the size of each part is pixel (pix). For example, a double-headed arrow illustrated inFIG.8indicates the shoulder width of the occupant (n pixels).

The skeleton estimation unit6estimates the skeleton size indicating the actual size of each part of the occupant based on the relative position of the head of the occupant with respect to the camera8estimated by the head position estimation unit4and the size of each part of the occupant in the image estimated by the body part estimation unit5. Specifically, the skeleton estimation unit6converts the size of each part of the occupant in the image estimated by the body part estimation unit5into a skeleton size indicating the actual size of each part of the occupant. When there are a plurality of occupants, the skeleton estimation unit6estimates the skeleton size of each part of each occupant.

Here, a case where the skeleton estimation unit6converts the shoulder width of the occupant in the image estimated by the body part estimation unit5into the actual shoulder width will be described with reference toFIGS.8and9.

InFIG.9, n represents the size of the shoulder width in the image, and corresponds to n illustrated inFIG.8, d represents the actual shoulder width. M represents the width of the image, which is determined by the angle of view θ of the camera8. x represents the distance between the camera8and the head of the occupant, and corresponds to the relative position of the head of the occupant with respect to the camera8estimated by the head position estimation unit4. The skeleton estimation unit6calculates the skeleton size being the actual shoulder width d, in accordance with following Equation (1).

As illustrated in above Equation (1), the skeleton estimation unit6calculates the skeleton size using the relative position of the head of the occupant with respect to the camera8estimated by the head position estimation unit4. Therefore, even when the seat position in which the occupant is seated changes, the skeleton estimation unit6accurately calculates the skeleton size of each part of the occupant.

Although in the above description, the case of converting the shoulder width of the occupant has been described, other parts of the occupant can be converted in the same manner. For example, the skeleton estimation unit6estimates at least two skeleton sizes from the sitting height, the shoulder width, the face size, and the arm length of the occupant.

The size of each part of the occupant in the image can be converted to a skeleton size that indicates the actual size of each part of the occupant because all parts of the occupant are assumed to exist at the relative position of the head of the occupant with respect to the camera8. It can be assumed that, although the shoulders, spine, and face are approximately the same as the relative position of the head of the occupant with respect to the camera8, this is not the case as the arms are allowed for large movement around the shoulders. For the arms, the body part estimation unit5may estimate the maximum arm size as the arm size of the occupant when the arm size is maximized through a change over time in the image. In this case, the skeleton estimation unit6converts the arm size in the image estimated by the body part estimation unit5into the actual arm size.

The physique determination unit7determines the physique of the occupant based on the skeleton size of each part of the occupant estimated by the skeleton estimation unit6. Specifically, the physique determination unit7inputs at least two skeleton sizes from the sitting height, shoulder width, face size, and arm length of the occupant estimated by the skeleton estimation unit6into a learned learner (not illustrated) that has been subjected to machine learning to determine the physique of the occupant, and executes the arithmetic processing of the learned learner; thereby, obtaining the determination result of the physique of the occupant from the learner. When there are a plurality of occupants, the physique determination unit7determines the physique of each occupant.

The learner has a Gaussian Mixture Model constructed from statistical data on the skeleton size of each part of a plurality of people. For example, when the respective skeleton sizes of the sitting height and shoulder width of the occupant as illustrated inFIG.10are input to the learner, the learner plots the respective skeletal sizes of the sitting height and the shoulder width in the Gaussian Mixture Model constructed from statistical data on respective skeleton sizes of sitting height and shoulder width and calculates the likelihood for each of a plurality of predetermined physique classes. Then, the learner determines that the class with the highest likelihood is the physique class of the occupant. The physique determination unit7acquires the determination result of the physique of the occupant from the learner, and classifies the physique of the occupant into classes in accordance with the physique of the occupant.

The physique class is determined, for example, based on the classification used in airbag control or seat position control. When there are two classes for classification, the physique of the occupant is classified into, for example, “large” or “small”. In addition, when there are three classes for classification, the physique of the occupant is classified into, for example, “large”, “medium”, or “small”. The learner has the Gaussian Mixture distribution for the number of classes. In the example ofFIG.11, it is illustrated that the physique of the occupant is classified as “small” from the respective skeleton sizes of the sitting height and shoulder width of the occupant illustrated inFIG.10. Accordingly, there is a correlation between the skeleton size of the occupant and the physique of the occupant.

Although in the above description, the case where the physique of the occupant is determined from the two skeleton sizes has been described, Embodiment is not limited thereto, and the physique of the occupant may be determined from three or more skeleton sizes. In this case, the Gaussian Mixture Model is constructed with statistical data of three or more skeleton sizes.

The determination result output unit10outputs the physique of the occupant determined by the physique determination unit7to the airbag control device11. Specifically, the determination result output unit10outputs the physique class of the occupant to the airbag control device11.

The airbag control device11controls the deployment of the airbag in accordance with the physique class of the occupant. For example, suppose that the physique classes of the occupant are classified into three categories: “large”, “medium”, and “small”. When “medium” is set as the standard, when it is “large”, the airbag control device11controls to apply a stronger force for the deployment of the airbag and deploy the airbag larger than that when it is “medium”, and when it is “small”, the airbag control device11controls to apply a weaker force for the deployment of the airbag and deploy the airbag smaller than that when it is “medium”.

Although in the above description, the case where the physique class of the occupant is output to the airbag control device11has been described, Embodiment is not limited thereto. In the following, an output example of the physique class of the occupant will be described.

For example, the determination result output unit10may output the physique class of the occupant to a seat position adjustment device that automatically adjusts the seat position. In this case, the seat position adjustment device automatically adjusts the seat position in accordance with the physique class of the occupant. When there are a plurality of occupants, the seat position adjustment device automatically adjusts the position for each seat in which each occupant is seated in accordance with the physique class of each occupant.

The determination result output unit10may output the physique class of the occupant to a side mirror position adjustment device that automatically adjusts the position of the side mirror. In this case, the side mirror position adjustment device automatically adjusts the position of the side mirror in accordance with the physique of the occupant.

The determination result output unit10may output the physique class of the occupant to a steering wheel position adjustment device that automatically adjusts the steering wheel position. In this case, the steering wheel position adjustment device automatically adjusts the position of the steering wheel in accordance with the physique of the occupant.

The determination result output unit10may output the physique class of the occupant to an abandonment detection device that detects whether or not a child is abandoned in the vehicle. In this case, the abandonment detection device may determine whether or not the child is a child from the physique of the occupant, and may determine that the abandonment has occurred when only the child is present in the vehicle for a certain period of time.

Operation

FIG.12is a flowchart illustrating an operation example of the physique determination apparatus1illustrated inFIG.1.

In Step S11, the image acquisition unit2acquires an image of an occupant in the vehicle taken by the camera8. In Step S12, the facial feature point estimation unit3estimates facial feature points of the occupant based on the image acquired by the image acquisition unit2.

In Step S13, the head position estimation unit4estimates the relative position of the head of the occupant with respect to the camera8based on the facial feature points estimated by the facial feature point estimation unit3. In Step S14, the body part estimation unit5estimates the sizes of a plurality of parts of the occupant in the image based on the image acquired by the image acquisition unit2.

In Step S15, the skeleton estimation unit6estimates a skeleton size indicating the actual size of each part of the occupant based on the relative position of the head of the occupant estimated by the head position estimation unit4and the size of each part estimated by the body part estimation unit5. In Step S16, the physique determination unit7determines the physique of the occupant based on the skeleton size of each part of the occupant estimated by the skeleton estimation unit6.

FIG.13is a flowchart illustrating an example of the operation of the physique determination apparatus9illustrated inFIG.2. Steps S21to S26inFIG.13correspond to Steps S11to S16inFIG.12; therefore, description thereof will be omitted here. In the following, Step S27will be described.

In Step S27, the determination result output unit10outputs the physique of the occupant determined by the physique determination unit7. In the example ofFIG.2, the determination result output unit10outputs the physique class of the occupant determined by the physique determination unit7to the airbag control device11.

Effect

According to Embodiment, the skeleton size indicating the actual size of each part of the occupant is estimated based on the relative position of the head of the occupant with respect to the camera8and the size of a plurality of parts of the occupant in the image and the physique of the occupant is determined based on the skeleton size, Consequently, the physique of all occupants in the vehicle can be determined by a single camera mounted in the vehicle, and this ensures a less expensive determination of the physiques of occupants than that by the conventional system in which sensors are provided in all seats.

In addition, the head position estimation unit4estimate the relative position of the head of the occupant with respect to the camera8based on the distance between the eyes or the size of the pupils of the occupant, which is less affected by any size of physique. The skeleton estimation unit6can accurately calculate the skeleton size of each part of the occupant using the accurately estimated relative position of the occupant. The physique determination unit7can determine the accurate physique of the occupant based on the accurate skeleton size of each part of the occupant. That is, the physique determination unit7can determine the accurate physique of the occupant even when the seat position in which the occupant is seated changes. Accordingly, the physique determination apparatuses1and9can accurately determine the physique of the occupant seated in the seat m the vehicle.

Hardware Configuration

Each function of the image acquisition unit2, the facial feature point estimation unit3, the head position estimation unit4, the body part estimation unit5, the skeleton estimation unit6, the physique determination unit7, and the determination result output unit10in the physique determination apparatuses1and9described above is realized by a processing circuit. That is, the physique determination apparatuses1and9include a processing circuit for acquiring the image of the occupant in the vehicle taken by the camera, estimating the facial feature points of the occupant, estimating the relative position of the head of the occupant with respect to the camera, estimating the sizes of a plurality of parts of the occupant in the image, estimating the skeleton size indicating the actual size of each part of the occupant, determining the physique of the occupant, and outputting the physique of the occupant. For the processing circuit, dedicated hardware may be applied, or a processor (also referred to as a Central Processing Unit (CPU), a central processing unit, a processing unit, an arithmetic unit, a microprocessor, a microcomputer, a Digital Signal Processor (DSP)) that executes a program stored in a memory may also be applied.

When the dedicated hardware is applied to the processing circuit, as illustrated inFIG.14, a processing circuit12corresponds, for example, to a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an Application Specific Integrated Circuit (ASIC), or a Field-Programmable Gate Array (FPGA), or the combination thereof. While each function of the image acquisition unit2, the facial feature point estimation unit3, the head position estimation unit4, the body part estimation unit5, the skeleton estimation unit6, the physique determination unit7, and the determination result output unit10may be realized by the processing circuit12, respectively, or may also be realized by one processing circuit12.

When a processor13illustrated inFIG.15is applied to the processing circuit12, each function of the image acquisition unit2, the facial feature point estimation unit3, the head position estimation unit4, the body part estimation unit5, the skeleton estimation unit6, the physique determination unit7, and the determination result output unit10is realized by software, firmware, or a combination of software and firmware. The software or firmware is written as a program and stored in a memory14. The processor13realizes each function by reading and executing the program recorded in the memory14. That is, the physique determination apparatuses1and9include the memory14for storing the program which, eventually, executes the steps of acquiring the image of the occupant in the vehicle taken by the camera, estimating facial feature points of the face of the occupant, estimating the relative position of the head of the occupant with respect to the camera, estimating the sizes of a plurality of parts of the occupant in the image, estimating the skeleton size indicating the actual size of each part of the occupant, determining the physique of the occupant, and outputting the physique of the occupant. Further, it can be said that these programs are programs to execute the procedure and method of the image acquisition unit2, the facial feature point estimation unit3, the head position estimation unit4, the body part estimation unit5, the skeleton estimation unit6, the physique determination unit7, and the determination result output unit10. Here, the memory may be, for example, a non-volatile or volatile semiconductor memory, such as a Random Access Memory (RAM), a Read Only Memory (ROM), a flash memory, an Erasable Programmable Read Only Memory (EPROM), an Electrically Erasable Programmable Read Only Memory (EEPROM), or the like, a magnetic disk, a flexible disk, an optical disk, a compact disk, a digital versatile disc (DVD) or the like, or any storage medium used in the future.

For each function of the image acquisition unit2, the facial feature point estimation unit3, the head position estimation unit4, the body part estimation unit5, the skeleton estimation unit6, the physique determination unit7, and the determination result output unit10, part of functions thereof may be realized by dedicated hardware and another part of the components is realized by software or the like.

Accordingly, the processing circuit can realize the above each function by hardware, software, firmware, or a combination thereof.

System Configuration

The physique determination apparatus described above is applicable, not only to an in-vehicle navigation device, that is, a car navigation device, but also to a Portable Navigation Device (PND) being mountable on a vehicle, and a navigation device constructed as a system by appropriately combining servers installed outside the vehicle, or devices other than navigation devices. In this case, each function or each component of the physique determination apparatus is distributed and arranged in each function for constructing the above system.

Specifically, as an example, the function of the physique determination apparatus can be arranged on the server. For example, as illustrated inFIG.16, the vehicle includes the camera8and the airbag control device11. Further, a server15includes the image acquisition unit2, the facial feature point estimation unit3, the head position estimation unit4, the body part estimation unit5, the skeleton estimation unit6, the physique determination unit7, and the determination result output unit10. With such a configuration, a physique determination system can be constructed.

As described above, even if each function of the physique determination apparatus is distributed and arranged in each function for constructing the system, the same effect as that of above Embodiment can be obtained.

Further, software that executes the operation according to above Embodiment may be incorporated into, for example, a server. The physique determination method realized by the server executing the software includes acquiring the image of the occupant in the vehicle taken by the camera, estimating facial feature points of the occupant based on the acquired image, estimating the relative position of the head of the occupant with respect to the camera based on the estimated facial feature points, estimating the sizes of a plurality of parts of the occupant in the image based on the acquired image, estimating the skeleton size indicating the actual size of each part of the occupant based on the estimated relative position of the head and the estimated size of each part, and determining the physique of the occupant based on the estimated skeleton size of each part of the occupant.

As described above, by incorporating the software that executes the operation in above Embodiment into the server and operating the server, the same effect as that in above Embodiment can be obtained.

The present invention can be combined, appropriately modified or omitted, without departing from the scope of the invention.

EXPLANATION OF REFERENCE SIGNS