Pedestrian face detection

A controller for a vehicle is programmed to detect a pedestrian in an image received from a camera, determine whether a face of the pedestrian is present in the image, and cause the vehicle to change lanes based on the absence of the face.

BACKGROUND

Autonomous vehicles have the ability to operate without the intervention of a human operator, e.g., driver, that is, a computer controller makes decisions about accelerating, braking, and/or steering the vehicle. A vehicle may be fully autonomous or semi-autonomous. A semi-autonomous vehicle is one that is autonomous only in particular situations, for example, highway driving or parallel parking, or with respect to certain vehicle subsystems, for example, braking but not acceleration or steering.

An autonomous vehicle includes sensors for tracking an external environment surrounding the vehicle. Some types of sensors are radar sensors, scanning laser range finders, light detection and ranging (LIDAR) devices, and image processing sensors such as cameras. The controller is in communication with the sensors and uses output from the sensors to analyze the external environment, for example, defining features of a surrounding landscape, detecting roads and lanes of roads on the landscape, interpreting signs and signals, and detecting proximate objects.

DETAILED DESCRIPTION

In the Figures, like numerals indicate like parts throughout the several views. Referring toFIGS. 1 and 2, a controller32for a vehicle30is programmed to detect a pedestrian99in an image received from a camera34,36, determine whether a face of the pedestrian99is present in the image, and cause the vehicle30to take an action based on whether a face is detected, e.g., change lanes based on an absence of the face.

The programming of the controller32improves an ability of the controller32to predict the intentions of pedestrians99and improves coordination between the vehicle30and other road users like pedestrians99. For example, if the face of the pedestrian99is detected, then the pedestrian99is more likely to have seen the vehicle30and is less likely to cross in front of the vehicle30. If the face of the pedestrian99is not detected, then the pedestrian99is less likely to have seen the vehicle30. With that information, the controller32can take additional precautions when passing by the pedestrian99, such as changing lanes to give the pedestrian99a wider berth.

Continuing withFIGS. 1 and 2, the vehicle30is an autonomous vehicle. The controller32may be capable of operating the vehicle30independently of the intervention of a human driver, completely, or to a greater or a lesser degree. The controller32may be programmed to operate propulsion40, steering42, brakes44, and/or other vehicle systems.

With reference toFIG. 2, the controller32is included in a control system38for carrying out various operations, including as described herein. The controller32is a computing device that generally includes a processor and a memory, the memory including one or more forms of computer-readable media, and storing instructions executable by the processor for performing various operations, including as disclosed herein. The memory of the controller32further generally stores remote data received via various communications mechanisms; e.g., the controller32is generally configured for communications on a communications network46, and/or for using other wired or wireless protocols, e.g., Bluetooth, etc. The controller32may also have a connection to an onboard diagnostics connector (OBD-II). Although one controller32is shown inFIG. 1for ease of illustration, it is to be understood that the controller32could include, and various operations described herein could be carried out by, one or more computing devices.

The controller32may transmit and receive data through the communications network46such as a controller area network (CAN) bus, Ethernet, Local Interconnect Network (LIN), and/or by any other wired or wireless communications network. The communications network46may put the controller32in communication with the propulsion40, the steering42, the brakes44, sensors48including the camera34,36, headlamps50, and a horn52.

The propulsion40of the vehicle30generates energy and translates the energy into motion of the vehicle30. The propulsion40may be a known vehicle propulsion subsystem, for example, a conventional powertrain including an internal-combustion engine coupled to a transmission that transfers rotational motion to wheels; an electric powertrain including batteries, an electric motor, and a transmission that transfers rotational motion to the wheels; a hybrid powertrain including elements of the conventional powertrain and the electric powertrain; or any other type of propulsion. The propulsion40can include an electronic control unit (ECU) or the like that is in communication with and receives input from the controller32and/or from a human driver. The human driver may control the propulsion40via, e.g., an accelerator pedal and/or a gear-shift lever.

The steering42is typically a known vehicle steering subsystem and controls the turning of the wheels. The steering42can include an electronic control unit (ECU) or the like that is in communication with and receives input from a steering wheel and/or the controller32. The steering42may be a rack-and-pinion system with electric power-assisted steering, a steer-by-wire system, as both are known, or any other suitable system.

The brakes44are typically a known vehicle braking subsystem and resist the motion of the vehicle30to thereby slow and/or stop the vehicle30. The brakes44may be friction brakes such as disc brakes, drum brakes, band brakes, etc.; regenerative brakes; any other suitable type of brakes; or a combination. The brakes44can include an electronic control unit (ECU) or the like that is in communication with and receive input from the controller32and/or a human driver. The human driver may control the brakes44via, e.g., a brake pedal.

The vehicle30may include the sensors48. The sensors48may detect internal states of the vehicle30, for example, wheel speed, wheel orientation, and engine and transmission variables. The sensors48may detect the position or orientation of the vehicle30, for example, global positioning system (GPS) sensors; accelerometers such as piezo-electric or microelectromechanical systems (MEMS); gyroscopes such as rate, ring laser, or fiber-optic gyroscopes; inertial measurements units (IMU); and magnetometers. The sensors48may detect the external world, for example, radar sensors, scanning laser range finders, light detection and ranging (LIDAR) devices, and image processing sensors such as the camera34,36. The sensors48may include communications devices, for example, vehicle-to-infrastructure (V2I) or vehicle-to-vehicle (V2V) devices.

With reference toFIG. 1, the camera34,36may be disposed at a front end of the vehicle30or at a top of the vehicle30. The camera34,36may be facing in a generally vehicle-forward direction. The vehicle30may include multiple cameras34,36, which may face different directions.

Each camera34,36has a field of vision56, which is an entire view encompassed by the camera34,36when the camera34,36is pointed in a particular direction. The camera34,36may have a field of vision56that includes an area58,60in front of and outside a forward-projected width of the vehicle30, in other words, an area58,60in front of the vehicle30excepting where the vehicle30would travel if moving straight forward. If the vehicle30has multiple cameras34,36, the first camera34may have a field of vision56including an area58in front of and left of the forward-projected width of the vehicle30, and the second camera36may have a field of vision56including an area60in front of and right of the forward-projected width of the vehicle30.

The sensors48may include an occupancy sensor54configured to detect occupancy of the vehicle30. The occupancy sensor54may be visible-light or infrared cameras directed at one or more of the seats, weight sensors inside the seats, sensors detecting whether seatbelts for the seats are buckled or unspooled, or other suitable sensors. The occupancy sensor54is in communication with the controller32via the communications network46.

The headlamps50may be fixed relative to the vehicle30and disposed at a front end of the vehicle30facing in a vehicle-forward direction. The headlamps50may be any lighting system suitable for illuminating a roadway in front of the vehicle30, including tungsten, halogen, high-intensity discharge (HID) such as xenon, light-emitting diode (LED), laser, etc. The headlamps50may be stationary or adaptive, that is, capable of rotating relative to the vehicle30.

The horn52produces a sound when actuated. The sound may resemble a “honk” and is loud enough to generally be perceived by other vehicles, pedestrians99, cyclists, etc. in the vicinity of the vehicle30.

FIG. 3is a process flow diagram illustrating an exemplary process300for reacting to a pedestrian99. The controller32may run the process300when the vehicle30is operating autonomously and approaching an intersection, crosswalk, etc. The process300begins in a block305, in which the controller32receives an image through the communications network46from the one or more cameras34,36.

Next, in a decision block310, the controller32detects whether a pedestrian99is present in the image received from the cameras34,36.FIG. 4shows an example image from the camera34,36including two pedestrians99. The controller32typically uses known techniques for object detection and classification to analyze such images, e.g., to determine a presence or absence of a pedestrian. The controller32may analyze a still image and/or a sequence of images or video to detect whether a pedestrian99is present in the image received from the camera34,36. If a pedestrian99is not detected, the process300proceeds back to the block305to receive another image and restart the process300.

If a pedestrian99is detected, next, in a decision block315, the controller32determines whether the pedestrian99is located in one of the areas58,60in front of and outside the forward-projected width of the vehicle30. If the pedestrian99is not located in one of the areas58,60, the process300returns to the block305to receive another image and restart the process300. If the pedestrian99is specifically located in the forward-projected width of the vehicle30, then pedestrian99is not located in one of the areas58,60, but the controller32may use a separate collision avoidance algorithm, as is known, to decide which actions to perform.

If the pedestrian99is located in one of the areas58,60in front of and outside the forward-projected width of the vehicle30, next, in a block320, the controller32detects the head of the pedestrian99in the image based on detecting the pedestrian99. The controller32typically uses known techniques for object detection and classification, e.g., to identify a head of a human being in an image.

Next, in a decision block325, the controller32determines whether a face of the pedestrian99is present in the image. Specifically, the controller32may determine whether a face of the pedestrian99is present by determining whether two eyes of the pedestrian99are present in the image, typically using known image-analysis techniques.FIGS. 5A-Cshow heads of pedestrians for which the controller32would find no face present, andFIGS. 5D-Eshow heads of pedestrians for which the controller32would find faces present. If the controller32determines that a face of the pedestrian99is present in the image, then the pedestrian99is more likely to be aware of the vehicle30, and the process300proceeds back to the block305to receive another image and restart the process300.

If the controller32determines that a face of the pedestrian99is absent in the image, next, in a block330, the controller32actuates the propulsion40to cease accelerating based on the absence of the face.

Next, in a block335, the controller32actuates the horn52to sound based on the absence of the face. The sound of the horn52may alert the pedestrian99that the vehicle30is nearby.

Next, the controller32actuates the headlamps50to illuminate based on the absence of the face. The light of the headlamps50may alert the pedestrian99that the vehicle30is nearby.

Next, in a decision block345, the controller32determines whether a lane change away from the pedestrian99is available. Specifically, as is known, the controller32may determine whether a lane is present that is traveling the same direction as the vehicle30and on the opposite side of the vehicle30than the pedestrian99, and if so, the controller32may determine whether the lane has sufficient space free of vehicles or other obstacles. If a lane change is available, the process300proceeds to a block350. If a lane change is not available, the process300proceeds to a decision block355.

After the decision block345if a lane change is available, in the block350, the controller32causes the vehicle30to change lanes based on the absence of the face of the pedestrian99in the image. Techniques for controlling a vehicle30to change lanes are known. As a result, the vehicle30may pass farther away from the pedestrian99than without the lane change.

After the decision block345if a lane change was not available, and after the block350if a lane was available, in the decision block355, the controller32determines whether to actuate the brakes44. For example, the controller32may determine based on object tracking of the pedestrian99whether the pedestrian99has a trajectory leading him or her in front of the vehicle30. If braking is not to be initiated, then the process300proceeds to a decision block360. If braking is to be initiated, then the process300proceeds to a block365.

After the decision block355if braking is not to be initiated, in the decision block360, the controller32determines whether the vehicle30contains occupants. The controller32may receive a signal indicating occupancy through the communications network46from the occupancy sensor54. If the vehicle30contains occupants, the process300ends. If the vehicle30does not contain occupants, the process300proceeds to a block365.

After the decision block355if braking is to be initiated or after the decision block360if braking was not to be initiated and the vehicle30does not contain occupants, in the block365, the controller32actuates the brakes44to brake based on the absence of the face of the pedestrian99in the image and possibly on the absence of occupants. After the block365, the process300ends.