Patent ID: 8704887

Claim:
A method, for fusing camera data and range sensor data received from a sensor subsystem of a vehicle to track an external object, comprising: determining, by an on-board computer of the vehicle, that that new object data corresponding to the object is available based on input received from the sensor sub-system of the vehicle, wherein the new object data comprises radar data, from a range sensor of the sensor subsystem, in the form of data points corresponding to the object; and an image, from a camera, of the sensor subsystem, in the form of pixels; registering, by the on-board computer, the new object data, wherein registering the data comprises obtaining the radar data and projecting the radar data into a plane of the image from the camera; estimating, by the on-board computer, an expected location and an expected appearance for the object according to a prediction algorithm to generate a predicted track corresponding to the object; and analyzing, by the on-board computer, motion for the object including comparing the predicted track with any existing track associated with the object and stored in a database of the on-board computer; wherein: a base pixel, of the pixels of the camera image, is in the plane of the image corresponding to a bottom part of the object, and is computed according to a linear projection including a matrix multiplication; and the linear projection including the matrix multiplication is represented by: u = Pr P = K ⁡ [ Rt ] t = - R ′ ⁢ r 0 R ′ = R ⁡ [ cos ⁢ ⁢ α 0 - sin ⁢ ⁢ α 0 1 0 sin ⁢ ⁢ α 0 cos ⁢ ⁢ α ] , u=[r i ,c i ,1] T denotes a row index and a column index of the base pixel; K represents parameters intrinsic to the camera; R represents a rotation matrix; r 0 represents a position of the camera in a vehicle frame; r=[ξ i ,η i ,0,1] T denotes a ground contact point of the object; and α denotes a pitch angle of the vehicle.