Patent Description:
Trailers are usually unpowered vehicles that are pulled by a powered tow vehicle. A trailer may be a utility trailer, a popup camper, a travel trailer, livestock trailer, flatbed trailer, enclosed car hauler, and boat trailer, among others. The tow vehicle may be a car, a crossover, a truck, a van, a sports-utility-vehicle (SUV), a recreational vehicle (RV), or any other vehicle configured to attach to the trailer and pull the trailer. The trailer may be attached to a powered vehicle using a trailer hitch. A receiver hitch mounts on the tow vehicle and connects to the trailer hitch to form a connection. The trailer hitch may be a ball and socket, a fifth wheel and gooseneck, or a trailer jack. Other attachment mechanisms may also be used. In addition to the mechanical connection between the trailer and the powered vehicle, in some example, the trailer is electrically connected to the tow vehicle. As such, the electrical connection allows the trailer to take the feed from the powered vehicle's rear light circuit, allowing the trailer to have taillights, turn signals, and brake lights that are in sync with the powered vehicle's lights.

Some of the challenges that face tow vehicle drivers are connecting the tow vehicle to the trailer, because more than one person is needed. For example, one person drives the vehicle, e.g., the driver, and another one or more people are needed to view the tow vehicle and the trailer and provide the driver with direction regarding the path the tow vehicle has to take to align with the hitch. If the people providing directions to the driver are not accustomed to hitching a tow vehicle to a trailer, then they may have difficulty providing efficient instructions for directing the path of the tow vehicle. Other approaches from this involve the driver repeatedly exiting the vehicle to check the relative locations of the hitch and coupler, then getting back in the vehicle and repeat until the hitch is under the coupler or using the vehicle back up camera.

Recent advancements in sensor technology have led to improved safety systems for vehicles. Arrangements and methods for detecting and avoiding collisions are becoming available. Such driver assistance systems use sensors located on the vehicle to detect an ongoing collision. In some examples, the system may warn the driver of one or more driving situations to prevent or minimize collisions. Additionally, sensors and cameras may also be used to alert a driver of possible obstacles when the vehicle is traveling in a forward direction. Therefore, it is desirable to provide a system that includes sensors to overcome the challenges faced by drivers of tow vehicles.

Additionally, vehicles with driver assist functions sometimes utilize motion estimation algorithms. Such a motion estimation algorithm makes use of a vehicle model and sensor inputs, such as wheel encoder values and the steering wheel angle, to estimate the relative position and velocity of the vehicle over time. The terms "odometry" and "dead reckoning" are often used for such schemes.

For example, an automated function for reversing a truck to a trailer may require an accurate motion estimation algorithm so that the truck can follow a user-selected path to the trailer. Such a motion estimation algorithm would utilize several vehicle kinematic parameters, such as steering ratio, steering offset, tire diameter, wheelbase, track width, etc. While some parameters, such as the wheelbase, may be known accurately, the tire diameter, steering ratio, and steering offset may not be known with sufficient accuracy/precision for good motion estimation. For the case of reversing a truck to a trailer coupler, accuracy is needed on the order of centimeters over several meters of motion. This requires very good knowledge of the truck kinematic parameters.

One approach to determining the parameters involves careful direct measurements of all unknown parameters. However, this is often inconvenient and imprecise.

Another approach involves the execution of some test maneuvers while monitoring the vehicle sensor inputs. For example, tire diameters can be determined by driving the vehicle straight for some calibrated distance, like <NUM> meters, and monitoring the wheel encoder values. Similarly, steering ratio could be determined by fixing the steering wheel at a particular angle (which can be read from the steering wheel angle sensor) driving the vehicle forward or backward, and measuring the resulting radius of the actual vehicle path.

Another approach to determining the parameters is similar but utilizes a reference Differential GPS system for measuring the vehicle motion, rather than marking out a calibrated distance, such as the <NUM> meters mentioned above, or measuring the vehicle path radius.

<CIT> describes a tow vehicle backing assist system comprising a camera installed at the rear of the tow vehicle that captures an image of a view behind the tow vehicle, wherein based upon the captured camera image it is determined whether or not a trailer to be connected with the tow vehicle is present behind the tow vehicle. Therefore, an image of a coupler at the trailer captured with the camera in advance during a system registration routine is stored in a memory as a template image. For the system registration routine the tow vehicle has to be in a stopped state so that the trailer is positioned over a predetermined distance behind the tow vehicle to ensure that an image of the side of the trailer on which the trailer coupler is mounted can be captured. Once it is judged to be in a stopped state, the camera captures an image which is displayed at a display installed in the tow vehicle and the tow vehicle operator selects a rectangular area which contains the trailer coupler as a coupler pattern registration area by operating a setting switch. Then the image over the selected area is stored in the memory and registered as the template image of the coupler pattern of the trailer. To increase the degree of reliability of detection of the trailer during execution of the vehicle backing assist operation even when the positional relationship between the tow vehicle and the trailer prior to the coupling operation is somewhat offset, further hitch pattern of the coupler of the trailer can be registered by capturing images of the coupler of the trailer along a diagonal direction and repeating the processing described above. During execution of the vehicle backing assist operation a template match processing for the camera image by searching the camera image data to detect an area achieving a high correlation value to the template image is executed. If the camera image includes a portion matching the template image, a trailer connection guide line extending rearward from the coupler at the tow vehicle is drawn and displayed over the image of the view behind the tow vehicle.

<CIT> describes a method of locating a position of a trailer with which a vehicle is going to align. An electronic control unit of the vehicle receives a plurality of image frames of the object from a camera affixed to the vehicle and based on the received image frames, the electronic control unit identifies a trailer hitch of the trailer in the image frames and tracks the trailer hitch in the image frames while the vehicle is moving. The electronic control unit determines a change in position of the trailer hitch in the image frames as the vehicle is moving and determines a distance to the trailer hitch based on the change of position of the vehicle detected by vehicle sensors and the change in position of the trailer hitch (e.g. a corresponding pixel location) in the image frames. In order to compare information from the image and the movement data of the vehicle, a translation of systems of reference is done, namely of an optical coordinate system with respect to the camera and a ground coordinate system with respect to the vehicle, based on a relationship between these two coordinate systems, wherein the electronic control unit is calibrated during manufacture of the vehicle to define this relationship. When the relationship between the optical coordinate system and ground coordinate system changes over time (e.g. due to changes in vehicle height because of changes in tire pressure or high load conditions), the relationship between the two coordinate systems is adjusted for the changes, wherein the electronic control unit performs additional calibrations between the two coordinate systems after manufacture of the vehicle. Therefore, the electronic control unit performs a calibration based on a detected ground plane in the image captured by the camera and the relationship between the optical coordinate system and ground coordinate system is adjusted based on the position of the ground plane in the image.

<CIT> describes a vehicle and trailer display system that includes a plurality of cameras disposed on the vehicle and a screen disposed in the vehicle operable to display images from the cameras. A controller is in communication with the cameras and the screen and is operable to receive a hitch angle corresponding to the angle between the vehicle and the trailer, wherein based on the hitch angle, the controller is operable to select a field of view of a camera to display on the screen. Therefore, first a first-time trailer configuration for the camera-based hitch angle detection system is performed based on a target attached to a specific area of the trailer. When the target is acquired by a trailer backup assist system based on images from one of the cameras and the driver has acknowledged the acquisition, trailer measurement information (e.g. a horizontal distance from the rear of the vehicle to the center of a hitch ball, a horizontal distance from the rear of the vehicle to a center of the target, a horizontal distance from the rear of the vehicle to a center of the axle or axles of the trailer, a vertical distance from the target to the ground, a horizontal offset of the target from a centerline of the hitch ball) is then entered by the driver or automatically detected using existing vehicle and trailer data including vehicle speed, wheel rotation, wheel steer angle, vehicle to trailer relative angle, and a rate of change of the vehicle to hitch angle. Then, a hitch angle calibration is done to calibrate a curvature control algorithm with the proper trailer measurements and calibrate the trailer backup assist system for any hitch angle offset that may be present. For this, the driver is instructed by way of a screen display to pull the vehicle-trailer combination straight forward until a hitch angle sensor calibration is complete. Any hitch angle offset value is stored in a memory, accessed as necessary by the curvature control algorithm, and the calibration ends. Upon completion of the configuration and activation of the trailer backup assist system, an automatic steering of the vehicle during trailer backup assist operations is executed.

<CIT> describes a system to calibrate the cameras of the vehicle multi-camera system without use of reference points on the vehicle. The system figures out that a camera is out of calibration and then figures out how to calibrate it, while the vehicle is being normally driven by the driver. While the vehicle is driven along a road, the cameras capture frames of image data, and the system identifies or tags features in the captured images and then, over a set of frames, the system matches the features to determine how they are moving in the captured image and relative to the vehicle movement. In addition, the system receives an input from a kinematic model that provides kinematic data that indicates exactly how the vehicle is moving in reality. Then, the camera is calibrated based on the determined movement of the features, in particular if they do not appear in a position in which they should appear due to the determined vehicle movement.

One general aspect includes a method of calibrating a driver assistance system, the method including: entering a calibration mode for a driver assistance system. The method of calibrating also includes moving a vehicle to a first location and heading. The method of calibrating also includes recording with a controller for the driver assistance system a current estimated position and heading of the vehicle at the first location. The method of calibrating also includes displaying a first image from at least one camera outwardly facing from the vehicle. The method of calibrating also includes selecting at least one feature in the first image by using an input mechanism and recording with a controller for the driver assistance system the at least one selected feature data at the first location and heading. The method of calibrating also includes moving a vehicle to at least a second location and heading. The method of calibrating also includes determining with a controller for the driver assistance system a current estimated position and heading of the vehicle at the at least second location. The method of calibrating also includes displaying a second image from the at least one camera. The method of calibrating also includes selecting at least one feature in the second image by using the input mechanism and recording with a controller for the driver assistance system the at least one selected feature data at the second location and heading, where the at least one selected feature is the same feature as the previously at least one selected feature. The method of calibrating also includes calculating with the controller new vehicle motion parameter values based on the first estimated position and heading, second estimated position and heading, first recorded at least one feature data, and second recorded at least one feature data.

According to the invention, the vehicle motion parameter values include at least one of: wheelbase, track width, tire diameter, steering offset and steering ratio, the moving the vehicle includes, steering, braking, and acceleration.

Implementations may include one or more of the following features. The method where the driver assistance system is a hitch assist system and the at least one feature is at least a hitch receiver on a trailer.

The method where the motion calibration method is completed a first time the driver assistance system is used.

The method where the motion calibration method updated at periodic intervals or at user discretion.

The method where the input mechanism is at least one of: a touchscreen, a knob control, a mouse, joystick, a slider bar.

The method further including using the new vehicle motion parameter values to execute at least one driver assistance function of the driver assistance system.

The method where determining the current estimated position and heading of the vehicle at the second location further comprises calculating the position and heading from wheel travel sensor data and steering angle sensor data over time that is detected by a sensor system while the vehicle is moving from the first position to the second position.

One general aspect includes a method of calibrating a driver assistance system, the method including: entering a calibration mode for a driver assistance system. The method of calibrating also includes determining with a controller for the driver assistance system a first estimated position and heading of the vehicle at a first location. The method of calibrating also includes capturing an image from at least one camera outwardly facing from the vehicle and selecting at least one feature in the image. The method of calibrating also includes moving a vehicle to at least a second location and heading and determining a second estimated position and heading of the vehicle. The method of calibrating also includes capturing a second image from at least one camera outwardly facing from the vehicle and selecting at least one feature in the second image, where the at least one selected feature in the second image the same feature as the at least one selected feature in the first image. The method of calibrating also includes calculating with the controller new vehicle motion parameter values based on the first estimated position and heading, second estimated position and heading, first recorded at least one feature data, and second recorded at least one feature data.

Implementations may include one or more of the following features. The method where the vehicle motion parameter values include at least one of: wheelbase, track width, tire diameter, steering offset and steering ratio, the moving the vehicle includes, steering, braking, and acceleration.

The method where the driver assistance system is a hitch assist system and the at least one feature is at least a hitch receiver on a trailer.

The method further including selecting the at least one feature with an input mechanism.

One general aspect includes a driver assistance system including: a controller having instructions for calibrating vehicle motion parameter values for the driver assistance system the instructions including; determining with a controller for the driver assistance system a first estimated position and heading of the vehicle at a first location; capturing an image from at least one camera outwardly facing from the vehicle and selecting at least one feature in the image; determining a second estimated position and heading of the vehicle after the vehicle has been moved to a second location; capturing a second image from at least one camera outwardly facing from the vehicle and selecting at least one feature in the second image, where the at least one selected feature in the second image the same feature as the at least one selected feature in the first image; and calculating with the controller new vehicle motion parameter values based on the first estimated position and heading, second estimated position and heading, first recorded at least one feature data, and second recorded at least one feature data.

Implementations may include one or more of the following features. The system where the driver assistance system is a hitch assist system and the at least one feature is at least a hitch receiver on a trailer.

The system where the controller calibrates vehicle motion parameter values is completed a first time the driver assistance system is used and the motion calibration method updated at periodic intervals or at user discretion.

The system further including an input mechanism to select the at least one feature, where the input mechanism is at least one of: a touchscreen, a knob control, a mouse, joystick, a slider bar.

The system where the current estimated position and heading of the vehicle at the second location is determined by calculating the position and heading from wheel travel sensor data and steering angle sensor data over time that is detected by a sensor system while the vehicle is moving from the first position to the second position.

Other objects, features and characteristics of the present invention, as well as the methods of operation and the functions of the related elements of the structure, the combination of parts and economics of manufacture will become more apparent upon consideration of the following detailed description and appended claims with reference to the accompanying drawings, all of which form a part of this specification. It should be understood that the detailed description and specific examples, while indicating the preferred embodiment of the disclosure, are intended for purposes of illustration only and are not intended to limit the scope of the disclosure.

A tow vehicle, such as, but not limited to a car, a crossover, a truck, a van, a sports-utility-vehicle (SUV), and a recreational vehicle (RV) may be configured to tow a trailer. The tow vehicle connects to the trailer by way of a trailer hitch. It is desirable to have a tow vehicle that is capable to having a more automated system and method for maneuvering towards a trailer and attaching to the trailer, thus reducing the need for a driver to drive the tow vehicle in a rearward direction while another one or more people provide the driver with directions regarding the path that the tow vehicle has to take to align with the trailer and ultimately a hitch of the trailer. As such, a tow vehicle with an assistance device for rearward driving provides a driver with a safer and faster experience when hitching the tow vehicle to the trailer.

Referring to <FIG>, in some implementations, a driver of a tow vehicle <NUM> wants to tow a trailer <NUM>. The tow vehicle <NUM> may be configured with a driver assistance system <NUM> to provide guidance to the driver to drive towards the selected trailer <NUM>. The tow vehicle <NUM> may include a drive system <NUM> that maneuvers the tow vehicle <NUM> across a road surface based on drive commands having a final x, y location and also a final heading. Alternatively, the drive command may have radius, speed and direction components, for example. As shown, the drive system <NUM> includes a front right wheel <NUM>, 112a, a front left wheel <NUM>, 112b, a rear right wheel <NUM>, 112c, and a rear left wheel <NUM>, 112d. The drive system <NUM> may include other wheel configurations as well. The drive system <NUM> may also include a brake system <NUM> that includes brakes associated with each wheel <NUM>, <NUM>-d, and an acceleration system <NUM> that is configured to adjust a speed and direction of the tow vehicle <NUM>. In addition, the drive system <NUM> may include a suspension system <NUM> that includes tires associated with each wheel <NUM>, <NUM>-d, tire air, springs, shock absorbers, and linkages that connect the tow vehicle <NUM> to its wheels <NUM>, <NUM>-d and allows relative motion between the tow vehicle <NUM> and the wheels <NUM>, <NUM>-d. The suspension system <NUM> improves the road handling of the tow vehicle <NUM> and provides a better ride quality by isolating road noise, bumps, and vibrations. In addition, the suspension system <NUM> is configured to adjust a height of the tow vehicle <NUM> allowing the tow vehicle hitch <NUM> to align with the trailer hitch <NUM>, which aids in connection between the tow vehicle <NUM> and the trailer <NUM>.

The tow vehicle <NUM> may move across the road surface by various combinations of movements relative to three mutually perpendicular axes defined by the tow vehicle <NUM>: a transverse axis X, a fore-aft axis Y, and a central vertical axis Z. The transverse axis x, extends between a right side R and a left side of the tow vehicle <NUM>. A forward drive direction along the fore-aft axis Y is designated as F, also referred to as a forward motion. In addition, an aft or rearward drive direction along the fore-aft direction Y is designated as R, also referred to as rearward motion. When the suspension system <NUM> adjusts the suspension of the tow vehicle <NUM>, the tow vehicle <NUM> may tilt about the X axis and or Y axis, or move along the central vertical axis Z.

A driver assistance system <NUM> may include a user interface <NUM>. The user interface <NUM> may be a display that is incorporated into the vehicle or may be provided on a separate device, such as a personal wireless device. The user interface <NUM> receives one or more user commands from the driver via one or more input mechanisms <NUM>, such as a control knob, or a touch screen display and/or displays one or more notifications to the driver. The user interface <NUM> is in communication with a vehicle controller <NUM>, which is in turn in communication with the sensor system <NUM> and a drive system <NUM>. In some examples, the user interface <NUM> displays an image of an environment of the tow vehicle <NUM> leading to one or more commands being received by the user interface <NUM> (from the driver) that initiate execution of one or more behaviors. The vehicle controller <NUM> includes a computing device (or processor) <NUM> (e.g., central processing unit having one or more computing processors) in communication with non-transitory memory <NUM> (e.g., a hard disk, flash memory, random-access memory) capable of storing instructions executable on the computing processor(s)).

The vehicle controller <NUM> executes a driver assistance system <NUM>, which in turn includes a path following sub-system <NUM>. The path following sub-system <NUM> receives a planned path <NUM> (FIGS. 3A and 3B) from a path planning system <NUM> and executes behaviors <NUM>-<NUM> that send commands <NUM> to the drive system <NUM>, leading to the tow vehicle <NUM> autonomously driving about the planned path <NUM> in a rearward direction R.

The path following sub-system <NUM> includes, as a braking behavior <NUM>, a speed behavior <NUM>, a steering behavior <NUM>, and possibly additional behaviors, such as a hitch connect behavior, and a suspension adjustment behavior. Each behavior <NUM>-<NUM> cause the tow vehicle <NUM> to take an action, such as driving backward, turning at a specific angle, speeding, slowing down, among others. The vehicle controller <NUM> may maneuver the tow vehicle <NUM> in any direction across the road surface by controlling the drive system <NUM>, more specifically by issuing commands <NUM> to the drive system <NUM>. For example, the vehicle controller <NUM> may maneuver the tow vehicle <NUM> from an initial position to a final position. In the final position, a hitch ball <NUM> of the tow vehicle <NUM> aligns with a hitch coupler <NUM> of the trailer <NUM> connecting the tow vehicle <NUM> and the selected trailer <NUM>.

The tow vehicle <NUM> may include a sensor system <NUM> to provide reliable and robust autonomous driving. The sensor system <NUM> may include different types of sensors that may be used separately or with one another to create a perception of the tow vehicle's environment that is used for the tow vehicle <NUM> to autonomously drive and make intelligent decisions based on objects and obstacles detected by the sensor system <NUM>. The sensors may include, but not limited to, one or more imaging devices (such as cameras) <NUM>, and sensors <NUM> such as, but not limited to, radar, sonar, LIDAR (Light Detection and Ranging, which can entail optical remote sensing that measures properties of scattered light to find range and/or other information of a distant target), LADAR (Laser Detection and Ranging), etc. In addition, the camera(s) <NUM> and the sensor(s) <NUM> may be used to alert the driver of possible obstacles when the tow vehicle <NUM> is traveling in the forward direction F or in the rearward direction R, by way of audible alerts and/or visual alerts via the user interface <NUM>. Therefore, the sensor system <NUM> is especially useful for increasing safety in tow vehicles <NUM> which operate under semi-autonomous or autonomous conditions.

The sensor system <NUM> sends sensor system data <NUM> to the driver assistance system <NUM>. The sensor system data <NUM> includes images <NUM> from the cameras <NUM>, 410a-d and sensor data <NUM> from the sensors <NUM>, 420a-d.

In some implementations, the tow vehicle <NUM> includes a rear camera <NUM>, 410a that is mounted to provide a view of a rear driving path for the tow vehicle <NUM>. Additionally, in some examples, the tow vehicle <NUM> includes a front camera <NUM>, 410b to provide a view of a front driving path for the tow vehicle <NUM>, a right camera <NUM>, 410c positioned on the right side of the tow vehicle <NUM>, and a left camera <NUM>, 410d positioned on the left side of the tow vehicle <NUM>. The left and right cameras <NUM>, 410c, 410d provide additional side views of the tow vehicle <NUM>. In this case, the tow vehicle <NUM> may detect object and obstacles positioned on either side of the tow vehicle <NUM>, in addition to the objects and obstacle detected along the front and rear driving paths. The camera(s) <NUM>, 410a-d may be a monocular camera, binocular camera, or another type of sensing device capable of providing a view of the rear travelling path of the tow vehicle <NUM>. Likewise, sensors 420a-d may be located at various location around the exterior of the vehicle <NUM>.

Referring back to <FIG>, once the path planning system <NUM> plans a path <NUM>, the path following sub-system <NUM> is configured to execute behaviors that cause the drive system <NUM> to autonomously follow the planned path <NUM>. Therefore, the path following sub-system <NUM> includes one or more behaviors <NUM>-<NUM> that once executed allow for the autonomous driving of the tow vehicle <NUM> along the planned path <NUM>. The behaviors <NUM>-<NUM> may include, but are not limited to a braking behavior <NUM>, a speed behavior <NUM>, a steering behavior <NUM>, a hitch connect behavior <NUM>, and a suspension adjustment behavior <NUM>.

In this embodiment the path planning system <NUM> receives information via the user interface <NUM> and the user input mechanism <NUM>. For example, the user interface <NUM> and input mechanism may be one or a combination of a screen, touch screen, personal device, control knob, joy stick, slider knob or other input mechanisms. In the example shown, the user interface <NUM> is a touch screen and the input mechanism <NUM> is a control knob.

The planned path <NUM> of the vehicle hitch ball <NUM> may be displayed as an overlay (shown by phantom line in <FIG>) on the user interface <NUM>. The initial planned panned path <NUM> overlay may be based on the vehicles <NUM> current steering behavior <NUM>. Using the input mechanism <NUM> the user may adjust a planned length and trajectory of the planned path <NUM> until the planned path <NUM> aligns with the trailer hitch receiver location <NUM>. Once the user has selected the desired planned path <NUM> length and trajectory (shown by phantom line in <FIG>) the path planning system <NUM> can determine the braking behavior <NUM>, speed behavior <NUM> and steering behavior <NUM> necessary to follow the planned path <NUM>. This information can be sent to the path following sub-system <NUM> to be executed.

Alternatively, the user interface <NUM> and input mechanism <NUM> may be used to input a vehicle trajectory and the system user may control the speed and braking of the vehicle <NUM>.

The braking behavior <NUM> may be executed to either stop the tow vehicle <NUM> or to slow down the tow vehicle based on the planned path <NUM>. The braking behavior <NUM> sends a signal or command <NUM> to the drive system <NUM>, e.g., the brake system <NUM>, to either stop the tow vehicle <NUM> or reduce the speed of the tow vehicle <NUM>.

The speed behavior <NUM> may be executed to change the speed of the tow vehicle <NUM> by either accelerating or decelerating based on the planned path <NUM>. The speed behavior <NUM> sends a signal or command <NUM> to the brake system <NUM> for decelerating or the acceleration system <NUM> for accelerating.

The steering behavior <NUM> may be executed to change the direction of the tow vehicle <NUM> based on the planned path. As such, the steering behavior <NUM> sends the acceleration system <NUM> a signal or command <NUM> indicative of an angle of steering causing the drive system <NUM> to change direction.

In addition to the user interface <NUM> and input mechanism <NUM> the path planning system <NUM> uses information from the sensor system <NUM> to align the input selection from the user with the physical locations of the vehicle <NUM> and the trailer <NUM> as recorded by the sensors <NUM> and the camera <NUM>. That is, the path planning system <NUM> must determine the path from on the user input based on the camera <NUM>, 410a-d image and the physical location of the vehicle <NUM> and trailer <NUM> as measured by the sensor system <NUM>. Therefore, a motion calibration system <NUM> to calibrate alignment between the sensors system <NUM> and the path following sub-systems <NUM> may be used.

Referring to <FIG> and <FIG> the motion calibration system <NUM> utilizes a motion estimation algorithm that accurately outputs relative vehicle position and speed. This motion estimation algorithm will be a function of several vehicle parameters, such as steering ratio, steering offset, wheelbase, track width, etc. For standard vehicle driving functions these parameters are likely not known accurately enough for good motion estimation necessary for a driver assistance system <NUM> which provides automated hitch assistance. The motion calibration system <NUM> provides an algorithm and a method <NUM> for determining these values more accurately. In the example application (trailer hitch assist), the method does not require any special equipment beside what is already on the vehicle <NUM>.

The method <NUM> provides for the user to "point to" the same fixed feature from different points of view using the user interface <NUM>, which displays an image from the camera <NUM>, 410a-d, and the input mechanism <NUM> to "point to" the features. That is, user selects at least one stationary object or feature displayed on the user interface <NUM> using the input mechanism to identify the selected feature or object. The feature or object may be the hitch ball receiver and may include addition objects or features relative to the trailer such as, trailer corners, decals, identifying marks, wheels, etc..

The vehicle is then driven to a new location where the camera <NUM>, 410a still has a view of the trailer <NUM> and the user re-selects the feature(s) location(s) using the user interface <NUM>. An optimization routine (e.g. quasi-Newton) adjusts vehicle kinematic parameters to achieve the smallest errors for the points selected by the user.

There is a need for an absolute reference to compare the motion estimation algorithm output to. Therefore, we propose an alternative absolute reference which utilizes the vehicle camera <NUM>, 410a-d and some input mechanism on the user interface <NUM>. In this embodiment the input mechanism may be a touch-screen, or the same control knob, or some combination thereof which allows the user to select one or more features on the displayed image <NUM>.

One embodiment of the motion calibration system <NUM> provides that a motion estimation algorithm is activated when the driver assistance system is activated. A motion "calibration mode" can be selectively entered or automatically entered the first time the system is used", shown at <NUM>. Alternatively, another embodiment provides the user (e.g. end user or factory end-of-line calibration technician) activating the special calibration mode. A first position and heading of the vehicle <NUM> is recorded. The first position may be set to (X,Y)=(<NUM>,<NUM>) and Heading=<NUM>.

In order to record a current estimated position and heading at a second location some appreciable vehicle motion may need to take place, shown at <NUM>. The user may drive the vehicle some distance (say <NUM> meters) while also turning the steering wheel so that the vehicle motion wasn't perfectly straight. During this motion the sensor system <NUM> detects at least wheel sensor data and steering wheel angle sensor data. Following this movement, the current estimated position and heading of the vehicle may be determined, shown at <NUM>. That is, the controller <NUM> can use the wheel ticks and steering wheel angles as functions of time during the movement from one location to another to calculate a second position and heading. Alternatively, image analyses from a second camera image, shown in <FIG>, compared to the first camera image, shown in <FIG> can be used to calculate a new position and heading.

Following this, the user would view the vehicle rear camera monitor (or front or side camera monitor) through the user interface <NUM> and use an input device <NUM> such as a pointing device (touchscreen, knob, mouse, etc.) to pick out a particular static world feature which is displayed on the image <NUM>, shown at <NUM>. This feature could be a trailer coupler, for example, for the case of a truck hitch assist system, but it could also be any other fixed point that is easy for the user to identify and point to using a pointing device <NUM> with the camera monitor <NUM>. Then, this procedure would be repeated one or more additional times with a new maneuver and the user pointing to the same feature at the new locations, shown at <NUM>.

Alternatively, the captured images can be analyzed by the controller <NUM> to select a feature in the first image and a feature in the second image and using feature recognition. The feature recognition may include defining a plurality of feature recognition points associated with the first feature and a plurality of feature recognition points associated with the first feature. Determining that the first feature and the second feature are the same object when the feature from the first image and the second image that have a sufficient number of matching recognition points.

To continue, after collection of the estimated position and heading information as well as corresponding trailer features from multiple vehicle positions and wheel motions and steering wheel angles as a function of time the motional calibration system can calculate a new set of motion parameters based on the collected data and store that information, shown at <NUM>. These steps provide a set of equations that can be simulated, i.e. integrated, (in numerical form) involving the imprecisely known vehicle parameters. The error in the integrated equations can be minimized (numerically) to improve upon the existing parameters and obtain more accurate ones. The problem can be solved with nonlinear least squares solver methods, for example, a quasi-Newton method (e.g. BFGS) can be used. The new motion calibration data can used to more accurately determine and predict the vehicle <NUM>. motion <NUM> based on the sensors system data <NUM> and the path planning system <NUM>. Therefore, the vehicle <NUM> can exit calibration mode, shown at <NUM>.

More specifically, we propose to optimize a function whose input is the set of parameters which aren't perfectly known (wheelbase, track width, tire diameter, steering ratio/offset, etc.) as well as the wheel motions and steering wheel angles as a function of time, and whose output is some measure of the error of the fixed points in the images (from different viewpoints).

As mentioned previously, the procedure needs to be repeated multiple times, depending on the number of parameters to be estimated. Each iteration provides <NUM> equations (associated with the <NUM> image point coordinates), and there are <NUM> + N unknowns, where N is the number of parameters to be determined. The "<NUM>" in "<NUM> + N" comes from the unknown world coordinates of the fixed point. So, we have <NUM> equations and <NUM> + N unknowns, and therefore, <NUM> must be at least <NUM> + N (to have as many equations as unknown parameters). Therefore the number of iterations is given by M in the following inequality: <MAT>.

For the case of optimizing <NUM> unknown parameter (say steering offset), we must do at least <NUM> iterations: (<NUM> + <NUM>)/<NUM>. For <NUM> unknown parameters (say wheelbase, track width, tire diameter, steering ratio, steering offset), we must do at least <NUM> iterations: (<NUM> + <NUM>)/<NUM>.

The motion calibration system may be used the first time the driver assistance system <NUM> is used and may also be updated at periodic intervals as suggested by the driver assistance system <NUM>, e.g. every <NUM> or <NUM> uses. Alternatively, it may be desirable to calibrate at user discretion, e. g when a new trailer is being used with hitch assist or the driver notices a change in the accuracy of performance by the vehicle <NUM>.

As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus and/or device (e.g., magnetic discs, optical disks, memory, Programmable Logic Devices (PLDs)) used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal.

Implementations of the subject matter and the functional operations described in this specification can be implemented in digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Moreover, subject matter described in this specification can be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a computer readable medium for execution by, or to control the operation of, data processing apparatus. The computer readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter effecting a machine-readable propagated signal, or a combination of one or more of them. The terms "data processing apparatus", "computing device" and "computing processor" encompass all apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers. A propagated signal is an artificially generated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal that is generated to encode information for transmission to suitable receiver apparatus.

In certain circumstances, multi-tasking and parallel processing may be advantageous.

A number of implementations have been described. Nevertheless, it will be understood that various modifications may be made.

Claim 1:
A method (<NUM>) of calibrating a driver assistance system (<NUM>) of a vehicle (<NUM>), the method (<NUM>) comprising:
entering a calibration mode for the driver assistance system (<NUM>);
determining with a controller (<NUM>) for the driver assistance system (<NUM>) a first estimated position and heading of the vehicle (<NUM>) at a first location;
capturing a first image from at least one camera (<NUM>, 410a, 410b, 410c, 410d) outwardly facing from the vehicle (<NUM>) and selecting at least one feature in the first image;
moving the vehicle (<NUM>) to at least a second location and heading and determining a second estimated position and heading of the vehicle (<NUM>);
capturing a second image from at least one camera (<NUM>, 410a, 410b, 410c, 410d) outwardly facing from the vehicle (<NUM>) and selecting at least one feature in the second image, wherein the at least one selected feature in the second image is the same feature as the at least one selected feature in the first image; and
calculating with the controller (<NUM>) new vehicle motion parameter values based on the first estimated position and heading, second estimated position and heading, first recorded at least one feature data, and second recorded at least one feature data, wherein the vehicle motion parameter values include at least one of: wheelbase, track width, tire diameter, steering offset and steering ratio,
and wherein moving the vehicle (<NUM>) includes: steering, braking, and accelerating.