Patent Description:
Accurate and consistent obstacle detection and navigation are key elements of autonomous driving applications. Typically, an autonomous vehicle utilizes various on-board sensors to detect obstacles, other aspects of the roadway, and/or other aspects of an environment around the vehicle. Examples of such sensors include, for example, one or more of vision sensors (e.g., camera(s)), radio detection and ranging (i.e., radar) sensors, and/or light detection and ranging (i.e., LiDAR) sensors. While it is possible for only one type of sensor to be utilized, it is far more preferable to fuse data from different types of sensors so as to provide the autonomous vehicle's control systems with more accurate, complete, and dependable information.

In order for sensor fusion to provide desired outputs, the individual sensors must be calibrated prior to usage of the autonomous vehicle and, over time, recalibrated to assure accurate results. In the past, each sensor modality has been calibrated separately, often using separate calibration targets optimized for each modality. Due to the low-resolution, high-variance nature of radar, radar calibration tolerances are much larger than those of other modalities. When fusion of low-level sensor data is desired, the larger variance nature of radar may lead to a mismatch in detection pairing with other sensor types, such as vision sensors. Thus, in order to utilize low-level sensor data for pairing, radar and vision sensors should ideally be calibrated simultaneously using a common target. However, vision sensor calibration in autonomous vehicles has often relied upon the use of a large, planar checkerboard pattern as the calibration target. These large checkerboard patterns are not suitable for calibration of radar sensors, as they result in a high variance radar signature and may create multipath patterns, leading to inaccurate calibration of the radar sensors.

A joint sensor calibration target for joint calibration of vision sensors and detection and ranging sensors is disclosed in<NPL>. The detection and ranging sensors are LiDAR sensors.

<CIT> discloses a sensor calibration target in the form of a trihedral radar reflector having three mutually perpendicular surfaces bounded outwardly by circular segments running along the equator and two mutually perpendicular meridians of an imaginary sphere, respectively.

<CIT> discloses a trihedral radar reflector for maritime navigation, having three mutually perpendicular surfaces bounded outwardly by circular segments running along the equator and two mutually perpendicular meridians of a hard foam sphere, respectively, wherein the trihedral radar reflector forms a cutout portion of the sphere.

<NPL>, discloses a trihedral radar reflector having three mutually perpendicular surfaces bounded outwardly by circular segments running along the equator and two mutually perpendicular meridians of a aluminum sphere, respectively, wherein the trihedral radar reflector forms a cutout portion of the sphere.

<CIT> discloses a joint sensor calibration target for joint calibration of vision sensors and radar sensors of a vehicle. The calibration target contains two reference features which are spaced from each other. One of the reference features is suited for calibration of vision sensors, and the other reference feature is suited for calibration of radar sensors and may be a triple mirror.

Accordingly, there is a need for a calibration target capable of simultaneously providing both radar and vision sensor calibration without interfering with the other's sensing modality and having other advantages.

The problems described above are addressed by the subject-matters of independent claims <NUM>, <NUM> and <NUM>.

As used in this document, the singular forms "a," "an," and "the" include plural references unless the context clearly dictates otherwise. As used in this document, the term "comprising" means "including, but not limited to. " Definitions for additional terms that are relevant to this document are included at the end of this Detailed Description.

Referring to <FIG>, an autonomous vehicle calibration system <NUM> in accordance with an aspect of the disclosure is shown. Calibration system <NUM> includes an autonomous vehicle <NUM>, which may capable of fully-autonomous or semi-autonomous operation. While illustrated as a conventional passenger car, it is to be understood that autonomous vehicle <NUM> may be configured as any appropriate automated device, e.g., a car, truck, van, train, aircraft, aerial drone and the like.

Autonomous vehicle <NUM> includes a plurality of sensor types used in the gathering of information used in vehicle navigation, obstacle avoidance, etc. Specifically, one or more cameras <NUM> may be provided, as well as one or more radar sensors <NUM>. Additionally, in some embodiments, one or more LiDAR sensors <NUM> may also be present. While not shown in <FIG>, it is to be understood that the camera(s) <NUM>, radar sensor(s) <NUM>, and/or LiDAR sensor(s) <NUM> are electrically coupled to an on-board processor configured to perform calculations and logic operations required to execute programming instructions. In this way, data from the camera(s) <NUM>, radar sensor(s) <NUM>, and/or LiDAR sensor(s) <NUM> may be combined so as to provide the autonomous vehicle's control systems with more accurate, complete, and dependable information.

Calibration system <NUM> also includes a calibration target <NUM>. As will be described in further detail below, calibration target <NUM> is configured as a single target capable of providing joint calibration of the camera(s) <NUM> and radar sensor(s) <NUM>. Calibration target <NUM> may be mounted upon a post <NUM> or other structure capable of elevating calibration target <NUM> above the ground surface <NUM> and in the field-of-view of both the camera(s) <NUM> and radar sensor(s) <NUM>. However, it is to be understood that calibration target <NUM> may be elevated above the ground surface <NUM> using any appropriate means and/or at any appropriate height. Furthermore, in some embodiments, during a calibration process, the autonomous vehicle <NUM> may be positioned such that both the camera(s) <NUM> and radar sensor(s) <NUM> are positioned at a distance of <NUM>-to-<NUM> meters away from the calibration target <NUM>. However, it is to be understood that the distance between the camera(s) <NUM> and radar sensor(s) <NUM> is not limited to <NUM>-to-<NUM> meters, and the calibration target <NUM> may vary based on, e.g., the size of the calibration target <NUM>, the position of the calibration target <NUM>, the positions of the camera(s) <NUM> and radar sensor(s) <NUM> on the autonomous vehicle <NUM>, etc..

Next, referring to <FIG>, a detailed view of calibration target <NUM> in accordance with an aspect of the disclosure is shown. As noted above, calibration target <NUM> is configured as a single target capable of aiding the calibration of both vision and radar sensors. First, calibration target <NUM> includes a substantially spherical portion <NUM>, which may be formed of one or more appropriate non-metallic material(s) having a low radar signature (e.g., less than <NUM> dB/m<NUM>). Such appropriate non-metallic material(s) may be, e.g., polystyrene, polypropylene, polyvinyl chloride (PVC), polyamide, polycarbonate (PC), polycarbonate and polybutylene terephthalate blend (PC-PBT), acrylonitrile butadiene styrene (ABS), acrylonitrile styrene acrylate (ASA), etc. Additionally, spherical portion <NUM> is visually opaque, may be painted or otherwise colored, and may have any suitable diameter, such as, e.g., <NUM> inches (<NUM>). In this way, spherical portion <NUM> provides for an ideal target for the calibration of camera(s) <NUM>, as the spherical shape and/or colored surface enables a low-variance vision angle and range estimate for the camera(s) <NUM>. Additionally, the spherical portion <NUM> does not interfere in any substantial respect with returns from the radar sensor(s) <NUM>, as the material (e.g., polystyrene, PVC, PC, etc.) has a low radar signature, and the spherical shape and relatively small diameter resists reflection of the electromagnetic waves from the radar sensor(s) <NUM>.

Within substantially spherical portion <NUM> is a cutout portion <NUM>, with cutout portion <NUM> accounting for approximately <NUM>/<NUM> of the overall volume of the calibration target <NUM>. A trihedral reflector <NUM> is provided within the cutout portion <NUM>, wherein the trihedral reflector <NUM> acts as a corner reflector capable of generating a strong radar echo for use in calibration of the radar sensor(s) <NUM>. More specifically, the trihedral reflector <NUM> includes three electrically-conductive surfaces 36A, 36B, 36C mounted at a <NUM>° angle relative to one another, allowing incoming electromagnetic waves from the radar sensor(s) <NUM> shown in <FIG> to be accurately backscattered in the direction from which they came.

Accordingly, even with a relatively small size, the trihedral reflector <NUM> provides a strong radar echo, particularly when compared with other surfaces (e.g., spheres, planar surfaces, etc.) of similar size. For example, a trihedral reflector <NUM> in which the length R of each opposite and adjacent side of triangular surfaces 36A, 36B, 36C is approximately <NUM> inches (<NUM>) may provide for a stable ~<NUM> dB/m<NUM> radar signal, thereby allowing for low-variance angle detection and a large radar field-of-view, particularly given the relatively small overall size of the trihedral reflector <NUM>. Additionally, the cutout portion <NUM> and trihedral reflector <NUM> sized and positioned such that they do not greatly interfere with the ability of the camera(s) <NUM> to calibrate using the spherical portion <NUM>.

Referring still to <FIG>, the trihedral reflector <NUM> is shown as an insert secured within the cutout portion <NUM>. In some embodiments, trihedral reflector <NUM> is a solid metallic insert, preformed and then placed into (i.e., completely embedded within) cutout portion <NUM>, leaving portions of the internal surfaces 37A, 37B, 37C of the cutout portion <NUM> uncovered. Each triangular surface 36A, 36B, 36C may be of equal size and shape.

However, as shown in <FIG>, and in accordance with another aspect of the disclosure, it is to be understood that the trihedral reflector may instead be formed of a metallic coating applied to the internal surfaces of the cutout portion <NUM>, thereby leaving no surfaces of the cutout portion <NUM> uncovered. Specifically, referring to <FIG>, a calibration target <NUM> in accordance with one embodiment is shown, with the cutout portion of spherical portion <NUM> having metallically coated surfaces 46A, 46B, 46C to form a trihedral reflector <NUM>. The metallic coating may be applied via any appropriate method, such as spray coating, adhesive coating, etc. The thickness of each surface 46A, 46B, 46C of the trihedral reflector <NUM> may be any appropriate thickness (e.g., about <NUM> microns) capable of the reflection of electromagnetic waves. Furthermore, each surface 46A, 46B, 46C should be non-porous, thereby allowing for accurate backscatter of the electromagnetic waves.

Referring again to <FIG>, while the length R of each opposite and adjacent side of triangular surfaces 36A, 36B, 36C is described above as being <NUM> inches (<NUM>) (i.e., equal to the radius of the spherical portion <NUM>), it is to be understood that the length of each surface 36A, 36B, 36C may be longer or shorter. In such an instance, the change x-y-z position of each surface 36A, 36B, 36C relative to the sphere center would need to be accounted for such that the calibration of the radar sensor(s) and the calibration of the camera(s) remains consistent.

Next, referring to both <FIG> and <FIG>, calibration target <NUM> further includes a base member <NUM>, which allows calibration target <NUM> to be mounted in a desired location (e.g., upon post <NUM>, as shown in <FIG>). As shown in <FIG>, base member <NUM> may include an inwardly-directed stem portion <NUM>, along with a circular face portion <NUM>, wherein the stem portion <NUM> is configured to extend within the spherical portion <NUM> so as to secure the base member <NUM> and allow for coupling at a mounting location. In some embodiments, the base member <NUM> is formed of a material (or materials) which have a low radar signal, such as, e.g., polypropylene, polyamide, PVC, PC, PC-PBT, ABS, ASA, etc. In this way, the base member <NUM> is essentially transparent to the radar signal and does not alter the calibration of the radar sensor(s). At the very least, the radar signal of the base member <NUM> should be low enough that it is essentially rendered moot relative to the high radar signal provided by the trihedral reflector <NUM>.

Additionally, in accordance with another aspect of the disclosure, the trihedral reflectors <NUM>, <NUM> shown and described above with respect to <FIG> may be angularly positioned within a cutout of the spherical portion <NUM> relative to the base member <NUM> such that each surface of the trihedral reflector <NUM>, <NUM> has equal exposure from a line-of-sight aligned with the shared corner of the three surfaces and parallel to the ground, similar to that which is shown in <FIG>. With such a configuration, the calibration target may be more tolerant of misalignment during calibration set-up.

<FIG> depicts an example of internal hardware that may be included in any of the electronic components of the calibration system, such as internal processing systems, external monitoring and reporting systems, or remote servers. An electrical bus <NUM> serves as an information highway interconnecting the other illustrated components of the hardware. Processor <NUM> is a central processing device of the system, configured to perform calculations and logic operations required to execute programming instructions. As used in this document and in the claims, the terms "processor" and "processing device" may refer to a single processor or any number of processors in a set of processors that collectively perform a set of operations, such as a central processing unit (CPU), a graphics processing unit (GPU), a remote server, or a combination of these. Read only memory (ROM), random access memory (RAM), flash memory, hard drives and other devices capable of storing electronic data constitute examples of memory devices <NUM>. A memory device may include a single device or a collection of devices across which data and/or instructions are stored. Various embodiments of the invention may include a computer-readable medium containing programming instructions that are configured to cause one or more processors, print devices and/or scanning devices to perform the functions described in the context of the previous figures.

An optional display interface <NUM> may permit information from the bus <NUM> to be displayed on a display device <NUM> in visual, graphic or alphanumeric format. An audio interface and audio output (such as a speaker) also may be provided. Communication with external devices may occur using various communication devices <NUM> such as a wireless antenna, an RFID tag and/or short-range or near-field communication transceiver, each of which may optionally communicatively connect with other components of the device via one or more communication system. The communication device(s) <NUM> may be configured to be communicatively connected to a communications network, such as the Internet, a local area network or a cellular telephone data network.

The hardware may also include a user interface sensor <NUM> that allows for receipt of data from input devices <NUM> such as a keyboard, a mouse, a joystick, a touchscreen, a touch pad, a remote control, a pointing device and/or microphone. Digital image frames also may be received from one or more cameras <NUM> that can capture video and/or still images. The system also may receive data from a motion and/or position sensor <NUM> such as an accelerometer, gyroscope or inertial measurement unit. The system also may receive data from a radar system <NUM> and/or a LiDAR system <NUM> such as that which was described above.

The above-disclosed features and functions, as well as alternatives, may be combined into many other different systems or applications. Various components may be implemented in hardware or software or embedded software.

Terminology that is relevant to the disclosure provided above includes;.

The term "vehicle" refers to any moving form of conveyance that is capable of carrying either one or more human occupants and/or cargo and is powered by any form of energy. The term "vehicle" includes, but is not limited to, cars, trucks, vans, trains, autonomous vehicles, aircraft, aerial drones and the like. An "autonomous vehicle" is a vehicle having a processor, programming instructions and drivetrain components that are controllable by the processor without requiring a human operator. An autonomous vehicle may be fully autonomous in that it does not require a human operator for most or all driving conditions and functions, or it may be semi-autonomous in that a human operator may be required in certain conditions or for certain operations, or that a human operator may override the vehicle's autonomous system and may take control of the vehicle.

Claim 1:
A joint sensor calibration target (<NUM>) for joint calibration of vision and radar sensors comprising:
a spherical portion (<NUM>) which comprises one or more non-metallic materials;
a cutout portion (<NUM>), wherein the cutout portion (<NUM>) is formed within the spherical portion (<NUM>) and comprises three equal surfaces (<NUM>); and
a trihedral reflector (<NUM>) positioned within the cutout portion (<NUM>).