Dynamic LIDAR sensor controller

A laser imaging, detection and ranging (LIDAR) system for an autonomous vehicle (AV) includes a LIDAR sensor comprising a plurality of configurable parameters, and a sensor controller. The sensor controller can execute sensor configuration logic to adjust one or more of the plurality of configurable parameters of the LIDAR sensor in response to AV feedback from a control system of the AV.

BACKGROUND

Automated or autonomous vehicles (AVs) may require continuous processing of sensor data provided by an on-board laser imaging, detection, and ranging (LIDAR) sensor system. For fixed-beam LIDAR systems, the granularity for detected objects, such as obstacles and potential road hazards, becomes increasingly coarser in relation to distance.

DETAILED DESCRIPTION

Current LIDAR technology include fixed-beam LIDAR systems that include laser sources, scanners, optical systems (e.g., a beam splitter), and photodetectors. For example, cutting edge LIDAR systems can include pulse rates on the order of one million pulses per second producing a detailed point cloud map of an AV's surroundings at ranges upwards of one hundred-plus meters. When using typical fixed-beam LIDAR systems for AVs traveling through road traffic, operational speed may be limited by the nature of the beam pattern produced by the LIDAR system. For example, in order to ensure safety for an AV traveling at 15 miles per hour (mph), the LIDAR system may require at least twelve separate beams to readily detect potential hazards with sufficient granularity and decelerate, maneuver, and/or stop the AV accordingly. However, when the AV travels at a very high speed (e.g., 60 mph, 75 mph, etc.), in order to achieve the same granularity for potential hazards in order to safely react, decelerate, and/or stop the AV, a fixed-beam LIDAR system may require well over seventy separate beams.

Increasing the number of fixed beams also places additional requirements for the LIDAR system. For example, the LIDAR will require more power, greater processing capability, larger or more sensitive photodetector and receiving equipment, constrained optics, and generally greater weight and more space. Furthermore, cost and waste quickly become an issue when increasing the number of fixed-beams, since the fixed-beam pattern or the fixed-beam LIDAR system must be tuned for a maximum operational speed of the AV. If AVs are going to operate safely with LIDAR technology on public highways at high speed, then alternative arrangements may be necessary to avoid spiraling costs, wasted power, additional equipment, and increased processing requirement.

To address the shortcomings of various fixed-beam LIDAR systems, a LIDAR sensor configuration system is provided with an adjustable-beam LIDAR sensor to control various adjustable parameters of the LIDAR sensor while an AV travels on a given road. The LIDAR configuration system can include a sensor controller that executes sensor configuration logic to adjust the configurable parameters in response to AV feedback from, for example, a control system of the AV. The configurable parameters of the LIDAR sensor can include a beam pattern (e.g., provided by a light source of the LIDAR), such as a vertical beam pattern that adjusts a vertical field of view of the LIDAR sensor. The configurable parameters can also include a rotational speed of the LIDAR system, a scan rate, a pulse rate, a beam frequency (e.g., a visible versus an infrared laser frequency), a photodetector sensitivity, and the like. The feedback data provided to the LIDAR configuration system can include a current speed of the AV, road conditions (e.g., type of road, road wetness, potholes, etc.), weather conditions (e.g., whether precipitation is detected), traffic conditions, pedestrian activity, road geometry (e.g., advance knowledge of road curves, gradients, etc. using a map or sensor data from the LIDAR itself) driving parameters (e.g., a turn rate, acceleration and/or braking of the AV), and the like.

According to examples described herein, the LIDAR configuration system can receive the feedback data from a control system of the AV, where the control system operates the steering, acceleration, and braking systems. Additionally or alternatively, the LIDAR configuration system can receive feedback as sensor data directly from a sensor array (e.g., LIDAR data from the LIDAR itself). In variations, the LIDAR configuration system can receive feedback data from an on-board computing system, such as a data processing system, of the AV. The LIDAR configuration system can respond to the feedback data by adjusting any number of the adjustable parameters of the LIDAR sensor.

For example, the LIDAR configuration system can dynamically adjust a vertical angular spacing between each beam based on the speed of the AV. Specifically, the LIDAR configuration system can dynamically increase the vertical angular spacing of the beams as the AV decreases speed, and dynamically decrease the vertical angular spacing of the beams as the AV increases speed. In many aspects, the LIDAR sensor system can include components having adjustable parameters, such as a rotational motor that can be adjusted to control a scan rate or horizontal sweep rate of the beam pattern, and mirror actuators that can be adjusted to control the vertical field of view or vertical sweep of the beam pattern. Each of these components can be dynamically adjusted by the LIDAR configuration system in response to the feedback from the AV.

As another example, precipitation can significantly reduce the effectiveness of the LIDAR sensor to detect potential road hazards. As such, the AV control system may operate the AV in a high-caution mode, reducing speed, increasing processing power, and maintaining large gaps between the AV and any potential hazard. In certain implementations, the LIDAR configuration system can also operate in a high-caution mode by, for example, increasing a scan rate, and adjusting the horizontal and vertical sweep pattern of the LIDAR sensor to provide more detailed data for the AV's on-board data processing system.

In some aspects, the optics of the LIDAR may be such that a general vertical angle of the beams may be adjusted, in addition to the vertical spacing between the beams. For example, the feedback data can include a road gradient of the current road traveled by the AV (e.g., a mapping resource can indicate that the road immediately in front of the AV curves upwardly). The LIDAR configuration system can compensate for the upward gradient of the road by adjusting the general vertical beam angle of the LIDAR sensor. That is, in addition to adjusting the angular spacing between beams, every beam may be also angled to adjust for the forward road gradient indicated in the feedback data. In certain implementations, the general angle may be adjusted to align with the detected angle, and may be limited based on the speed of the AV.

For example, National Highway Traffic and Safety Administration (NHTSA) regulations exist for public roads that correlate road speed with road gradient and gradient transitions. For example, the greater the speeds, the lower the gradient transition or curve of the road. According to certain aspects, the LIDAR configuration dynamically set boundaries for the general vertical angle of the beam pattern based on the speed of the AV, and in accordance with the NHTSA regulations, since the LIDAR configuration system can expect, with certainty, that the gradient will not increase or decrease beyond a certain rate.

In addition to the adjustable-beam LIDAR systems described herein, a fixed-beam LIDAR system is disclosed having optimized, uneven beam spacing for a wide variety of travel conditions. The fixed-beam LIDAR system can be calibrated to provide optimal beam spacing for short, medium, and long distance range, all in one set of lasers. In one example, the beam angles can progressively decrease or converge along the length of the laser configuration (e.g., from the bottom to top). In variations, the beam angles can be optimally calibrated and configured individually. For example, the beam angles can be individually calibrated based on distance, anticipated speeds, vehicle dynamics, typical road gradients, and the like, and can comprise uneven beam spacing for optimal use in a wide variety of environments (e.g., urban scenarios and well as open, rural roads). In further variations, a combination of fixed-beam and adjustable beam laser configuration can be implemented in LIDAR sensors described herein.

Among other benefits, the examples described herein achieve a technical effect of providing adjustability for LIDAR sensors in order to increase data quality, reduce costs, reduce processing requirements, and reduce the number of beams necessary for typical road travel for AVs.

As used herein, the AV's LIDAR system implements remote sensing using laser beams, which can include diode lasers, fiber lasers, and the like. “LIDAR” is used herein as a representation of any light detection and ranging systems utilized on an AV. Such systems may be referred to as “LIDAR” or “LADAR” systems. For the sake of brevity, “LIDAR” is used throughout to represent any of such systems should distinctions be made in the common nomenclature for future reference.

As used herein, a computing device refer to devices corresponding to desktop computers, cellular devices or smartphones, personal digital assistants (PDAs), field programmable gate arrays (FPGAs), laptop computers, tablet devices, television (IP Television), etc., that can provide network connectivity and processing resources for communicating with the system over a network. A computing device can also correspond to custom hardware, in-vehicle devices, or on-board computers, etc. The computing device can also operate a designated application configured to communicate with the network service.

Some examples described herein can generally require the use of computing devices, including processing and memory resources. For example, one or more examples described herein may be implemented, in whole or in part, on computing devices such as servers, desktop computers, cellular or smartphones, personal digital assistants (e.g., PDAs), laptop computers, printers, digital picture frames, network equipment (e.g., routers) and tablet devices. Memory, processing, and network resources may all be used in connection with the establishment, use, or performance of any example described herein (including with the performance of any method or with the implementation of any system).

System Description

FIG. 1is a block diagram illustrating an example AV100including a LIDAR sensor configuration module135, as described herein. The AV100can include an adjustable-beam LIDAR sensor105that can provide LIDAR data102to an on-board data processing system110of the AV100. In some examples, the LIDAR sensor105can comprise a light source (e.g., a laser), a photodetector, scanner components (e.g., which can include a mirror(s), one or more motor(s), and one or more actuator(s)), and circuitry to couple to various components of the AV100. The data processing system110can utilize the LIDAR data102to detect the situational conditions of the AV100as the AV100travels along a current route. For example, the data processing system110can identify potential obstacles or road hazards—such as pedestrians, bicyclists, objects on the road, road cones, road signs, animals, etc.—in order to enable an AV control system120to react accordingly.

In certain implementations, the data processing system110can utilize localization maps133stored in a database130of the AV100in order to perform localization and pose operations to determine a current location and orientation of the AV100in relation to a given region (e.g., a city). The localization maps133can comprise previously recorded sensor data, such as stereo camera data, radar maps, and/or LIDAR maps that enable the data processing system110to compare the LIDAR data102from the LIDAR sensor105with a current localization map134to identify such obstacles and potential road hazards in real time. The data processing system110can provide the processed sensor data113—identifying such obstacles and road hazards—to AV control system120, which can react accordingly by operating the steering, braking, and acceleration systems125of the AV100.

In many implementations, the AV control system120can receive a destination119from, for example, an interior interface system115of the AV100. The interior interface system115can include any number of touch-screens or voice sensors that enable a passenger139to provide a passenger input141indicating the destination119. For example, the passenger139can type the destination119into a mapping engine175of the AV100, or can speak the destination119into the interior interface system115. Additionally or alternatively, the destination119can be received by the AV100as a communication from a backend system that manages routes for a fleet of AVs. The backend system can be operative to facilitate passenger pick-ups and drop-offs to generally service pick-up requests, facilitate delivery such as packages, food, goods, or animals, and the like.

Based on the destination119(e.g., a pick-up location), the AV control system120can utilize the mapping engine175to receive route data132indicating a route to the destination119. In variations, the mapping engine175can also generate map content126dynamically indicating the route traveled to the destination119. The route data132and/or map content126can be utilized by the AV control system120to maneuver the AV100to the destination119along the selected route. For example, the AV control system120can dynamically generate control commands121for the AV's steering, braking, and acceleration system125to actively drive the AV100to the destination119along the selected route. Optionally, the map content126showing the current route traveled can be streamed to the interior interface system115so that the passenger(s)139can view the route and route progress in real time.

In many examples, while the AV control system120operates the steering, braking, and acceleration systems125along the current route on a high level, the processed data113provided to the AV control system120can indicate low level occurrences, obstacles, and potential hazards to which the AV control system120can react. For example, the processed data113can indicate a pedestrian crossing the road, traffic signals, stop signs, other vehicles, road conditions, traffic conditions, bicycle lanes, crosswalks, pedestrian activity (e.g., a crowded adjacent sidewalk), and the like. The AV control system120can respond to the processed data113by generating control commands121to reactively operate the steering, braking, and acceleration systems125accordingly.

According to examples described herein, the AV100can include a LIDAR configuration module135to receive AV feedback data123from the AV control system120in order to configure various adjustable parameters of the LIDAR sensor105. The AV feedback data123can include data indicating the current speed of the AV100, any of the described obstacles and/or potential hazards, weather conditions identified by the data processing system110(e.g., rain or snow), forward road features (e.g., an imminent gradient of the road), traffic conditions, a turn rate and/or an acceleration rate, and the like.

The LIDAR configuration module135can respond to the AV feedback data123by adjusting one or more adjustable parameters of the LIDAR sensor105. For example, the LIDAR configuration module135can generate configuration commands138in response to the AV feedback data123to adjust a rotational parameter109of the LIDAR sensor105(e.g., the rotational speed of the motor), a vertical field of view (VFOV) parameter101of the LIDAR sensor105, a number of emitted LIDAR beams107by the LIDAR sensor105, and/or a beam spacing103or angular spacing between the LIDAR beams107themselves.

One or more components of the LIDAR sensor105can comprise non-mechanical aspects that cause the LIDAR beams107to adjust their beam angles in response to autonomous driving characteristics by the AV control system120, such as vehicle velocity, acceleration, braking inputs, steering inputs, and the like. In one aspect, the angle of the LIDAR beam107may be wholly adjusted non-mechanically, or may be adjusted through a combination of mechanical and non-mechanical features. In variations, the LIDAR configuration module135can generate configuration commands138that are executable on mechanical components of the LIDAR sensor105to adjust the beam spacing103of the LIDAR beams107in response to the AV feedback data123.

As an example, when the AV100is traveling at low speeds, the LIDAR configuration module135can dynamically increase the angular beam spacing103between the LIDAR beams107since (i) the reaction and stopping distances are much lower at low speeds, and (ii) an increased VFOV may necessary to detect objects close to the AV100. Conversely, when the AV100accelerates to higher speeds, the LIDAR configuration module135can dynamically narrow the VFOV parameter101, decreasing the angular beam spacing103, since (i) the reaction and stopping distances increase, thereby requiring finer granularity in the LIDAR data102to detect objects further down the road, (ii) a decreased field of view may be suitable for increased speeds since more proximate objects can be detected earlier, and (iii) at higher speeds, NHTSA guidelines specify road geometries that make decreased field of view suitable. Detailed discussion is provided with regard to the LIDAR configuration module135with respect toFIG. 2below.

FIG. 2is a block diagram illustrating an example LIDAR sensor configuration system200utilized in connection with an adjustable-beam LIDAR sensor210, as described herein. The LIDAR configuration system200may be implemented as a component of the AV100described in connection withFIG. 1. Furthermore, the LIDAR configuration system200shown and described with respect toFIG. 2, can include the same or similar functionality as the LIDAR configuration module135shown and described with respect toFIG. 1. Referring toFIG. 2, the LIDAR configuration system200can include an AV control interface285to receive the AV feedback data230from the AV control system220. As described herein, the AV feedback data230can include various aspects of the AV's speed232, the road conditions234(e.g., road gradient, wet versus dry conditions, lane count, etc.), traffic conditions236(e.g., light, moderate, or heavy traffic), detected hazards238(e.g., identified pedestrians, bicyclists, road objects, etc.), and/or driving parameters239(e.g., acceleration rate, braking rate, and/or turning rate).

Furthermore, as provided herein, the LIDAR configuration system200is shown as a standalone module for illustrative purposes. However, various functions of the LIDAR configuration system200may be performed by separate processing components of the AV100itself. For example, one or more of the data processing system110, the AV control system120, or one or more sensor processor(s) contained within the sensor system itself (e.g. the adjustable beam LIDAR sensor210shown inFIG. 2) can perform any number of functions or actions described in connection with the LIDAR configuration system200. Additionally, any of the various commands transmitted between the blocks shown inFIG. 2may be transmitted and received via a computer network either wirelessly or via wired communication.

The AV control interface285can provide the AV feedback data230to a configuration optimizer270which can process the AV feedback data230to optimize the LIDAR configurations for the LIDAR sensor210accordingly. In certain examples, the configuration optimizer270can execute configuration logic271to perform a lookup272in a number of lookup tables (LUTs275) to select an optimal set of configurations277from any number of LIDAR configurations279logged or chronicled in the LUTs275. In variations, the configuration optimizer270can execute the configuration logic271to dynamically determine the optimal set of configurations277to be executed by a LIDAR controller250of the LIDAR configuration system200. The dynamically determined configuration sets277can consequently be dynamically executed by the LIDAR controller250to generate the configuration commands252that actively adjust the configurable parameters of the LIDAR sensor210.

As provided herein, the dynamically executed set of configurations277can cause the LIDAR controller to generate configuration commands252that operate on adjustable parameters of the LIDAR sensor210, such as a rotational motor212that controls a rotational rate of the LIDAR beams and/or a scan rate of the LIDAR scanner218. The LIDAR controller250can further generate configuration commands252that adjust a pulse rate and/or frequency of the laser by tuning a laser source216of the LIDAR sensor210. For example, the LIDAR controller250can increase or decrease power to the laser source216, increase or decrease the pulse rate (e.g., to increase or decrease granularity of the point cloud), and/or modulate the frequency of the beams themselves (e.g., modifying the reflectance parameters of the LIDAR sensor210).

In many aspects, the LIDAR controller250can generate configuration commands252to operate mirror actuators214of the LIDAR sensor210which, in turn, can adjust the VFOV of the LIDAR sensor210. Specifically, the LIDAR controller250can increase or decrease the VFOV of the LIDAR sensor210by operating the mirror actuators214in response to the speed of the AV. In some aspects, the mirror actuators214can split the emitted beams between positive VFOV beams, which detect the AV environment above a sensor plane parallel to the road, and negative VFOV beams, which detect the AV environment below the sensor plane. In response to the AV feedback data230the configuration optimizer270may generate and/or select a configuration set277can causes the LIDAR controller250to adjust a vertical beam pattern of the LIDAR sensor210for the positive VFOV beams differently in comparison to the negative VFOV beams. For example, when the AV is stopped at a stop light in a dense pedestrian environment, the configuration optimizer270may select a more spread out negative VFOV beam pattern to identify potential hazards within a much broader VFOV (e.g., a child standing next to the AV).

Examples described herein are not limited to mirror embodiments having mirror actuators. It is contemplated that VFOV adjustments can be made with adjustable beam splitters, directional laser apertures, or adjustable dual oscillating mirrors and/or polygonal mirrors. As an example, an adjustable laser grating of a LIDAR sensor210can be configured to dynamically adjust the vertical sweep of the beam pattern by compressing or spreading the angular spacing of the beams.

The LIDAR sensor210can further include a scanner and optics system218which can be configurable by the LIDAR controller250. For example, the configuration optimizer270can select a set of configurations277that cause the scanner and optics system218to increase a scan rate in response to detecting precipitation. The increased scan rate can be executed in conjunction with, for example, the AV control system operating in a high-caution mode. Additionally, the LIDAR sensor210can include a photodetector219which, in certain aspects, can be voltage-adjustable for increased or decreased sensitivity.

Examples described herein improve upon current LIDAR technology by providing a LIDAR configuration system200that can dynamically configure the adjustable parameters of the AV's LIDAR sensor210in response to AV feedback230received from the AV control system220and/or other subsystems of the AV. In many aspects, the LIDAR configuration system200can dynamically adjust a vertical sweep pattern of the LIDAR sensor210by adjusting the angle, or the angular spacing between the beams. By adjusting the VFOV of the LIDAR sensor210, the LIDAR configuration system200can require less beams than current fixed-beam systems, reducing costs and increasing data quality for the AV's on-board data processing system.

Methodology

FIG. 3is a high level flow chart describing an example method of dynamically configuring a LIDAR sensor. In the below discussion ofFIG. 3, reference may be made to like reference characters representing like features as shown and described with respect toFIGS. 1 and 2. For example, the high level method described with respect toFIG. 3may be performed by an example LIDAR configuration module135shown and described with respect toFIG. 1, or the LIDAR configuration system200shown and described with respect toFIG. 2. Referring toFIG. 3, the LIDAR configuration system200can receive AV data230from subsystems of the AV100, such as the AV control system120or the data processing system110(300). The AV data230can include information such as the AV's speed (305), and road conditions (310) which can indicate potential hazards as the AV100travels along a current route.

In response to the AV data230, the LIDAR configuration system200can dynamically configure the adjustable parameters of the LIDAR sensor system210of the AV100(315). For example, the LIDAR configuration system200can dynamically adjust an angular beam spacing between the beams to control a VFOV (320) based on the speed of the AV100. As another example, the LIDAR configuration system200can control a scan rate of the LIDAR sensor system210in response to the road conditions (325).

FIG. 4is a low level flow chart describing an example method of configuring a LIDAR sensor, as described herein. In the below discussion ofFIG. 4, reference may be made to like reference characters representing like features as shown and described with respect toFIGS. 1 and 2. For example, the low level method described with respect toFIG. 4may be performed by an example LIDAR configuration module135shown and described with respect toFIG. 1, or the LIDAR configuration system200shown and described with respect toFIG. 2. Referring toFIG. 4, the LIDAR configuration system200can receive AV feedback data230from the AV control system120(400). The AV feedback data230from the AV control system120can include the current AV speed (402) and/or driving parameters of the AV100(404).

In certain implementations, the AV feedback data230can also be received from the data processing system110of the AV100(405). This data may include potential hazards on the road (407) and/or precipitation data (409). Based on the received feedback data230from the AV control system120and the on-board data processing system110, the LIDAR configuration system200can dynamically perform a lookup272in a set of LUTs275or perform an optimization to select a set of configurations277for the LIDAR sensor system210(410). The LIDAR configuration system200may then dynamically execute the set of configurations277on the LIDAR system210(415).

In many aspects, execution of the configuration set277causes the LIDAR configuration system200to adjust a VFOV or the beams angles of the LIDAR sensor system210(417). Additionally or alternatively, the configuration set277can cause the LIDAR configuration system200to adjust a scan rate (419) and/or a beam count (416) for the LIDAR sensor system210. In still other aspects, the LIDAR configuration system200can adjust a general vertical angle of the beams in response to detecting a road gradient (418).

Hardware Diagram

FIG. 5is a block diagram that illustrates a computer system upon which examples described herein may be implemented. A computer system500can be implemented on, for example, a server or combination of servers. For example, the computer system500may be implemented as part of a LIDAR configuration system135, which itself may be implemented as a part of the AV's on-board data processing system110. In the context ofFIG. 1, the LIDAR configuration system135may include a sensor controller that executes sensor configuration logic or instructions, and can be implemented using a computer system such as described byFIG. 5. The LIDAR configuration system135may also be implemented using a combination of multiple computer systems as described in connection withFIG. 5.

In one implementation, the computer system500includes processing resources510, a main memory520, a read-only memory (ROM)530, a storage device540, and a communication interface550. The computer system500includes at least one processor510for processing information stored in the main memory520, such as provided by a random access memory (RAM) or other dynamic storage device, for storing information and instructions which are executable by the processor510. The main memory520also may be used for storing temporary variables or other intermediate information during execution of instructions to be executed by the processor510. The computer system500may also include the ROM530or other static storage device for storing static information and instructions for the processor510. A storage device540, such as a magnetic disk or optical disk, is provided for storing information and instructions.

The communication interface550enables the computer system500to communicate with one or more AV subsystems580over a network link (e.g., a wireless or wired link). In accordance with examples, the computer system500receives feedback data582from the AV subsystems580. The executable instructions stored in the memory530can include configuration instructions522, which the processor510executes to determine a set of configurations to configure the adjustable parameters of the AV's LIDAR sensor system210based on the feedback data582.

The processor510is configured with software and/or other logic to perform one or more processes, steps and other functions described with implementations, such as described byFIGS. 1 through 4, and elsewhere in the present application.

Examples described herein are related to the use of the computer system500for implementing the techniques described herein. According to one example, those techniques are performed by the computer system500in response to the processor510executing one or more sequences of one or more instructions contained in the main memory520. Such instructions may be read into the main memory520from another machine-readable medium, such as the storage device540. Execution of the sequences of instructions contained in the main memory520causes the processor510to perform the process steps described herein. In alternative implementations, hard-wired circuitry may be used in place of or in combination with software instructions to implement examples described herein. Thus, the examples described are not limited to any specific combination of hardware circuitry and software.

It is contemplated for examples described herein to extend to individual elements and concepts described herein, independently of other concepts, ideas or system, as well as for examples to include combinations of elements recited anywhere in this application. Although examples are described in detail herein with reference to the accompanying drawings, it is to be understood that the concepts are not limited to those precise examples. As such, many modifications and variations will be apparent to practitioners skilled in this art. Accordingly, it is intended that the scope of the concepts be defined by the following claims and their equivalents. Furthermore, it is contemplated that a particular feature described either individually or as part of an example can be combined with other individually described features, or parts of other examples, even if the other features and examples make no mentioned of the particular feature. Thus, the absence of describing combinations should not preclude claiming rights to such combinations.