System and method for detecting errors and improving reliability of perception systems using logical scaffolds

An artificial intelligence perception system for detecting one or more objects includes one or more processors, at least one sensor, and a memory device. The memory device includes an image capture module, an object identifying module, and a logical scaffold module. The image capture module and the object identifying module cause the one or more processors to obtain sensor information of a field of view from a sensor, identify an object within the sensor information, and determine at least one property of the object. The logical scaffold module causes the one or more processors to determine, by a logical scaffold, when the at least one property of the object as determined by the object identifying module is one of a true condition or a false condition.

TECHNICAL FIELD

The subject matter described herein relates, in general, to systems and methods for detecting errors and improving reliability of perception systems.

BACKGROUND

The background description provided is to present the context of the disclosure generally. Work of the inventor, to the extent it may be described in this background section, and aspects of the description that may not otherwise qualify as prior art at the time of filing, are neither expressly nor impliedly admitted as prior art against the present technology.

Recent progress in artificial intelligence (“AI”) has led to possible deployment in a wide variety of domains. Current AI programs differ from traditional programs in their reliance on data. The specification, input-output semantics, and executable generation procedure are all data-driven.

Unlike traditional software development, in AI programs, a specification is not formally articulated. Indeed, in many of the most promising recent applications of AI, such as vision and human intent prediction, it is not feasible to write a formal specification. Instead, an implicit specification is provided via a test set, and the goal is to achieve a certain performance over the test set. Traditional software development specifies the input-output semantics of the program in a programming language. In AI programs, the engineer provides a training dataset, and the program must match the input-output statistics of the dataset.

As such, like other AI programs, AI perception systems, such as those used with automobiles and elsewhere, generally do not have a formal logic specification. Some of this is because one generally cannot define an object to be detected in a perception system by using the object itself. For example, suppose one is given a perception system that detects stop signs. One cannot use formal logic to express the property “if this image contains a stop sign, then the detector should flag a stop sign.” The reason is that one cannot formally encode what it means for an image to contain a stop sign. If this were possible, there would be no need for neural-network-based perception systems.

Instead, the perception system utilizes a number of different algorithms, including artificial intelligence-based algorithms that are used to detect and classify objects based on images or other information provided to the perception system from one or more sensors. However, the algorithms used to detect and classify objects may make errors regarding both detection and/or classification of the objects.

SUMMARY

This section generally summarizes the disclosure and is not a comprehensive explanation of its full scope or all its features.

In one embodiment, an artificial intelligence perception system for detecting one or more objects includes one or more processors, at least one sensor, and a memory device. The memory device includes an image capture module, an object identifying module, and a logical scaffold module. The image capture module and the object identifying module cause the one or more processors to obtain sensor information of a field of view from a sensor, identify an object within the sensor information, and determine at least one property of the object. The logical scaffold module causes the one or more processors to determine, by a logical scaffold, when the at least one property of the object as determined by the object identifying module is one of a true condition or a false condition. The true condition may indicate that all the properties of the object as determined by the object identifying module satisfies a logical criterion, while the false condition indicates that the at least one property of the object as determined by the object identifying module fails the logical criterion.

In another embodiment, a method for detecting one or more objects by an artificial intelligence perception system includes the steps of obtaining sensor information of a field of view from a sensor, identifying, by the artificial intelligence perception system, an object within the sensor information, determining, by the artificial intelligence perception system, at least one property of the object based on the sensor information, and determining, by a logical scaffold, when the at least one property of the object as determined by the artificial intelligence perception system is one of a true condition or a false condition. The true condition may indicate that all the properties of the object as determined by object identifying module satisfies a logical criterion, while the false condition indicates that the at least one property of the object as determined by the object identifying module fails the logical criterion.

In yet another embodiment, a non-transitory computer-readable medium for detecting one or more objects by an artificial intelligence perception system includes instructions that when executed by one or more processors cause the one or more processors to obtain sensor information of a field of view from a sensor, identify, by the artificial intelligence perception system, an object within the sensor information, determine, by the artificial intelligence perception system, at least one property of the object based on the sensor information, and determine, by a logical scaffold, when the at least one property of the object as determined by the artificial intelligence perception system is one of a true condition or a false condition. The true condition may indicate that all the properties of the object as determined by object identifying module satisfies a logical criterion, while the false condition indicates that the at least one property of the object as determined by the object identifying module fails the logical criterion.

Further areas of applicability and various methods of enhancing the disclosed technology will become apparent from the description provided. The description and specific examples in this summary are intended for illustration only and are not intended to limit the scope of the present disclosure.

DETAILED DESCRIPTION

Described is an AI perception system that may be utilized in a vehicle. The AI perception system utilizes AI that has been trained to receive sensor information from one or more sensors, determines the presence of any objects within the sensor information, and determines one or more properties regarding the objects within the sensor information. These properties could include any one of a number of different things, such as the type of object, the velocity of the object, the relationship of the object with other objects, etc.

The AI perception system also utilizes a logical scaffold module that utilizes explicit logical specifications. The logical scaffold analyzes the properties determined by the AI perception system and determines if these properties are logical based on one or more criterion expressed in a logic, such as temporal logic. If the properties are not logical, the logical scaffold module can output an indicator indicating that the properties determined by the AI perception system are incorrect, allowing retraining of the AI perception system using the same or similar sensor information previously captured by the AI perception system.

Logical scaffolds may be lightweight formal properties that provide some information about the relationship of the program inputs and outputs. These logical scaffolds can be written in languages for which monitoring algorithms exist, such as Signal Temporal Logic, Signal Convolutional Logic, Timed Quality Temporal Logic, and many others. Logical scaffolds may arise from a number of different sources, including a formalization of physical laws, domain knowledge, and common sense. The logical scaffolds may be more general than reasonableness monitors and model assertions because logical scaffolds can be used for different types of AI programs beyond perception.

Referring toFIG. 1, an example environment10in which a vehicle100having an AI perception system170is shown. It should be understood that the example environment10is just one type of environment in which a vehicle100having the AI perception system170may operate within. The purpose of describing the environment10is to provide an example of how logical scaffolds can be utilized by the AI perception system170.

In this example, the vehicle100is traveling down a road14. The road14includes an intersection17that intersects the road14with another road15. In this example, the intersection17is a two way stop, wherein vehicles traveling along the road14are required to stop at the intersection17. A pair of stops signs18,20may be located near the intersection17so as to indicate the need for vehicles approaching the intersection17along the road14to stop at the intersection17, thus giving vehicles approaching the intersection17from the road15the right of way.

Near the intersection17are crosswalks26,28,30, and32that allow pedestrians, bicyclists, and the like to cross the roads14and/or15in a relatively safe manner. Crosswalks26and32extend across the road14, while crosswalks28and30extend across the road15. In this example, pedestrians36and38are crossing the crosswalk28. The environment10also includes another vehicle34that is traveling along the road15. In this example, the vehicle34is located within the intersection17. Finally, the environment10also includes one or more trees16that are located near the stop sign18.

The vehicle100with the AI perception system170includes a sensor system120that may include any one of a number of different sensors. A more detailed description of the vehicle100, the sensor system120, and the AI perception system170will be given later in this specification. In this example, the sensor system120includes a sensor that has a field-of-view13that generally includes the trees16, the stop signs18and20, the pedestrians36and38, and the vehicle34.

In general, the AI perception system170is able to utilize information generated by the sensor system120to determine the presence of any objects within the field-of-view13, including the trees16, the pedestrians36and38, the stop sign18, and the vehicle34. In addition to determining the presence of objects, the AI perception system170is also able to determine properties regarding the objects. These properties will be described later in this specification but could include information regarding a category of the object (pedestrian, bicycle, vehicle, sign, tree, etc.), a velocity of the object, and location of the object.

The AI perception system170may be a trained artificial intelligence system. In one example, the AI perception system170may be trained using a variety of training sets, sometimes referred to as training data. The training data can be data that was captured by a sensor system, such as the sensor system120and annotated to train the AI perception system170. However, the training of an artificial intelligence system, such as the AI perception system170generally utilizes implicit specifications.

The AI perception system170may also utilize logical scaffolds that may be incorporated into the AI perception system170as a logical scaffold module, which will be described in greater detail later in this specification. The purpose of the logical scaffolds for the AI perception system170is to add a list of logical requirements. For example, the AI perception system170may determine that the pedestrians36and38are traveling at a speed of approximately 2 mph, the vehicle34is traveling at 30 mph, and that the stop signs18,20, and trees16are not moving at all. In this example, the logical scaffolds agree with the properties determined by the AI perception system170.

However, if the AI perception system170determines that the stop sign18was traveling at 30 mph or the pedestrians36or38are traveling at 70 mph, the logical scaffolds would flag this as a mistake determined by the AI perception system170, as pedestrians cannot travel at 70 mph and trees and signs should not be moving. As such, the logical scaffolds add a form of explicit specifications to the AI perception system170. Once a mistake has been flagged, the data collected by the sensor system120can be evaluated and can be used to retrain the AI perception system170so as to create a more reliable AI perception system170. The logical scaffolds may be used during the training of the AI perception system170or can be utilized during the run time of the AI perception system170.

Referring toFIG. 2, an example of a vehicle100is illustrated. As used herein, a “vehicle” is any form of powered transport. In one or more implementations, the vehicle100is an automobile. While arrangements will be described herein with respect to automobiles, it will be understood that embodiments are not limited to automobiles. In some implementations, the vehicle100may be any robotic device or form of powered transport that, for example, includes one or more automated or autonomous systems, and thus benefits from the functionality discussed herein.

In various embodiments, the automated/autonomous systems or combination of systems may vary. For example, in one aspect, the automated system is a system that provides autonomous control of the vehicle according to one or more levels of automation, such as the levels defined by the Society of Automotive Engineers (SAE) (e.g., levels 0-5). As such, the autonomous system may provide semi-autonomous control or fully autonomous control as discussed in relation to the autonomous driving module(s)160.

The vehicle100also includes various elements. It will be understood that in various embodiments it may not be necessary for the vehicle100to have all of the elements shown inFIG. 2. The vehicle100can have any combination of the various elements shown inFIG. 2. Further, the vehicle100can have additional elements to those shown inFIG. 2. In some arrangements, the vehicle100may be implemented without one or more of the elements shown inFIG. 2. While the various elements are shown as being located within the vehicle100inFIG. 1, it will be understood that one or more of these elements can be located external to the vehicle100. Further, the elements shown may be physically separated by large distances and provided as remote services (e.g., cloud-computing services).

In either case, the vehicle100includes the AI perception system170. The AI perception system170may be incorporated within the autonomous driving module(s)160or may be separate, as shown. The AI perception system170may, as will be explained in greater detail later in this specification, receive sensor information from the sensor system120, determine the presence of any objects within information from the sensor system120, categorize any identified objects, and then utilize logical scaffolds to determine if the previous determinations regarding the presence of the objects and/or the category of the objects is correct.

With reference toFIG. 2, one embodiment of the AI perception system170is further illustrated. As shown, the AI perception system170includes a processor110. Accordingly, the processor110may be a part of the AI perception system170or the AI perception system170may access the processor110through a data bus or another communication path. In one or more embodiments, the processor110is an application-specific integrated circuit that is configured to implement functions associated with an image capture module220, an object identifying module230, a logical scaffold module235, and an output module237.

In general, the processor110is an electronic processor such as a microprocessor that is capable of performing various functions as described herein. In one embodiment, the AI perception system170includes a memory210that stores the image capture module220, the object identifying module230, the logical scaffold module235, and the output module237. The memory210is a random-access memory (RAM), read-only memory (ROM), a hard disk drive, a flash memory, or other suitable memory for storing the modules220and230. The modules220,230,235, and237are, for example, computer-readable instructions that, when executed by the processor110, cause the processor110to perform the various functions disclosed herein.

Furthermore, in one embodiment, the AI perception system170includes a data store240. The data store240is, in one embodiment, an electronic data structure such as a database that is stored in the memory210or another memory and that is configured with routines that can be executed by the processor110for analyzing stored data, providing stored data, organizing stored data, and so on. Thus, in one embodiment, the data store240stores data used by the modules220,230,235, and237in executing various functions. In one embodiment, the data store240includes training data250that may include data used to train the AI perception system170, sensor data260that may include sensor data captured by the sensor system120, and scaffold data270that may include one or more logical scaffolds written in one or more logic-based languages, such as Signal Temporal Logic, Signal Convolutional Logic, Timed Quality Temporal Logic and many others. These languages differ from traditional languages in that they allow for specifying timed properties of real-valued (or continuous) signals.

Accordingly, the image capture module220may include instructions that cause the processor110to obtain information from one or more sensors that form the sensor system120. In one example, information received from the sensor system may be in the form of one or more images captured by the camera126of the sensor system120of a field-of-view of the environment in which the vehicle100operates. Of course, it should be understood that in lieu of images from the camera126, other information could be utilized as well such as information from the radar sensors123, the sonar sensors125, and the LIDAR sensors124.

The object identifying module230includes instructions that, when executed by the processor110, causes the processor110to identify an object within the sensor information and determine a property of the object based on the sensor information. The properties of the object that may be determined by the object identifying module230may include any one of a number of different properties. For example, the properties could include temporal properties, ontological properties, and/or a relationship related property.

With regards to ontological properties, these properties may include the motion of the object, the velocity of the object, the location of the object, and/or a transition of the object from one type of object to another type of object. For example, the object identifying module230could determine that the pedestrians36and38ofFIG. 1are indeed pedestrians. Additionally, the object identifying module230could also determine certain properties regarding the pedestrians36and38, such as the motion, velocity, and location of the pedestrians36and38. Furthermore, the system can determine if the pedestrians36and38transitions from one type of object to another. For example, the object identify module230can determine if the object changes from being a pedestrian to a street sign, such as the stop sign18.

With regards to temporal properties, the temporal properties may include the amount of time that the object is within the sensor information and the consistency of the object within the sensor information. In one example, the temporal properties determined by the object identify module230could include the amount of time that the pedestrians36and/or38are located within the sensor information and if they are consistently within the sensor information. Consistency within the sensor information could include if the pedestrians36and/or38suddenly disappear from the sensor information and/or suddenly reappear in the sensor information.

With regard to relationship-related properties, these properties could include the spatial relationship between the objects in another object. For example, the object identifying module230could determine the distances of the pedestrians36and/or38to a number of different objects, such as the trees16, the vehicle34, and other objects.

The logical scaffold module235includes instructions that cause the processor110to determine when the property of the object, as determined by the object identifying module230, is either a true condition or a false condition. As stated before, the logical scaffold module235may utilize scaffold data270that was written in temporal logic, such as Signal Temporal Logic, Signal Convolutional Logic, Timed Quality Temporal Logic and many others. These languages differ from traditional languages in that they allow for specifying timed properties of real-valued (or continuous) signals.

As such, the scaffold data270may include any one of a number of different logical scaffolds that include certain discrete requirements written in the previously mentioned languages. For example, the scaffold data270could include information regarding pedestrians, such as pedestrians36and38ofFIG. 1. In one example, the scaffold data270could include general requirements that the pedestrians have a velocity between 0 mph and 25 mph. With regards to other objects, such as the trees16and the stop signs18and20, the scaffold data270could include requirements that the velocity of these objects be 0 mph. With regards to the vehicle34, logical scaffolds could include requirements that the velocity is between 0 mph and 150 mph in further requirements that the vehicle34be traveling on a roadway, such as the road14and/or16.

Again, these are just examples of some of the logical scaffolds that could be developed. These logical scaffolds may be based on physical properties, such as that trees and stop signs are fixed and do not move, but other properties as well, such as common sense. As such, the logical scaffolds provide discrete requirements so as to prevent and/or detect when the AI perception system170detects and/or categorizes any objects detected in a way that defies the requirements of the logical scaffolds.

In the event that the logical scaffold module235determines that the AI perception system170has detected and/or categorized an object in a way that defies one of the logical scaffolds stored within the scaffold data270, the logical scaffold module235causes the processor110to output of a false condition. Conversely, if the detection and/or categorization of the object satisfies the logical scaffolds, the logical scaffold module235causes the processor110to output a true condition. The outputting of the true condition or false condition may be performed by the output module237which may include instructions that cause the processor110to output either true condition or false condition based on the determination made by the logical scaffold module235.

In the event that a false condition is outputted by the output module237, any one of a number of different actions may be performed. In one example, the output module237also includes instructions that cause the processor110to provide some further information regarding the information from the sensor system120that was fed into the object identifying module230that eventually caused the false condition. In one example, the output module237could cause the processor110to store the information from the sensor system120that resulted in the false condition in the sensor data260. By so doing, this information can be utilized to retrain the AI perception system170so that the AI perception system170next time correctly detects and/or categorizes the object in the stored sensor data. The information from the sensor system120may be one or more images that were captured by the camera126but could also include other information captured from the other environment sensors122, such as the radar sensors123, the LIDAR sensors124, and/or the sonar sensors125.

Referring toFIG. 4, a method300for detecting one or more objects by an AI perception system, such as the AI perception system170, is shown. The method300will be described from the viewpoint of the vehicle100ofFIG. 2and the AI perception system170ofFIG. 3. However, it should be understood that this is just one example of implementing the method300. While method300is discussed in combination with the AI perception system170, it should be appreciated that the method300is not limited to being implemented within the AI perception system170but is instead one example of a system that may implement the method300.

The method300begins at step302, wherein the image capture module220causes the processor110to obtain sensor information of a field-of-view from a sensor. In one example, the sensor information could include information from any of the sensors making up the sensor system120. In one example, the sensor information could be from the environment sensors122, such as the radar sensor123, the LIDAR sensor124, the sonar sensor125, the camera126, or combinations thereof.

In step304, the object identifying module230causes the processor110to identify an object within the sensor information. For example, the object identifying module230may include instructions that are able to process the sensor information to determine the presence of any objects. These objects could include any of the objects previously described inFIG. 1, such as the tree16, the stop signs18and20, the vehicle34, and/or the pedestrians36and38. However, it should be understood that the objects detected by the object identifying module230could include any one of a number of different objects or any type of object present in the environment in which the AI perception system170is operating within.

In step306, the object identifying module230causes the processor110to determine at least one property of the object based on the sensor information. The properties of the object that may be determined by the object identifying module230may include any one of a number of different properties. For example, the properties could include temporal properties, ontological properties, and/or a relationship related property.

With regards to ontological properties, these properties may include the motion of the object, the velocity of the object, the location of the object, and/or a transition of the object from one type of object to another type of object. For example, the object identifying module230could determine that the pedestrians36and38ofFIG. 1are indeed pedestrians. Additionally, the object identifying module230could also determine certain properties regarding the pedestrians36and38, such as the motion, velocity, and location of the object. Furthermore, the system can determine if the object transitions from one type of object to another. For example, the object identify module230can determine if the object changes from being a pedestrian to a street sign, such as the stop sign18. Another example could include a situation where a transition is allowed, such as when a pedestrian mounts a bicycle.

With regards to temporal properties, the temporal properties may include the amount of time that the object is within the sensor information in the consistency of the object within the sensor information. In one example, the temporal properties determined by the object identify module230could include the amount of time that the pedestrians36and/or38are located within the sensor information and if they are consistently within the sensor information. Consistency within the sensor information could include if the pedestrians36and/or38suddenly disappear from the sensor information and/or suddenly reappear in the sensor information. Another example of a temporal property would be how quickly and often a biker transitions to a pedestrian and vice versa. Changing from pedestrian to biker to pedestrian to biker very fast is not realistic.

With regard to relationship-related properties, these properties could include the spatial relationship between the objects in another object. For example, the object identifying module230could determine the distances of the pedestrians36and/or38to a number of different objects, such as the trees16, the vehicle34, and other objects. Another example could include a situation where crosswalk should not end in the middle of the road.

In step308, the logical scaffolds module235causes the processor110to determine if one of the properties of the object, as determined by the object identifying module230, is a true condition or a false condition. As stated before, the logical scaffold module235may utilize scaffold data270that was written in temporal logic, such as Signal Temporal Logic, Signal Convolutional Logic, Timed Quality Temporal Logic and many others. These languages differ from traditional languages in that they allow for specifying timed properties of real-valued (or continuous) signals.

As such, the scaffold data270may include any one of a number of different logical scaffolds that include certain discrete requirements written in the previously mentioned languages. For example, the scaffold data270could include information regarding pedestrians, such as pedestrians36and38ofFIG. 1. In one example, the scaffold data270could include general requirements that the pedestrians have a velocity between 0 mph and 25 mph. With regards to other objects, such as the trees16and the stop signs18and20, the scaffold data270could include requirements that the velocity of these objects be 0 mph. With regards to the vehicle34, logical scaffolds could include requirements that the velocity is between 0 mph and 150 mph in further requirements that the vehicle34be traveling on a roadway, such as the road14and/or16.

Again, these are just examples of some of the logical scaffolds that could be developed. These logical scaffolds may be based on physical properties, such as that trees and stop signs are fixed and do not move, but other properties as well, such as common sense. As such, the logical scaffolds provide discrete requirements so as to prevent and/or detect when the AI perception system170detects and/or categorizes any objects detected in a way that defies the requirements of the logical scaffolds.

If it is determined in step308that the property, as determined by the object identify module230is a true condition, the method300returns to step302. In the event that it is determined that the property, as determined by the object identifying module230, is a false condition, the method300proceeds to step310. In step310, the output module237may cause the processor110to output an indicator indicating that a false condition has been determined. In such a case, the output module237may cause the processor110to store the sensor information that caused the false condition to occur. The sensor information may be stored within the sensor data260.

In step312, the output module237may cause the processor110to retrain the AI perception system170using the sensor information that causes the false condition to occur. As such, the training data250may be updated to include the sensor information which may be annotated and then used to train the AI perception system170with the expectation that the AI perception system170will not make the same or similar mistakes again.

With regard to training the AI perception system170, special note should be made regarding the use of logical scaffolds. The logical scaffolds, such as those stored within the scaffold data270, may be used to train the AI perception system170, while the AI perception system170is not deployed in the field. For example, while the AI perception system170is being trained by images, the logical scaffold module235can cause the processor110to continuously monitor the determinations made by the AI perception system170while it is being trained. In the event that the AI perception system170incorrectly determines the presence of an object and/or certain properties of an object as specified by one or more logical scaffolds stored within the scaffold data270, the sensor information used to train the AI perception system170that caused a false condition to rise, can be used to retrain the AI perception system170.

As such, the logical scaffolds stored in the scaffold data270can be used in a runtime environment, i.e., when the AI perception system170is deployed or can be utilized while the AI perception system170is being trained.

Referring toFIG. 5, a flow diagram400illustrates one way of training a perception model402, similar to the AI perception system170. Here, the flow diagram400begins with training data404that is used to train the perception model402. Logical scaffolds can be utilized as a way to determine if there is a violation of the specification. When there is a violation of the specification, a penalty may be applied to the loss function406During the training of the perception model402, as opposed to waiting for the perception model402to be deployed. Backpropagation408can then compute the gradient of the loss function with respect to the weights of the network.

FIG. 6illustrates a flow diagram500for training a latent space utilizing logical scaffolds. Here, data502is fed into an encoder504, which is then provided to an unlearned latent space506. This information is, in turn, provided to an untrained generative program508. The logical scaffolds510are able to determine if the generative program is making appropriate decisions based on the specifications written in temporal logic that form the logical scaffolds510. From there, a smooth satisfaction metric512can be determined in backpropagation514can be performed.

FIG. 7illustrates a flow diagram600that allows for learning explanations of pre-trained generative models while utilizing logical scaffolds. Here, the latent space602provides data to the generative program604, which in turn generates signals606. The generated signals606are provided to a parameter optimization module608. The parameter optimization module also receives a smooth parametric scaffold610. From there, the parameter optimization module608is able to generate and explainability evaluation612which can explain and provide some evaluation of the generative program604.

In one or more arrangements, the map data116can include one or more terrain maps117. The terrain map(s)117can include information about the ground, terrain, roads, surfaces, and/or other features of one or more geographic areas. The terrain map(s)117can include elevation data in the one or more geographic areas. The map data116can be high quality and/or highly detailed. The terrain map(s)117can define one or more ground surfaces, which can include paved roads, unpaved roads, land, and other things that define a ground surface.

As an example, in one or more arrangements, the sensor system120can include one or more radar sensors123, one or more LIDAR sensors124, one or more sonar sensors125, and/or one or more cameras126. In one or more arrangements, the one or more cameras126can be high dynamic range (HDR) cameras or infrared (IR) cameras.

The vehicle100can include an input system130. An “input system” includes any device, component, system, element, or arrangement or groups thereof that enable information/data to be entered into a machine. The input system130can receive an input from a vehicle passenger (e.g., a driver or a passenger). The vehicle100can include an output system135. An “output system” includes any device, component, or arrangement or groups thereof that enable information/data to be presented to a vehicle passenger (e.g., a person, a vehicle passenger, etc.).

The processor(s)110, the AI perception system170, and/or the autonomous driving module(s)160may be operable to control the navigation and/or maneuvering of the vehicle100by controlling one or more of the vehicle systems140and/or components thereof. For instance, when operating in an autonomous mode, the processor(s)110, the AI perception system170, and/or the autonomous driving module(s)160can control the direction and/or speed of the vehicle100. The processor(s)110, the AI perception system170, and/or the autonomous driving module(s)160can cause the vehicle100to accelerate (e.g., by increasing the supply of fuel provided to the engine), decelerate (e.g., by decreasing the supply of fuel to the engine and/or by applying brakes) and/or change direction (e.g., by turning the front two wheels). As used herein, “cause” or “causing” means to make, force, direct, command, instruct, and/or enable an event or action to occur or at least be in a state where such event or action may occur, either in a direct or indirect manner.

The vehicle100can include one or more autonomous driving module(s)160. The autonomous driving module(s)160can be configured to receive data from the sensor system120and/or any other type of system capable of capturing information relating to the vehicle100and/or the external environment of the vehicle100. In one or more arrangements, the autonomous driving module(s)160can use such data to generate one or more driving scene models.