Patent Description:
Autonomous vehicle navigation is a technology that can allow a vehicle to sense the position and movement of vehicles around an autonomous vehicle and, based on the sensing, control the autonomous vehicle to safely navigate towards a destination. An autonomous vehicle may operate in several modes. In some cases, an autonomous vehicle may allow a driver to operate the autonomous vehicle as a conventional vehicle by controlling the steering, throttle, clutch, gear shifter, and/or other devices. In other cases, a driver may engage the autonomous vehicle navigation technology to allow the vehicle to be driven by itself. <CIT> discloses examples of prior art autonomous vehicle navigation methods and systems.

When a vehicle is driven, the vehicle's operations is dependent at least in part on the environment in which the vehicle is operated. This document describes an invention including systems, apparatus, and methods to determine visibility conditions of the environment in which the vehicle operates to perform autonomous driving related operations on the vehicle.

The invention is a method, computer program computer readable medium and system as defined in the appended claims. In accordance with a first aspect, there is provided a method of autonomous driving operation comprises determining, by a computer located in an autonomous vehicle, a visibility related condition of an environment in which the autonomous vehicle is operating; adjusting, based at least on the visibility related condition, a set of one or more values of one or more variables associated with a driving related operation of the autonomous vehicle; and causing the autonomous vehicle to be driven to a destination by causing the driving related operation of one or more devices located in the autonomous vehicle based on at least the set of one or more values, wherein the autonomous vehicle is determined to operate in the bad visibility condition in response to determining that the location of the autonomous vehicle is within a pre-determined distance of a canyon or a tunnel.

In yet another exemplary aspect, the above-described method is embodied in a non-transitory computer readable storage medium comprising code that when executed by a processor, causes the processor to perform the methods described in this patent document.

In yet another exemplary aspect, a device that is configured or operable to perform the above-described methods is disclosed.

Developments in autonomous driving technology have led to a development of semi-trailer truck that can be autonomously driven to deliver goods to a destination. When a semi-trailer truck is driven to its destination, the semi-trailer truck can experience different types of environments that can affect its driving related operations. In an example scenario, an in-vehicle control computer located in a semi-trailer truck can determine that the semi-trailer truck is driven at night on a road having one lane. In this example scenario, the in-vehicle control computer can adjust one or more values of one or more driving related variables so that the semi-trailer truck can perform its driving related operations that are appropriate for night-time drawing on a road with limited number of lanes. In this example scenario, the value(s) of the driving related variable(s) can be different than those for a semi-trailer truck driven during the day on a highway with multiple lanes. The driving related variables may include, for example, a maximum speed of the semi-trailer truck or minimum distance between the semi-trailer truck and a vehicle immediately in front of the semi-trailer truck.

In another example scenario, the in-vehicle control computer can, based on determining visibility related characteristics of a semi-trailer truck's environment (e.g., fog, smoke, location in a canyon, detection of lane markers, etc.,), adjust the driving related rules that can be used to perform driving related operations on the semi-trailer truck. The driving related rules can include, for example, normal driving rules, defensive rules, no lane change allowed rule, or a canyon driving rules. Each of the driving related rules can be pre-defined so that each rule can be associated with one or more pre-defined driving related variables and one or more corresponding values that can be pre-defined.

Section I of this patent document provides an overview of the devices/systems located on or in an autonomous vehicle, such as an autonomous semi-trailer truck. Section II of this patent document describes techniques to determine visibility-related conditions in an environment in which the semi-trailer truck operates. Section III of this patent document describes techniques employed by the in-vehicle control computer to adjust driving related operations based on the visibility-related conditions. The term "visible" or "visibility" can include, for example, lighting condition of the environment in which the semi-trailer truck operates, whether certain objects (e.g., traffic lane markers or traffic lights) are detected in images obtained from cameras onboard the semi-trailer truck, whether certain information about the environment in which the semi-trailer truck is operated is detected (e.g., location information by GPS onboard the semi-trailer truck or environment data obtained by sensors, such as cameras, onboard the semi-trailer truck), or a presence of a number of vehicle(s) located in front of the semi-trailer truck (e.g., indicating a heavy traffic condition). Section IV of this patent document describes technique employed by the in-vehicle control computer to adjust the driving related operations based on a condition of the semi-trailer truck (e.g., braking condition). The example headings for the various sections below are used to facilitate the understanding of the disclosed subject matter and do not limit the scope of the claimed subject matter in any way. Accordingly, one or more features of one example section can be combined with one or more features of another example section.

<FIG> shows a block diagram of an example vehicle ecosystem <NUM> in which an in-vehicle control computer <NUM> located in the vehicle <NUM> can determine a visibility-related condition of an environment in which the vehicle <NUM> is operating. As shown in <FIG>, the vehicle <NUM> may be a semi-trailer truck. The vehicle ecosystem <NUM> includes several systems and components that can generate and/or deliver one or more sources of information/data and related services to the in-vehicle control computer <NUM> that may be located in a vehicle <NUM>. The in-vehicle control computer <NUM> can be in data communication with a plurality of vehicle subsystems <NUM>, all of which can be resident in the vehicle <NUM>. A vehicle subsystem interface <NUM> is provided to facilitate data communication between the in-vehicle control computer <NUM> and the plurality of vehicle subsystems <NUM>. In some arrangements, the vehicle subsystem interface <NUM> can include a controller area network (CAN) controller to communicate with devices in the vehicle subsystems <NUM>.

The vehicle <NUM> may include various vehicle subsystems that support of the operation of vehicle <NUM>. The vehicle subsystems may include a vehicle drive subsystem <NUM>, a vehicle sensor subsystem <NUM>, and/or a vehicle control subsystem <NUM>. The components or devices of the vehicle drive subsystem <NUM>, the vehicle sensor subsystem <NUM>, and the vehicle control subsystem <NUM> as shown as examples. In some arrangements, additional components or devices can be added to the various subsystems or one or more components or devices can be removed without affecting the visibility determination related features described in this patent document. The vehicle drive subsystem <NUM> may include components operable to provide powered motion for the vehicle <NUM>. In an example arrangement, the vehicle drive subsystem <NUM> may include an engine or motor, wheels/tires, a transmission, an electrical subsystem, and a power source.

The vehicle sensor subsystem <NUM> may include a number of sensors configured to sense information about an environment in which the vehicle <NUM> is operating or a condition of the vehicle <NUM>. As further explained in this patent document, the visibility determination module <NUM> in the in-vehicle control computer <NUM> can determine the extent of the visibility of the environment based on information provided by sensors (e.g., light sensor or cameras) in the vehicle sensor subsystem <NUM>. The vehicle sensor subsystem <NUM> may include one or more cameras or image capture devices, one or more temperature sensors, an inertial measurement unit (IMU), a Global Positioning System (GPS) transceiver, a laser range finder/LIDAR unit, a RADAR unit, and/or a wireless communication unit (e.g., a cellular communication transceiver). The vehicle sensor subsystem <NUM> may also include sensors configured to monitor internal systems of the vehicle <NUM> (e.g., an O<NUM> monitor, a fuel gauge, an engine oil temperature, etc.,). In some arrangements, the vehicle sensor subsystem <NUM> may include sensors in addition to the sensors shown in <FIG>.

The IMU may include any combination of sensors (e.g., accelerometers and gyroscopes) configured to sense position and orientation changes of the vehicle <NUM> based on inertial acceleration. The GPS transceiver may be any sensor configured to estimate a geographic location of the vehicle <NUM>. For this purpose, the GPS transceiver may include a receiver/transmitter operable to provide information regarding the position of the vehicle <NUM> with respect to the Earth. The RADAR unit may represent a system that utilizes radio signals to sense objects within the environment in which the vehicle <NUM> is operating. In some arrangements, in addition to sensing the objects, the RADAR unit may additionally be configured to sense the speed and the heading of the objects proximate to the vehicle <NUM>. The laser range finder or LIDAR unit may be any sensor configured to sense objects in the environment in which the vehicle <NUM> is located using lasers. The cameras may include one or more cameras configured to capture a plurality of images of the environment of the vehicle <NUM>. The cameras may be still image cameras or motion video cameras.

The vehicle control subsystem <NUM> may be configured to control operation of the vehicle <NUM> and its components. Accordingly, the vehicle control subsystem <NUM> may include various elements such as a throttle and gear, a brake unit, a navigation unit, a steering system and/or an autonomous control unit. The throttle may be configured to control, for instance, the operating speed of the engine and, in turn, control the speed of the vehicle <NUM>. The gear may be configured to control the gear selection of the transmission. The brake unit can include any combination of mechanisms configured to decelerate the vehicle <NUM>. The brake unit can use friction to slow the wheels in a standard manner. The brake unit may include an Anti-lock brake system (ABS) that can prevent the brakes from locking up when the brakes are applied. The navigation unit may be any system configured to determine a driving path or route for the vehicle <NUM>. The navigation unit may additionally be configured to update the driving path dynamically while the vehicle <NUM> is in operation. In some arrangements, the navigation unit may be configured to incorporate data from the GPS transceiver and one or more predetermined maps so as to determine the driving path for the vehicle <NUM>. The steering system may represent any combination of mechanisms that may be operable to adjust the heading of vehicle <NUM> in an autonomous mode or in a driver-controlled mode.

The autonomous control unit may represent a control system configured to identify, evaluate, and avoid or otherwise negotiate potential obstacles in the environment of the vehicle <NUM>. In general, the autonomous control unit may be configured to control the vehicle <NUM> for operation without a driver or to provide driver assistance in controlling the vehicle <NUM>. In some arrangements, the autonomous control unit may be configured to incorporate data from the GPS transceiver, the RADAR, the LIDAR, the cameras, and/or other vehicle subsystems to determine the driving path or trajectory for the vehicle <NUM>.

The traction control system (TCS) may represent a control system configured to prevent the vehicle <NUM> from swerving or losing control while on the road. For example, TCS may obtain signals from the IMU and the engine torque value to determine whether it should intervene and send instruction to one or more brakes on the vehicle <NUM> to mitigate the vehicle <NUM> swerving. TCS is an active vehicle safety feature designed to help vehicles make effective use of traction available on the road, for example, when accelerating on low-friction road surfaces. When a vehicle without TCS attempts to accelerate on a slippery surface like ice, snow, or loose gravel, the wheels can slip and can cause a dangerous driving situation. TCS may also be referred to as electronic stability control (ESC) system.

Many or all of the functions of the vehicle <NUM> can be controlled by the in-vehicle control computer <NUM>. The in-vehicle control computer <NUM> may include at least one processor <NUM> (which can include at least one microprocessor) that executes processing instructions stored in a non-transitory computer readable medium, such as the memory <NUM>. The in-vehicle control computer <NUM> may also represent a plurality of computing devices that may serve to control individual components or subsystems of the vehicle <NUM> in a distributed fashion. In some embodiments, the memory <NUM> may contain processing instructions (e.g., program logic) executable by the processor <NUM> to perform various methods and/or functions of the vehicle <NUM>, including those described for the visibility determination module <NUM> and the driving operation module <NUM> as explained in this patent document. For instance, the processor <NUM> executes the operations associated with visibility determination module <NUM> for determining visibility of the environment in which the vehicle <NUM> operates based on sensor data, as further described in Section II. The processor <NUM> executes the operations associated with driving operation module <NUM> for determining various driving related operations of the vehicle <NUM> based on the visibility determined by the visibility determination module <NUM>, as further described in Section III. In embodiments, the driving operation module <NUM> can adjust the driving related operations based on a condition of the vehicle <NUM>, as described in Section IV.

The memory <NUM> may contain additional instructions as well, including instructions to transmit data to, receive data from, interact with, or control one or more of the vehicle drive subsystem <NUM>, the vehicle sensor subsystem <NUM>, and the vehicle control subsystem <NUM>. The in-vehicle control computer <NUM> may control the function of the vehicle <NUM> based on inputs received from various vehicle subsystems (e.g., the vehicle drive subsystem <NUM>, the vehicle sensor subsystem <NUM>, and the vehicle control subsystem <NUM>).

The visibility determination module <NUM> can determine visibility related conditions of the environment in which the autonomous vehicle <NUM> is operating based on data received from one or more sensors that are part of the vehicle sensor subsystem <NUM>. The data received from the one or more sensors can be analyzed by the visibility determination module <NUM> to determine the amount of light of the environment in which the autonomous vehicle <NUM> is operating.

In some embodiments, the visibility determination module <NUM> can receive sensor data from one or more cameras or a light sensor. The sensor data may include images obtained from the one or more cameras or an output value (e.g., voltage or resistance value) from the light sensor. In an example implementation, the visibility determination module <NUM> can determine the extent of light by analyzing the pixels of the images to determine the amount of light of the environment in which the autonomous vehicle <NUM> is operating. In another example implementation, the visibility determination module <NUM> can determine the amount of light based on the output value provided by the light sensor. If the amount of light is lower than a threshold value, then the visibility determination module <NUM> can determine that the vehicle <NUM> is operating in poor visibility conditions. The poor visibility conditions can be situations when the vehicle <NUM> is driving during evening or night time or when the vehicle is driving through a canyon or tunnel or in bad lighting conditions (e.g., fog).

In some embodiments, the data received from the one or more sensors can be analyzed by the visibility determination module <NUM> to determine whether certain objects on the road (e.g., lane markers) are discernable from the sensor data. The visibility determination module <NUM> can determine visibility related conditions of the environment in which the autonomous vehicle <NUM> is operating based on whether certain objects located on the road (e.g., lane markers or traffic lights) are discernable. <FIG> shows an example scenario where an autonomous vehicle <NUM> is driving on a lane on a road <NUM>. The lane on which the autonomous vehicle <NUM> is driven includes two sets of multiple lane markers - one set of multiple lane markers 210a are located on a first side of the lane and another set of multiple lane markers 210b are located on a second side of the lane opposite to the first side. The autonomous vehicle <NUM> includes a camera <NUM> and/or a LiDAR sensor <NUM> with which the visibility determination module can determine a presence of the lane markers over a distance from the location of the autonomous vehicle <NUM>. The camera <NUM> and/or the LiDAR sensor <NUM> are structured to point to an area <NUM> in front the autonomous vehicle <NUM> so that the camera <NUM> and/or LiDAR sensor <NUM> can obtain sensor data (e.g., images or point cloud data) of the front area <NUM>.

In one example implementation, the visibility determination module can receive an image from the camera <NUM> and can determine whether a number of lane markers greater than or equal to a threshold number of lane markers are present within a distance (e.g., <NUM> to <NUM> meters as shown in <FIG>) in front of a location of the autonomous vehicle <NUM>. The distance in front of a location of the autonomous vehicle <NUM> can be pre-determined. For example, the visibility determination module can determine that the image shows the presence of a total of six lane markers (three lane markers on each side of the lane) on the road over a distance of <NUM> meters. If the threshold number of lane markers is four, then the visibility determination module can determine that a good visibility condition exists since the visibility determination module determines the presence of six lane markers in the image from the camera <NUM>. And, if the visibility determination module determines a presence of less than four lane markers over the example distance of <NUM> meters, then the visibility determination module can determine that a bad visibility condition exists for the example implementation where the threshold number of lane markers is four.

In another example implementation, the visibility determination module can receive sensor data (e.g., point cloud data) from the LiDAR sensor <NUM> and can determine whether a number of lane markers is greater than or equal to a threshold number of lane markers are present within a distance (e.g., <NUM> to <NUM> meters as shown in <FIG>) in front of a location of the autonomous vehicle <NUM>. The light reflected from the lane markers can indicate the presence of and the number of lane markers in the point cloud data over the distance in front of a location of the autonomous vehicle <NUM>. For example, the visibility determination module can determine that over a distance of <NUM> meters, the point cloud data shows the presence of a total of eight lane markers (four lane markers on each side of the lane) on the road. If the threshold number of lane markers is six, then the visibility determination module can determine that a good visibility condition exists since the visibility determination module determines the presence of eight lane markers in the point cloud data from the LiDAR sensor <NUM>. And, if the visibility determination module determines a presence of less than six lane markers over the example distance of <NUM> meters, then the visibility determination module can determine that a bad visibility condition exists for the example implementation where the threshold number of lane markers is six.

In yet another example implementation, the visibility determination module can determine whether a good visibility condition exists based on whether other objects on the road are discernable from the sensor data. For example, if the visibility determination module determines, based on location information provided by a GPS device located in the autonomous vehicle <NUM>, that the autonomous vehicle <NUM> is traveling on a local road (e.g., not a highway), then the visibility determination module can determine whether a presence of a traffic light is detected from the sensor data (e.g., image data or point cloud data) when the autonomous vehicle <NUM> is within a range of distances (e.g., between <NUM> meters and <NUM> meters) of a traffic intersection. Having a range of distances, as opposed to a single distance, is a beneficial technical feature to prevent false positives. For example, if a single distance is used, then a presence of a traffic light detected when the autonomous vehicle <NUM> reaches <NUM> meters from the location of the traffic intersection can be considered a good visibility condition even though such a detection can be considered a bad visibility condition. To prevent such false positive outcomes, the range of distances (e.g., between <NUM> meters and <NUM> meters) can be used to determine whether a traffic light is detected when the autonomous vehicle <NUM> is within the range of distances. In some embodiments, the range of distances may be pre-determined.

In an implementation example, if the visibility determination module determines that the sensor data indicates a presence of a traffic light at the traffic intersection when the autonomous vehicle <NUM> is within <NUM> to <NUM> meters of a traffic intersection, then the visibility determination module determines a presence of a good visibility condition. And, if the visibility determination module determines that the sensor data does not indicate a presence of a traffic light at the traffic intersection when the autonomous vehicle <NUM> is within <NUM> meters of a traffic intersection, then the visibility determination module determines a presence of a bad visibility condition.

In some embodiments, visibility determination module may determine whether an image contains sufficient amount of relevant gray-scale or luminance values for object identification such as lane markers or traffic lights detectors to perform reasonably well by evaluating the visibility entropy of one or multiple images. For example, the visibility determination module may determine a presence of a lane marker or a traffic light in an image upon determining that a set of gray-scale values or a set of luminance values of an object in the image is associated with the lane marker or the traffic light. The visibility entropy calculations can be performed using the Shannon Entropy that can measure the overall uncertainty of a random variable. The visibility entropy determination can include determining a distribution of signals so that a bias in gain, contrast or homogeneity of the signal can be determined.

The visibility determination module <NUM> can determine visibility related conditions of the environment in which the autonomous vehicle <NUM> is operating based on the weather conditions, time of day and/or location of the autonomous vehicle <NUM>.

In some embodiments, the in-vehicle control computer can receive weather conditions via a wireless transceiver located in the vehicle <NUM>. The weather condition can indicate a weather of a region where the vehicle <NUM> is operated. Based on the indicated weather conditions, the visibility determination module <NUM> can determine whether a good or a bad visibility condition exists in a location where the vehicle <NUM> is operated. For example, if the weather condition indicates a rainy or snowy condition, the visibility determination module can determine that a bad visibility condition exists for the area in which the vehicle <NUM> is operated. In some embodiments, the in-vehicle control computer can repeatedly obtain weather conditions which may vary as the vehicle <NUM> drives from one location to another location. The visibility determination module can repeatedly and dynamically adjust the visibility determinations based on the most relevant weather information associated with the locations where the vehicle <NUM> is operating.

In some embodiments, a pre-determined time schedule stored in the in-vehicle control computer can indicate the times when a good visibility condition may be present and the times when a bad visibility condition may be present. In an example implementation, the pre-determined time schedule can be as shown in Table <NUM> below.

Based on the time and/or date when the autonomous vehicle <NUM> is operating, the visibility determination module <NUM> can determine whether a good visibility condition exists or a bad visibility condition exists.

The visibility determination module <NUM> determines whether a good (or bad) visibility condition exists based on the location of the autonomous vehicle. If the visibility determination module <NUM> determines, based on the location of the vehicle <NUM> obtained from a GPS device onboard the vehicle <NUM>, that the vehicle <NUM> is within a distance of a location of a canyon or a tunnel, then the visibility determination module <NUM> can proactively adjust the visibility related determinations. For example, if the vehicle <NUM> is driving on a road on May <NUM> at 11am, which can be considered a good visibility condition, and if the vehicle <NUM> is within a pre-determined distance of a tunnel (e.g., <NUM> meters), then the visibility determination module <NUM> can determine that the visibility condition is a bad visibility condition in anticipation of traveling through the tunnel so that the vehicle <NUM> can employ appropriate driving related operations (as further explained in Section II).

In some embodiments, the visibility determination module <NUM> can employ any combination of the techniques described above. For example, as explained above, the visibility determination module <NUM> can employ a pre-determined time schedule with a location of the vehicle <NUM> to more precisely determine a visibility condition of the environment where the vehicle <NUM> is (or will be) operating. In another example, the visibility determination module <NUM> can determine that the visibility condition is good based on a location of the vehicle <NUM> along with sensor data obtained from the cameras and/or LiDAR sensors (e.g., the visibility determination module <NUM> can determine that the visibility condition is good if the amount of light is greater than a threshold value even if the vehicle <NUM> is driving through a canyon location which can be associated with a bad visibility condition).

In some embodiments, the various threshold values or distance values (e.g., threshold value for amount of light or threshold number of lane markers or a distance in front of the location of the autonomous vehicle) can be pre-determined or can be dynamically adjusted when the vehicle <NUM> is driving. The visibility determination module can adjust the threshold values or distance values with which the visibility determination module determines whether a good or a bad visibility condition exists based on, for example, a presence of a number of vehicles in front of the vehicle <NUM>. Using an example mentioned above, if the visibility determination module determines that an image from a camera shows the presence of a total of six lane markers (three lane markers on each side of the lane) on the road over a distance of <NUM> meters and if the visibility determination module determines that a number of vehicles within a distance (e.g., <NUM> meters) in front of the vehicle <NUM> is more than a threshold value (indicating a high-traffic scenario), then the visibility determination module can increase the threshold number of lane markers to eight. In this example, the visibility determination module can determine that a bad visibility condition exists by determining that the number of lane markers detected is less than the threshold number of lane markers.

Using another example mentioned above, if the visibility determination module determines that the vehicle <NUM> is operation on a local road (e.g., not a highway) and if the visibility determination module determines that a number of vehicles located within a distance of the vehicle <NUM> is less than a threshold value (indicating a low-traffic scenario), then the visibility determination module can decrease a range of the distances (e.g., from <NUM> to <NUM> meters to <NUM> meters to <NUM> meters). As explained above, if the visibility determination module determines that a traffic light is detected within the decreased range of distances, then the visibility determination module can determine that a good visibility condition exists. And if the visibility determination module determines that a traffic light is not detected within the decreased range of distances, then the visibility determination module can determine that a bad visibility condition exists.

The techniques employed by the visibility determination module <NUM> as explained in Section II can be performed repeatedly so that the driving related operations of the vehicle <NUM> can be appropriately managed given the various driving related conditions that the vehicle <NUM> may face as it travels from one location to another.

The driving operation module <NUM> can adjust the driving related operations of the vehicle <NUM> so that vehicle <NUM> can be safely driven based on the visibility conditions determined the visibility determination module <NUM> as described in Section II. A driving related operation can include the driving operation module <NUM> sending commands to the engine, brakes, transmission, or steering to autonomously drive the vehicle <NUM>. In some embodiments, the driving operation module <NUM> can adjust the driving related operations by adjusting one or more values of one or more driving related variables. For example, if the visibility determination module <NUM> determines that a bad visibility condition exists for the environment in which the vehicle <NUM> is operating, then the driving operation module <NUM> can adjust a maximum speed to be lower than that for a good visibility condition and/or the driving operation module <NUM> can adjust a minimum distance between the vehicle <NUM> and a vehicle immediately in front of and in the same lane as the vehicle <NUM> so that the minimum distance can be higher than that for a good visibility condition.

In some embodiments, the driving operation module <NUM> can refer to a pre-determined table (as shown in the example Table <NUM> below) that can include a list of pre-determined variables and a list of pre-determined values associated with the variables for both a good visibility condition and a bad visibility condition. The driving operation module <NUM> can adjust the driving related operations based on the one or more values by, for example, applying brakes to decrease the speed of the vehicle <NUM> upon determining that a current speed of the vehicle <NUM> is greater than the maximum speed or upon determining that a distance between the vehicle <NUM> and another vehicle immediately in front of the vehicle <NUM> is less than a minimum distance.

The driving operation module <NUM> can adjust the driving related operations by adjusting the value(s) associated with the driving-related variable(s). For example, using the example Table <NUM>, if the visibility determination module determines that the visibility condition changes from good to bad, then the driving operation module <NUM> can adjust the maximum speed from <NUM> mph to <NUM> mph, increase the minimum distance from <NUM> meters to <NUM> meters, turn on headlights. In some embodiments, the minimum distance between the vehicle <NUM> and a vehicle immediately in front of and in the same lane as the vehicle <NUM> can be determined by using a time measurement. For example, a time measurement between the vehicle <NUM> and a vehicle immediately in front of the vehicle <NUM> can be <NUM> seconds for a good visibility condition and <NUM> seconds for a bad visibility condition. Based on the time measurements associated with the visibility conditions and based on a current speed of the vehicle <NUM>, the driving operation module <NUM> can determine the minimum distance to the vehicle in front of the autonomous vehicle <NUM>.

In some embodiments, if the visibility determination module determines that the vehicle <NUM> is operating in a bad visibility condition, then the driving operation module <NUM> can use Table <NUM> to determine routing and navigation to avoid routes without traffic lights if the vehicle <NUM> is operating on a local road (i.e., not on a highway). The routing and navigation used by the driving operation module <NUM> can be based on presence of traffic lights when a based visibility condition is determined so that the vehicle <NUM> can be operated on roads with traffic lights to improve safety. In some embodiments, other types of variables can be included in the pre-determined table of variables. For example, when a bad visibility condition is determined, the driving operation module <NUM> can use a pre-defined lane change variable that can indicate that the vehicle <NUM> can only perform lane changes in emergencies (e.g., to avoid other vehicles). In a related example, when a good visibility condition is determined, the driving operation module <NUM> can use a pre-defined lane change variable that can indicate that the vehicle <NUM> can perform as many lane changes as desired.

In some embodiments, the driving operation module <NUM> can refer to a pre-determined table (as shown in the example Table <NUM> below) that can include a list of pre-determined variables and a list of pre-determined values associated with the variables for both a good visibility condition and a bad visibility condition and for road types (e.g., multiple lanes versus single lane).

Using the example Table <NUM>, if the visibility determination module determines that the visibility condition changes from good to bad and if the visibility determination module determines that the vehicle <NUM> is operating on a road with multiple traffic lanes, then the driving operation module <NUM> can adjust the maximum speed from <NUM> mph to <NUM> mph, increase the minimum distance from <NUM> meters to <NUM> meters, turn on headlights and operate in the merging lane. The merging lane can include the right-most lane on the highway, where the right-most lane is the lane onto which vehicles merge when entering the highway.

In some embodiments, the driving operation module <NUM> can adjust the driving related operations by adjusting a driving related rule with which the in-vehicle control computer can instruct the various devices on the vehicle <NUM> to perform their driving related operations. A driving related rule can be based on the visibility condition determined by the visibility determination module <NUM> and/or additional criteria, such as a total number of traffic lanes including the lane on which the vehicle <NUM> is operating, or a number of vehicles within a pre-determined distance of the vehicle <NUM>. Table <NUM> shows an example of a pre-determined table for driving related rules.

Using the example values shown in Table <NUM>, if the visibility determination module <NUM> determines a good visibility condition, and the if the visibility determination module <NUM> determines that a number of vehicles within a pre-determined distance in front of the vehicle <NUM> is less than a threshold value, then the driving operation module <NUM> can select a "Normal Driving" rule and the values for the variables shown in Table <NUM>. The visibility determination module <NUM> can determine a "heavy traffic" condition (mentioned in the third column from left in Table <NUM>) upon determining that a number of the vehicles within a pre-determined distance of a location of the vehicle <NUM> is more than a threshold value. The visibility determination module <NUM> can determine that a vehicle <NUM> is operating in a canyon or is driving downhill (mentioned in the fourth column from left in Table <NUM>) based on information received from a GPS device located in the vehicle <NUM>. The visibility determination module <NUM> can determine that the vehicle <NUM> is operating on a road having a single lane (mentioned in the fifth column from left in Table <NUM>) based on sensor data provided by the camera(s) and/or LiDAR sensor located on the vehicle <NUM>. The visibility determination module <NUM> can determine that the vehicle <NUM> is operating on a lane onto which one or more other vehicles can merge (mentioned in the fifth column from left in Table <NUM>) based on sensor data provided by the camera(s) and/or LiDAR sensor located on the vehicle <NUM>.

In some embodiments, one or more values of the one or more variables for each column can be adjusted based on a total weight hauled by the vehicle <NUM>. For example, the visibility determination module can use the values in Tables <NUM> to <NUM> when the visibility determination module determines that the vehicle <NUM> has a weight greater than a threshold value (e.g., weight of vehicle is <NUM>,<NUM> lbs. and threshold value is <NUM>,<NUM> lbs. In another example, the values in Tables <NUM> to <NUM> can be adjusted if the visibility determination module determines that the weight of the vehicle is lower than the threshold value. In an example implementation, using the "normal driving" condition of Table <NUM>, the vehicle determination module can decrease the minimum threshold to <NUM> meters upon determining that the weight of the vehicle is lower than the threshold value and can keep the maximum speed to <NUM> mph so as to not exceed the speed limit. In some embodiments, the maximum speeds indicated in Tables <NUM> to <NUM> can be adjusted based on the speed limits of various local roads or state laws, which can be provided by GPS information.

Based on the driving related conditions determined (e.g., canyon driving) by the visibility determination module <NUM>, the driving operation module <NUM> can select one or more values of one or more variables associated with the determined driving related condition. The driving operation module <NUM> can send instructions to one or more devices (e.g., engine, brakes, transmission) based on the one or more values of the one or more variables.

The driving operation module <NUM> can adjust the driving related operations based on a condition of the semi-trailer truck. For example, the driving operation module <NUM> can monitor whether brakes are performing as desired by comparing an amount of brakes applied to a stopping distance that can be measured by the sensor data provided by the sensors (e.g., images provided by a camera). If the driving operation module <NUM> determines that the stopping distance is greater than an expected stopping distance for the applied amount of brakes, then the driving operation module <NUM> can change the value(s) of the variables indicated in Table <NUM> from a "normal driving" condition to a "defensive driving" condition. In some embodiments, a specific value associated with a driving related operation can be changed for a specific condition of the semi-trailer truck. Using the above-mentioned above, if the driving operation module <NUM> determines that the stopping distance is greater than an expected stopping distance for the applied amount of brakes, then the driving operation module <NUM> can increase the value of the "minimum distance to vehicle in front of autonomous vehicle" shown in Tables <NUM> to <NUM>.

In some embodiments, a LiDAR sensor can be used to improve visibility when a bad visibility condition is determined. A LiDAR sensor can be used to detect objects though bad visibility conditions such as a fog. Thus, for example, a driving operation module <NUM> can use the point cloud data provided by LiDAR sensors to obtain visibility information when the visibility determination module <NUM> determines that the visibility is bad.

<FIG> shows a flow diagram for performing driving related operations of an autonomous vehicle based on a visibility condition. Operation <NUM> includes determining, by a computer located in an autonomous vehicle, a visibility related condition of an environment in which the autonomous vehicle is operating. Operation <NUM> includes adjusting, based at least on the visibility related condition, a set of one or more values of one or more variables associated with a driving related operation of the autonomous vehicle. Operation <NUM> includes causing the autonomous vehicle to be driven to a destination by causing the driving related operation of one or more devices located in the autonomous vehicle based on at least the set of one or more values.

In some embodiments, the determining the visibility related condition includes: performing a first determination, based on sensor data provided by a light sensor, that an amount of light of the environment is less than a threshold value, where the light sensor is located on or in the autonomous vehicle; and performing a second determination, in response to the first determination, that the environment is associated with a bad visibility condition, where the set of one or more values are adjusted to be same as a second set of one or more values associated with the bad visibility condition. In some embodiments, the determining the visibility related condition includes: performing a first determination, based on an image provided by a camera, that an amount of light of the environment is less than a threshold value, where the camera is located on or in the autonomous vehicle; and performing a second determination, in response to the first determination, that the environment is associated with a bad visibility condition, where the set of one or more values are adjusted to be same as a second set of one or more values associated with the bad visibility condition.

In some embodiments, the determining the visibility related condition includes: performing a first determination, based on a senor data provided by a light sensor or a camera, that an amount of light of the environment is greater than or equal to a threshold value, where the light sensor or the camera is located on or in the autonomous vehicle; and performing a second determination, in response to the first determination, that the environment is associated with a good visibility condition, where the set of one or more values are adjusted to be same as a third set of one or more values associated with the good visibility condition. In some embodiments, the determining the visibility related condition includes: performing a first determination, based on information provided by a global positioning system (GPS) transceiver located on the autonomous vehicle, that the autonomous vehicle is operating within a range of a first distance and a second distance of a traffic intersection, where the first distance is different from the second distance; performing a second determination that a traffic light is not detected in images provided by a camera located on or in the autonomous vehicle when the autonomous vehicle is operated with the range of the first distance and the second distance; and performing a third determination, in response to the first determination and the second determination, that the environment is associated with a bad visibility condition, where the set of one or more values are adjusted to be same as a second set of one or more values associated with the bad visibility condition.

In some embodiments, the determining the visibility related condition includes: performing a first determination, based on information provided by a global positioning system (GPS) transceiver located on the autonomous vehicle, that the autonomous vehicle is operating within a range of a first distance and a second distance of a traffic intersection, where the first distance is different from the second distance; performing a second determination that a traffic light is detected in images provided by a camera located on or in the autonomous vehicle when the autonomous vehicle is operated with the range of the first distance and the second distance; and performing a third determination, in response to the first determination and the second determination, that the environment is associated with a good visibility condition, where the set of one or more values are adjusted to be same as a third set of one or more values associated with the good visibility condition.

In some embodiments, the first distance and the second distance are based on a number of vehicles located within a distance in front of the autonomous vehicle. In some embodiments, the determining the visibility related condition includes: performing a first determination, based on an image provided by a camera, that a number of lane markers located on a road within a distance of a location of the autonomous vehicle is less than a threshold value, where the camera is located on or in the autonomous vehicle; and performing a second determination, in response to the first determination, that the environment is associated with a bad visibility condition, where the set of one or more values are adjusted to be same as a second set of one or more values associated with the bad visibility condition. In some embodiments, the determining the visibility related condition includes: performing a first determination, based on a point cloud data provided by a light detection and ranging (LiDAR) sensor, that a number of lane markers located on a road within a distance of a location of the autonomous vehicle is less than a threshold value, where the LiDAR sensor is located on or in the autonomous vehicle; and performing a second determination, in response to the first determination, that the environment is associated with a bad visibility condition, where the set of one or more values are adjusted to be same as a second set of one or more values associated with the bad visibility condition.

In some embodiments, the determining the visibility related condition includes: performing a first determination, based on a point cloud data or an image respectively provided by a light detection and ranging (LiDAR) sensor or a camera, that a number of lane markers located on a road within a distance of a location of the autonomous vehicle is greater than or equal to a threshold value, where the LiDAR sensor or the camera is located on or in the autonomous vehicle; and performing a second determination, in response to the first determination, that the environment is associated with a good visibility condition, where the set of one or more values are adjusted to be same as a third set of one or more values associated with the good visibility condition. In some embodiments, the threshold value is based on a number of vehicles located within a distance in front of the autonomous vehicle.

In some embodiments, the determining the visibility related condition includes determine whether the environment is associated with either a good visibility condition or a bad visibility condition based on a weather condition of the environment, and where the adjusting the set of one or more values includes: adjusting the set of one or more values to be same as a second set of values associated with the good visibility condition, or adjusting the set of one or more values to be same as a third set of values associated with the bad visibility condition.

In some embodiments, the determining the visibility related condition includes determining whether the environment is associated with either a good visibility condition or a bad visibility condition based on a location of the autonomous vehicle, where the adjust the set of one or more values includes: adjust the set of one or more values to be same as a second set of values associated with the good visibility condition, or adjust the set of one or more values to be same as a third set of values associated with the bad visibility condition. In some embodiments, the autonomous vehicle is determined to operate in the bad visibility condition in response to determining that the location of the autonomous vehicle is within a pre-determined distance of a canyon or a tunnel. In some embodiments, the determining the visibility related condition includes determining whether the environment is associated with either a good visibility condition or a bad visibility condition based on a date and time when the autonomous vehicle is operating in the environment, where the adjusting the set of one or more values includes: adjusting the set of one or more values to be same as a second set of values associated with the good visibility condition, or adjusting the set of one or more values to be same as a third set of values associated with the bad visibility condition.

In some embodiments, the one or more variables includes a maximum speed for the autonomous vehicle or a minimum distance between the autonomous vehicle and another vehicle located in a same lane as the autonomous vehicle and immediately in front of the autonomous vehicle. In some embodiments, the adjusting the set of one or more values is based the visibility related condition and a number of lanes on a road on which the autonomous vehicle is operating. In some embodiments, the causing the driving related operations of the one or more devices includes: sending instructions to apply brakes on the autonomous vehicle upon determining that a current speed of the autonomous vehicle is greater than a maximum speed or upon determining that a distance between the autonomous vehicle and another vehicle immediately in front of the autonomous vehicle is less than a minimum distance, and where the one or more values includes the maximum speed and the minimum distance.

In this document the term "exemplary" is used to mean "an example of" and, unless otherwise stated, does not imply an ideal or a preferred embodiment. In the patent document the term "semi-trailer truck" is used to describe features of the visibility determination and the driving operation of the semi-trailer truck. However, the visibility determination techniques described can be applied to other kinds of vehicles.

While this document contains many specifics, these should not be construed as limitations on the scope of an invention that is claimed, but rather as descriptions of features specific to particular embodiments. The scope of the present invention is determined by the attached claims.

Claim 1:
A method of autonomous driving operation, comprising:
determining, by a computer (<NUM>) located in an autonomous vehicle (<NUM>), a visibility related condition of an environment in which the autonomous vehicle is operating;
adjusting, based at least on the visibility related condition, a set of one or more values of one or more variables associated with a driving related operation of the autonomous vehicle; and
causing the autonomous vehicle to be driven to a destination by causing the driving related operation of one or more devices located in the autonomous vehicle based on at least the set of one or more values,
wherein the autonomous vehicle is determined to operate in the bad visibility condition in response to determining that the location of the autonomous vehicle is within a pre-determined distance of a canyon or a tunnel.