Patent ID: 12227039

DETAILED DESCRIPTION

As required, detailed embodiments of the present disclosure are disclosed herein; however, it is to be understood that the disclosed embodiments are merely exemplary and may be embodied in various and alternative forms. The figures are not necessarily to scale; some features may be exaggerated or minimized to show details of particular components. Therefore, specific structural and functional details disclosed herein are not to be interpreted as limiting, but merely as a representative basis for teaching one skilled in the art to variously employ the present disclosure.

Vehicles include external components that wear over time, e.g., tires. Many vehicle manufacturers recommend that drivers inspect vehicle tires for signs of wear or damage, and confirm tire pressure as part of a pre-trip vehicle inspection. For autonomous vehicles (AVs), a driver is not always present to perform such inspection. Tire wear is dependent on many factors including: driving surface, driving speeds, dynamic driving conditions, amount of low-speed steering adjustments, etc. For a fleet of autonomous vehicles, tire tread and pressure may be inspected as part of regular AV “pre-drive” fleet operations. These inspections may be manually intensive and require a significant amount of data entry, which may be prone to human error. Furthermore, as tires wear and tire tread depth decreases, vehicle traction decreases. Accordingly, the proposed systems and methods of the present disclosure provide solutions to monitor vehicle tire conditions, using one or more sensors of the sensor system.

AVs are equipped with a vast array of advanced sensors which are designed to detect physical characteristics of the environment surrounding the AV. The present disclosure proposes using some of the advanced sensors, e.g., the lidar and camera sensors positioned near the front tires to monitor tire conditions, such as tread wear, alignment, and inflation, when the front tires are rotated into the field of view (FOV) of the sensors.

By observing the raw sensor data when a portion of the tire tread is within the sensor FOV, a correlation between the sensor data and tire tread depth can be determined. The tire tread depth may be used as one indicator of tire wear. In this way, an AV system monitors the tires directly, and may provide alerts to fleet operators when the sensor data indicates that the tires or associated steering systems need servicing.

In accordance with aspects of the disclosure, a lidar sensor for monitoring proximate objects, i.e., a near-field lidar sensor, is mounted aft and above each front tire such that when the front tires are rotated to a large steering angle, a portion of the tire tread is within the corresponding lidar sensor FOV. The AV system uses lidar returns, i.e., light pulses that reflect from the tire as it is rotated into the FOV, as well as returns while the tire is stationary in the FOV, to determine the tire conditions.

A sensor that is sensitive and accurate enough to measure the tire tread depth directly with high enough resolution and accuracy may be cost prohibitive, or not ideally suited for the primary objective of an AV sensor for monitoring the external driving environment during driving.

The present disclosure utilizes the existing lidar sensor of the AV system with a statistical analysis of the lidar data returns to determine the variation in tire tread depth. For 5-10 lidar sweeps per second, when sampling over several seconds, a random spatial sampling of the tire can be achieved. There will be lidar returns from down in the tread, as well as from the outer edge of the tire tread. The present disclosure also uses a probability distribution of measurements that varies with tire wear, e.g., for tires with deep tread depth, the standard deviation of measurements will be greater than for tires where there is very little height difference between the bottom of the tread pattern and the remaining exposed tire tread.

The AV system uses statistical methods that analyze the probability distribution of the average distance for the tire surface in the FOV as a correlation to the tread depth. For example, the AV system may analyze the first, second, third, and/or fourth moment of the probability distribution, which correspond to the standard deviation, the variation, the skew, and the kurtosis, respectively. Specifically, when tire tread is deep, i.e., the tire is new and not worn, the statistical variation from a series of sequential lidar sweeps will be greater for deeper tire tread depths than when the tire is worn. In this way, it is not necessary to measure the tread depth directly. Rather, with repeated lidar sweep samples of the tire tread, the variation of measured distances for lidar returns can be used as a proxy for the tread depth. When tires have a deep tread depth, i.e., new tires, the standard deviation will be greater. When the tire tread depth is shallow, i.e., worn tires, the standard deviation will be much less.

In yet another aspect of the disclosure, tires can be monitored for uneven tread wear, which may be a sign of incorrect tire inflation or incorrect steering alignment (e.g., toe, camber, etc.). By segmenting the lidar returns into different portions of the tire, lidar measurements can be used to determine if a tire is wearing on the outer edge, in the middle tread pattern, or on the inside edge of the tire. Said another way, aspects of the present disclosure include a system configured to detect tread depth differences across various regions of a single tire.

In accordance with aspects of the disclosure, the AV system may control a camera sensor in conjunction with an external illumination source, to observe the shadow patterns of the grooves in tire tread patterns. A new tire with deep tread depth, will cast a larger shadow from the same illumination source compared to a tire with a shallow tread depth.

Many tires also are equipped with wear bar indicators that are spaced between the treads. By observing when a wear bar is even with the tread, it is possible to determine the amount of remaining tire tread depth. Using a camera to visually inspect a tire in conjunction with an external light source, the tire tread depth can be inferred by either the presence or the absence of the shadows cast by the wear bar indicators. There are several variations that are more fully described in more detail below.

With reference toFIG.1, a sensor system for monitoring tire conditions is illustrated in accordance with aspects of the disclosure and generally referenced by numeral100. The sensor system100includes multiple sensor assemblies, including a side sensor assembly102, that are mounted to an autonomous vehicle (AV)104. The side sensor assembly102includes one or more sensors106to monitor a field-of-view (FOV)108, represented by dashed lines inFIG.1, proximate to the AV104. The sensor106monitors tire conditions, e.g., wear, alignment, and inflation, during turning maneuvers when a tire110is within the FOV108.

The sensor system100includes multiple sensor assemblies to monitor a 360-degree FOV around the AV104, both in the near-field and the far-field. The sensor system100includes the side sensor assembly102, a top sensor assembly112, a front central sensor assembly114, two front side sensor assemblies116, and one or more rear sensor assemblies118, according to aspects of the disclosure. Each sensor assembly includes one or more sensors, e.g., a camera, a lidar sensor, and a radar sensor.

The side sensor assembly102may be mounted to a side of the AV104. In the example ofFIG.1, the side sensor assembly102is mounted to a side mirror assembly120. The side sensor assembly102includes one or more sensors106, e.g., a lidar sensor and a camera, that is oriented downward toward the ground to monitor the FOV108, according to aspects of the disclosure. The sensor106also monitors tire conditions when the tire110is turned into the FOV108. The sensor system100also includes a second side sensor assembly (not shown) that is mounted to an opposite side of the AV104for monitoring the opposing front tire.

The top sensor assembly112is mounted to a roof of the AV104and includes a lidar sensor and one or more cameras. The lidar sensor rotates about a vertical axis to scan a 360-degree FOV about the AV104in a far-field. The front central sensor assembly114is mounted to the front of the AV104, e.g., to the hood or bumper, and includes at least a radar sensor for monitoring a front FOV for large objects, e.g., vehicles, in front of the AV104. The front central sensor assembly114may also include one or more cameras. The front side sensor assemblies116and the rear sensor assemblies118each include a camera and/or a lidar sensor for monitoring the FOVs in front of and behind the AV104.

FIG.2illustrates communication between an AV system200and other systems and devices according to aspects of the disclosure. The sensor system100is included in the AV system200, and communicates with a controller202through a transceiver204. The sensor system100includes multiple sensor assemblies, e.g., the side sensor assembly102and the top sensor assembly112. Each sensor assembly102,112includes one or more sensors, e.g., a lidar sensor206, a camera208, and a radar sensor (not shown). The camera208may be a visible spectrum camera, an infrared camera, etc., according to aspects of the disclosure, to capture images of the tire110. The sensor system100may include additional sensors, such as a sound navigation and ranging (SONAR) sensor, temperature sensors, position sensors (e.g., global positioning system (GPS), etc.), location sensors, fuel sensors, motion sensors (e.g., inertial measurement units (IMU), etc.), humidity sensors, occupancy sensors, or the like. The sensor system100provides sensor data209that is indicative of the external environment of the AV104. The controller202analyzes the sensor data to identify and determine the location of external objects relative to the AV104, e.g., the location of traffic lights, remote vehicles, pedestrians, etc.

The AV system200also communicates with one or more vehicle systems210, e.g., an engine, a transmission, a navigation system, a brake system, etc. through the transceiver204. The controller202may receive information from the vehicle systems210that is indicative of present operating conditions, e.g., vehicle speed, engine speed, turn signal status, brake position, vehicle position, steering angle, and tire identification. The controller202may also control one or more vehicle systems210, e.g., a propulsion system, a braking system, and a steering system, based on the sensor data209from the sensor system100. The controller202may communicate directly with the vehicle systems210, or communicate indirectly with the vehicle systems210over a vehicle communication bus e.g., a CAN bus212.

The AV system200may also communicate with external objects214, e.g., remote vehicles and structures, to share the external environment information and/or to collect additional external environment information. The AV system200may include a vehicle-to-everything (V2X) transceiver216that is connected to the controller202for communicating with the objects214. For example, the AV system200may use the V2X transceiver216for communicating directly with a remote vehicle vehicle-to-vehicle (V2V) communication, a structure (e.g., a sign, a building, or a traffic light) by vehicle-to-infrastructure (V2I) communication, and a motorcycle by vehicle-to-motorcycle (V2M) communication. Each V2X device may provide information indictive of its own status, or the status of another V2X device.

The AV system200may communicate with a remote computing device218over a communications network220using one or more of the transceivers204,216, e.g., to provide a message or visual that indicates the location of the objects214relative to the AV104, and current tire conditions, based on the sensor data209. The remote computing device218may include one or more servers to process one or more processes of the technology described herein. The remote computing device218may also communicate data with a database222over the network220. AV fleet operators may monitor the status of an AV fleet using the remote computing device218to receive alerts that indicate that the tires or associated steering systems of the AV104need servicing.

The AV system200also includes a user interface224to provide information to a user of the AV104. The controller202may control the user interface224to provide a message or visual that indicates the location of the objects214relative to the AV104, and current tire conditions, based on the sensor data209.

Although the controller202is described as a single controller, it may contain multiple controllers, or may be embodied as software code within one or more other controllers. The controller202includes a processing unit, or processor226, that may include any number of microprocessors, ASICs, ICs, memory (e.g., FLASH, ROM, RAM, EPROM and/or EEPROM) and software code to co-act with one another to perform a series of operations. Such hardware and/or software may be grouped together in assemblies to perform certain functions. Any one or more of the controllers or devices described herein include computer executable instructions that may be compiled or interpreted from computer programs created using a variety of programming languages and/or technologies. The controller202also includes memory228, or non-transitory computer-readable storage medium, that is capable of executing instructions of a software program. The memory228may be, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semi-conductor storage device, or any suitable combination thereof. In general, the processor226receives instructions, for example from the memory228, a computer-readable medium, or the like, and executes the instructions. The controller202, also includes predetermined data, or “look up tables” that are stored within memory, according to aspects of the disclosure.

FIG.3illustrates an exemplary architecture of a lidar sensor300, such as the lidar sensor206of the side sensor assembly102, according to aspects of the disclosure. The lidar sensor300includes a base302that is mounted to the AV104, e.g., to the side mirror assembly120. The base302includes a motor304with a shaft306that extends along a longitudinal axis A-A. The lidar sensor300also includes a housing308that is secured to the shaft306and mounted for rotation relative to the base302about Axis A-A. The housing308includes an opening310and a cover312that is secured within the opening310. The cover312is formed of a material that is transparent to light, e.g., glass. Although a single cover312is shown inFIG.3, the lidar sensor300may include multiple cover s312.

The lidar sensor300includes one or more emitters316for transmitting light pulses320through the cover312and away from the AV104. The light pulses320are incident on one or more objects, e.g., the tire110, and reflect back toward the lidar sensor300as reflected light pulses328. The lidar sensor300also includes one or more light detectors318for receiving the reflected light pulses328that pass through the cover312. The detectors318also receive light from external light sources, e.g., the sun. The lidar sensor300rotates about Axis A-A to scan the region within the FOV108. The lidar sensor300may rotate 360 degrees about the axis, and ignore data reflected off of the AV104. The emitters316and the detectors318may be stationary, e.g., mounted to the base302, or dynamic and mounted to the housing308.

The emitters316may include laser emitter chips or other light emitting devices and may include any number of individual emitters (e.g., 8 emitters, 64 emitters, or 128 emitters). The emitters316may transmit light pulses320of substantially the same intensity or of varying intensities, and in various waveforms, e.g., sinusoidal, square-wave, and sawtooth. The lidar sensor300may include one or more optical elements322to focus and direct light that is passed through the cover312.

The detectors318may include a photodetector, or an array of photodetectors, that is positioned to receive the reflected light pulses328. According to aspects of the disclosure, the detectors318include a plurality of pixels, wherein each pixel includes a Geiger-mode avalanche photodiode, for detecting reflections of the light pulses during each of a plurality of detection frames. In other embodiments, the detectors318include passive imagers.

The lidar sensor300includes a controller330with a processor332and memory334to control various components, e.g., the motor304, the emitters316, and the detectors318. The controller330also analyzes the data collected by the detectors318, to measure characteristics of the light received, and generates information about the environment external to the AV104. The controller330may be integrated with another controller, e.g., the controller202of the AV system200. The lidar sensor300also includes a power unit336that receives electrical power from a vehicle battery338, and supplies the electrical power to the motor304, the emitters316, the detectors318, and the controller330.

Referring collectively toFIGS.3-4, the emitters316of the lidar sensor300emit light pulses320that are incident on a tire410and reflect back as reflected light pulses328. The detectors318receive the reflected light pulses328and provide corresponding sensor data to the controller330. The controller330may determine a distance between the lidar sensor300and the tire410based on the sensor data.

The tire410includes tread412that is separated into segments414by longitudinal grooves416and lateral grooves418. As the lidar sensor206rotates, the light pulses320scan laterally across the tire410to form scanlines or scan patterns420, e.g., one of the emitters316generates a first scan pattern422. The controller330may determine a height of the tread412along one of the scan patterns420based on changes in the distance measurements along one of the scan patterns420, e.g., the difference between the distance to the tread412at a first segment424as compared to the distance to the longitudinal groove416at point426.

FIG.4illustrates a new tire410andFIG.5illustrates a worn tire510, which may be representative of the new tire410after it has been driven for an extended distance, e.g., more than 60,000 miles. Like the new tire410, the worn tire510includes tread512that is separated into segments514by longitudinal grooves516and lateral grooves518. As the lidar sensor300rotates, the light pulses320scan laterally across the tire to form scan patterns520. The controller330may determine the tread height for the worn tire510based on changes in the distance measurements along one of the scan patterns520, e.g., the difference between the distance to the tread512and the distance to the longitudinal grooves516or the lateral grooves518. The height of the tread412of the new tire410is greater than the height of the tread512of the worn tire510, therefore the distance measurement to the new tread412is less than the distance measurement to the worn tread512.

FIG.6is a graph600illustrating a comparison of distance measurements based on lidar sensor data of the new tire410to the worn tire510. The graph600includes a first curve602that represents the distance measurement to the tread412, in meters (m), of the new tire410along the first scan pattern422, and a second curve604that represents the distance measurement (m) to the tread512along the first scan pattern522. While meters are provided as an example distance increment, it should be appreciated that other distance increments may be suitable according to aspects of the present disclosure. The horizontal axis of the graph600represents a lateral distance across each tire410,510. For example, with reference toFIG.8, zero (0) represents a central location832. Similarly, 20 mm and −20 mm represent midpoints838and840. Along the same convention, −40 mm represents the inner edge836.

Referring back toFIG.6, the curves602,604overlap due to the sensitivity of the lidar sensor300, which makes it difficult to detect tire wear using a direct measurement approach. This is because the lidar sensors300are configured to monitor the external driving environment for large objects, e.g., vehicles or pedestrians, that are greater distances from the AV104as opposed to monitoring wear in small increments, e.g., in millimeters (mm).

FIGS.7-9illustrate a statistical analysis of the measured distance to a tire.FIG.7illustrates a new tire710andFIG.8illustrates a worn tire810, which may represent the new tire710after it has been driven a large distance, e.g., more than 60,000 miles. The new tire710includes tread712that is separated into segments714by longitudinal grooves716and lateral grooves718. As the lidar sensor300rotates, the light pulses320scan laterally across the new tire710to form scan patterns720. The controller330analyzes the scan patterns720from multiple sequential sweeps across the new tire710, e.g., 10-100 sweeps, using statistical methods to analyze the probability distribution, e.g., the standard deviation, the variance, the skew, and/or the kurtosis of the average distance for the tire surface in the FOV as a correlation to the tread depth.

The worn tire810also includes tread812that is separated into segments814by longitudinal grooves816and lateral grooves818. As the lidar sensor300rotates, the light pulses320scan laterally across the worn tire810to form scan patterns820. The controller330analyzes the scan patterns820from multiple sequential sweeps across the worn tire810, e.g., 10-100 sweeps, using statistical methods that analyze the probability distribution, e.g., the standard deviation, the variance, the skew, and/or the kurtosis of the average distance for the tire surface in the FOV as a correlation to the tread depth.

The worn tire810may be segmented into multiple regions across the lateral surface of the tire, as depicted a dashed box830inFIG.8. The segments may be arranged relative to a central location832, an outer edge834, an inner edge836, and midpoints838,840between the central location832and the edges834,836, respectively. The regions include a central region842between the midpoints838and840, an outer region844between the outer edge834and midpoint838, and an inner region846between the inner edge836and midpoint840. By segmenting the tire810into multiple regions, the controller330may analyze the sensor data to determine multiple tire conditions, e.g., wear, alignment, and inflation.

FIG.9is a graph900illustrating a statistical model of the standard deviation of distance measurements over multiple sequential scans of the new tire710and the worn tire810. The horizontal axis of the graph900represents a lateral distance across each of the new tire710and the worn tire810. The graph900includes a first curve902that represents the range error standard deviation of lidar returns per scan of the new tire710along a first scan pattern722, and a second curve904that represents the range error standard deviation of lidar returns per scan of the worn tire810along a first scan pattern822. Relative to the direct measurements reflected above in reference to graph600, the curves902,904do not significantly overlap in the graph900, which makes it easier to distinguish the worn tire810from the new tire710.

With reference toFIGS.6and9, measuring tire conditions, e.g., tread wear, directly using lidar sensor distance data from an AV system200may be difficult, as shown by the overlapping curves602,604in graph600. This is because the lidar sensor is designed to monitor for large objects spaced at a distance apart from the AV, as opposed to detecting slight variation in tire tread depth. A lidar sensor that is sensitive and accurate enough to measure the tire tread depth directly with high enough resolution and accuracy may be cost prohibitive, and/or not ideally suited for the primary objective of an AV sensor for monitoring the external driving environment during driving. However, as shown in graph900, the controller330may use statistical methods that analyze the probability distribution (e.g., the standard deviation of the average distance for the tire surface in the FOV) as a correlation to the tire tread depth.

Specifically, when tire tread is deep, such as the tread712of the new tire710, the standard deviation from a series of sequential lidar scans (first curve902), will be greater than the standard deviation from a series of sequential lidar scans of the worn tire810, as represented by the second curve904. In this way, it is not necessary to measure the tread depth directly. Rather, with repeated samples of the tire tread, the probability distribution (e.g., the standard deviation of measured distances for lidar returns) can be used as a proxy for the tread depth. When tires have a deep tread depth, e.g., the new tire710, the standard deviation will be greater, as shown in curve902. When the tire tread depth is lower, e.g., the worn tire810, the standard deviation will be much less. AlthoughFIG.9illustrates one moment of the probability distribution, i.e., the standard deviation, the AV system200may analyze other moments of the probability distribution of the average distance for the tire surface in the FOV as a correlation to the tread depth, according to aspects of the disclosure. For example, the AV system200may analyze the variance, the skew, and/or the kurtosis, because, like the standard deviation, these moments tend to smaller values as tread depth decreases due to tire wear.

With reference toFIG.10, a flow chart depicting a method for monitoring tire conditions based on lidar sensor data is illustrated in accordance with one or more embodiments and is generally referenced by numeral1000. The method1000is implemented using software code that is executed by the processor332and contained within the memory334(FIG.3) according to aspects of the disclosure. While the flowchart is illustrated with a number of sequential steps, one or more steps may be omitted and/or executed in another manner without deviating from the scope and contemplation of the present disclosure.

At step1002, the AV system200controls the position of the front tire to a predetermined steering angle, e.g., 40 degrees. The predetermined steering angle is great enough that the tire tread is within a FOV of at least one sensor. Then, at step1004, the AV system200takes lidar distance measurements for a predetermined number of sweeps, e.g., 10-100 while the tire is positioned at the predetermined steering angle. In other embodiments, the AV system200monitors the steering angle during AV operation and takes the measurements of the tire tread after the steering angle is equal to a predetermined angle.

At step1006, the AV system200performs a statistical analysis of the lidar sweeps to determine the probability distribution, e.g., the range error standard deviation or variation. At step1008, the AV system200evaluates the variation to determine if the variation is less than a predetermined threshold variation. For example, the graph900ofFIG.9illustrates a threshold variation of 2 mm, which is referenced by numeral906. If the variation is less than the threshold variation for all regions of the tire, i.e., the central region842, the outer region844, and the inner region846of the worn tire810(FIG.8), the AV system200proceeds to step1010and sends a signal indicative of a worn tire message. The AV system200may provide the worn tire message to a user interface224to display to a user, or to a vehicle system210or a remote computing device218. The threshold may be different for different segments of the tire, e.g., 2 mm for a central region of the tire, and 4 mm for an outer region of the tire. If the AV system200determines that the variation is not less than the threshold for all regions, it proceeds to step1011.

At step1011, the AV system200determines if the variation is less than the threshold within the outer region844and the inner region846of the tire810. If the variation is less than the threshold within both the outer region844and the inner region846, the AV system200proceeds to step1012and issues a signal indicative of an underinflation message. If the AV system200determines that the variation is not less than the threshold for both the inner and outer region, it proceeds to step1013.

At step1013, the AV system200determines if the variation is less than the threshold within the outer region844or the inner region846of the tire810. If the variation is less than the threshold within either the outer region844or the inner region846, the AV system200proceeds to step1014and issues a signal indicative of a misalignment message. If the AV system200determines that the variation is not less than the threshold for the inner or outer region, it proceeds to step1016.

At step1016, the AV system200determines if the variation is less than the threshold within the central region842of the tire810. If the variation is less than the threshold within the central region842, the AV system200proceeds to step1018and issues a signal indicative of an overinflation message. If the AV system200determines that the variation is not less than the threshold for the central region, it returns to step1002.

The controller330may save tire condition data for each tire and compare tire condition data over time. For example, the controller330may receive tire identification from a vehicle tire pressure monitoring system that allows the controller330to identify a tire, even if it is moved or rotated to another position on the vehicle.

With reference toFIG.11, a sensor system for monitoring tire conditions is illustrated in accordance with aspects of the disclosure and generally referenced by numeral1100. The sensor system1100coordinates with an external lighting system1101to monitor tire conditions. The sensor system1100includes multiple sensor assemblies, including a side sensor assembly1102, that are mounted to an autonomous vehicle (AV)1104. The side sensor assembly1102includes one or more sensors1106to monitor a field-of-view (FOV)1108that is proximate to the AV1104. The sensor1106monitors tire conditions, e.g., wear, alignment, and inflation, during turning maneuvers when a tire1110enters the FOV1108, e.g., such that 80% of the width of the tire is within the FOV. The one or more sensors1106include a camera1112. The camera1112is a near field (NF) camera to capture images of the tire1110. The camera1112is controlled to take a series of pictures at a frame rate of at least 10 frames per second, according to aspects of the disclosure.

The external lighting system1101includes a light source1114that is arranged to project light1115onto the tire1110to generate shadows that are used to monitor the tire conditions. The sensor system1100utilizes sensor data from one or more of the sensor assemblies to park the AV1104such that the tire1110is located at a predetermined position relative to the light source1114. The external lighting system1101may also include a fixture1116that is sized to receive and locate the tire1110relative to the light source1114. The external lighting system1101also includes a second light source (not shown) for monitoring the front—right tire (not shown).

With reference toFIGS.12-14, the tire tread creates shadows that vary based on tire conditions.FIG.12illustrates a new tire1210with tread1212that is separated into segments by longitudinal grooves1216. The tread1212blocks light1215from the light source1114(shown inFIG.11) when it is arranged at a first steering angle (α1) to generate a shadow with a first shadow length (SN1) that is monitored by the camera1112. The tread1212also blocks light1218from the light source1114when it is arranged at a second steering angle (α2) to generate a shadow with a second shadow length (SN2) that is monitored by the camera1112.FIG.13illustrates a worn tire1310arranged at the first steering angle (α1), with tread1312that is separated into segments by longitudinal grooves1316. The tread1312blocks light1315from the light source1114at the first steering angle (α1) to generate a shadow with a first shadow length (SW1) that is monitored by the camera1112.FIG.14illustrates the worn tire1310arranged at the second steering angle (α2), with tread1312that is separated into segments by longitudinal grooves1316. The tread1312blocks light1415from the light source1114at the second steering angle (α2) to generate a shadow with a second shadow length (SW2) that is monitored by the camera1112.

The AV system200compares the shadow length to predetermined data that correlates shadow length with tire conditions, such as wear. The AV system200may also compare the shadow length against previous measurements to determine a change in tread depth indicative of wear. The tread1212may be separated into regions (as shown inFIG.8) and the predetermined data may correlate certain tire conditions with shadow length at different regions. For example, low shadow length across the face of the tire1210may indicate wear, whereas low shadow length at one edge of the tire may indicate misalignment, and low shadow length at the center may indicate overinflation. As shown inFIGS.12-14, the shadow length (SN1, SN2) of the new tire1210at both the first steering angle (α1) and the second steering angle (α2) is large due to the deep tread1212. The shadow length (SW1) of the worn tire1310at α1is less than the shadow length (SN1) of the new tire1210at α1due to the shallow tread1312.FIG.14further illustrates that the shadow length (SW2) of the worn tire1310at the second steering angle (α2) is greater than the shadow length (SW1) of the worn tire1310at the first steering angle (α1) due to the worn tire1310turning away from the light1415.

With reference toFIG.15a flow chart depicting a method for monitoring tire conditions based on camera sensor data is illustrated in accordance with one or more embodiments and is generally referenced by numeral1500. The method1500is implemented using software code that is executed by the processor332and contained within the memory334(FIG.3) according to aspects of the disclosure. While the flowchart is illustrated with a number of sequential steps, one or more steps may be omitted and/or executed in another manner without deviating from the scope and contemplation of the present disclosure.

At step1502, the AV system200controls the AV104to park at a predetermined location relative to the light source1114. At step1504, the AV system200turns the tire1110to a first steering angle, e.g., 5 degrees, and takes a first picture. At step1506, the AV system200turns the tire1110to a second steering angle, e.g., 40 degrees, and takes a second picture. At step1508, the AV system200determines a shadow length for each picture. Then at step1510, the AV system200determines a rate of change of the shadow length. At step1512, the AV system200compares the shadow length rate of change to a threshold rate of change corresponding to a predetermined rate of change of the steering angle. If the shadow length rate of change exceeds the threshold rate of change, the AV system200proceeds to step1514and issues a worn tire message.

According to aspects of the disclosure, the AV system200may control the camera1112to take a series of pictures as it controls the tire1210to rotate toward the light source1114between two steering angles at a constant rate over a period of time, to calculate the rate of change of shadow length based on many data points. For example, the camera1112may take pictures at a rate of 10 frames/second as the AV system200controls the tire1210to rotate through its maximum steering angle range, e.g., from 0 to 40 degrees at a constant rate over a three second period of time. The AV system200may filter the rate of change of the shadow length to remove any outliers.

The AV system200monitors both front tires, according to aspects of the disclosure. For example, the AV system200may rotate the front left tire from its maximum steering wheel angle to its minimum steering wheel angle (which corresponds to the maximum steering wheel angle for the front right tire) at a constant rate while taking pictures at a frame rate of 10 frames/second. Then the AV system200may analyze the images from both cameras to evaluate both tires.

A sensor that is sensitive and accurate enough to measure the tire tread depth directly with high enough resolution and accuracy may be cost prohibitive, or not ideally suited for the primary objective of an AV sensor for monitoring the external driving environment during driving. The AV system100utilizes existing sensors, e.g., the lidar sensor206and/or the camera208, to monitor tire conditions. The AV system100uses the existing lidar sensor206with a statistical analysis of the lidar data returns to determine the variation in tire tread depth. The AV system100also uses the existing camera208in cooperation with an external lighting system1101to monitor tire wear.

The sensor system100may be implemented in an AV system200, which includes one or more controllers, such as computer system1600shown inFIG.16. The computer system1600may be any computer capable of performing the functions described herein. The computer system1600also includes user input/output interface(s)1602and user input/output device(s)1603, such as buttons, monitors, keyboards, pointing devices, etc.

The computer system1600includes one or more processors (also called central processing units, or CPUs), such as a processor1604. The processor1604is connected to a communication infrastructure or bus1606. The processor1604may be a graphics processing unit (GPU), e.g., a specialized electronic circuit designed to process mathematically intensive applications, with a parallel structure for parallel processing large blocks of data, such as mathematically intensive data common to computer graphics applications, images, videos, etc.

The computer system1600also includes a main memory1608, such as random-access memory (RAM), that includes one or more levels of cache and stored control logic (i.e., computer software) and/or data. The computer system1600may also include one or more secondary storage devices or secondary memory1610, e.g., a hard disk drive1612; and/or a removable storage device1614that may interact with a removable storage unit1618. The removable storage device1614and the removable storage unit1618may be a floppy disk drive, a magnetic tape drive, a compact disk drive, an optical storage device, tape backup device, and/or any other storage device/drive.

The secondary memory1610may include other means, instrumentalities or other approaches for allowing computer programs and/or other instructions and/or data to be accessed by computer system1600, e.g., an interface1620and a removable storage unit1622, e.g., a program cartridge and cartridge interface (such as that found in video game devices), a removable memory chip (such as an EPROM or PROM) and associated socket, a memory stick and USB port, a memory card and associated memory card slot, and/or any other removable storage unit and associated interface.

The computer system1600may further include a network or communication interface1624to communicate and interact with any combination of remote devices, remote networks, remote entities, etc. (individually and collectively referenced by reference number1628). For example, the communication interface1624may allow the computer system1600to communicate with remote devices1628over a communication path1626, which may be wired and/or wireless, and which may include any combination of LANs, WANs, the Internet, etc. The control logic and/or data may be transmitted to and from computer system1600via communication path1626.

In an embodiment, a tangible, non-transitory apparatus or article of manufacture comprising a tangible, non-transitory computer useable or readable medium having control logic (software) stored thereon is also referred to herein as a computer program product or program storage device. This includes, but is not limited to, the computer system1600, the main memory1608, the secondary memory1610, and the removable storage units1618and1622, as well as tangible articles of manufacture embodying any combination of the foregoing. Such control logic, when executed by one or more data processing devices (such as the computer system1600), causes such data processing devices to operate as described herein.

The term “vehicle” refers to any moving form of conveyance that is capable of carrying either one or more human occupants and/or cargo and is powered by any form of energy. The term “vehicle” includes, but is not limited to, cars, trucks, vans, trains, autonomous vehicles, aircraft, aerial drones and the like. An “autonomous vehicle” (or “AV”) is a vehicle having a processor, programming instructions and drivetrain components that are controllable by the processor without requiring a human operator. An autonomous vehicle may be fully autonomous in that it does not require a human operator for most or all driving conditions and functions, or it may be semi-autonomous in that a human operator may be required in certain conditions or for certain operations, or that a human operator may override the vehicle's autonomous system and may take control of the vehicle. Notably, the present solution is being described herein in the context of an autonomous vehicle. However, the present solution is not limited to autonomous vehicle applications. The present solution may be used in other applications such as robotic applications, radar system applications, metric applications, and/or system performance applications.

Based on the teachings contained in this disclosure, it will be apparent to persons skilled in the relevant art(s) how to make and use embodiments of this disclosure using data processing devices, computer systems and/or computer architectures other than that shown inFIG.16. In particular, embodiments can operate with software, hardware, and/or operating system implementations other than those described herein.

It is to be appreciated that the Detailed Description section, and not any other section, is intended to be used to interpret the claims. Other sections can set forth one or more but not all exemplary embodiments as contemplated by the inventor(s), and thus, are not intended to limit this disclosure or the appended claims in any way.

While this disclosure describes exemplary embodiments for exemplary fields and applications, it should be understood that the disclosure is not limited thereto. Other embodiments and modifications thereto are possible, and are within the scope and spirit of this disclosure. For example, and without limiting the generality of this paragraph, embodiments are not limited to the software, hardware, firmware, and/or entities illustrated in the figures and/or described herein. Further, embodiments (whether or not explicitly described herein) have significant utility to fields and applications beyond the examples described herein.

Embodiments have been described herein with the aid of functional building blocks illustrating the implementation of specified functions and relationships thereof. The boundaries of these functional building blocks have been arbitrarily defined herein for the convenience of the description. Alternate boundaries can be defined as long as the specified functions and relationships (or equivalents thereof) are appropriately performed. Also, alternative embodiments can perform functional blocks, steps, operations, methods, etc. using orderings different than those described herein.

References herein to “one embodiment,” “an embodiment,” “an example embodiment,” or similar phrases, indicate that the embodiment described can include a particular feature, structure, or characteristic, but every embodiment can not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it would be within the knowledge of persons skilled in the relevant art(s) to incorporate such feature, structure, or characteristic into other embodiments whether or not explicitly mentioned or described herein. Additionally, some embodiments can be described using the expression “coupled” and “connected” along with their derivatives. These terms are not necessarily intended as synonyms for each other. For example, some embodiments can be described using the terms “connected” and/or “coupled” to indicate that two or more elements are in direct physical or electrical contact with each other. The term “coupled,” however, can also mean that two or more elements are not in direct contact with each other, but yet still co-operate or interact with each other. The breadth and scope of this disclosure should not be limited by any of the above-described exemplary embodiments, but should be defined only in accordance with the following claims and their equivalents.

While exemplary embodiments are described above, it is not intended that these embodiments describe all possible forms of the disclosure. Rather, the words used in the specification are words of description rather than limitation, and it is understood that various changes may be made without departing from the spirit and scope of the disclosure. Additionally, the features of various implementing embodiments may be combined to form further embodiments.