Patent ID: 12260746

DETAILED DESCRIPTION

Exemplary embodiments of the technology are described below. It should be understood that these are illustrative embodiments and that the invention is not limited to these particular embodiments.

FIG.1shows an exemplary OBU containing a communication module101, a data collection module102, and a vehicle control module103. The data collection module102collects data related to a vehicle and a human104and then sends it104to an RSU through communication module101. Also, OBU can receive data of RSU105through communication module101. Based on the data of RSU105, the vehicle control module103helps control the vehicle.

FIG.2illustrates an exemplary framework of a lane management sensing system and its data flow.

The RSU exchanges information between the vehicles and the road and communicates with TCUs, the information including weather information, road condition information, lane traffic information, vehicle information, and incident information.

FIG.3illustrates exemplary workflow of a basic prediction process of a lane management sensing system and its data flow. In some embodiments, fused multi-source data collected from vehicle sensors, roadside sensors and the cloud is processed through models including but not limited to learning based models, statistical models, and empirical models. Then predictions are made at different levels including microscopic, mesoscopic, and macroscopic levels using emerging models including learning based, statistic based, and empirical models.

FIG.4shows exemplary planning and decision making processes in an IRIS. Data401is fed into planning module402according to three planning level respectively407,408, and409. The three planning submodules retrieve corresponding data and process it for their own planning tasks. In a macroscopic level404, route planning and guidance optimization are performed. In a mesoscopic level405, special event, work zone, reduced speed zone, incident, buffer space, and extreme weather are handled. In a microscopic level406, longitudinal control and lateral control are generated based on internal algorithm. After computing and optimization, all planning outputs from the three levels are produced and transmitted to decision making module403for further processing, including steering, throttle control, and braking.

FIG.5shows exemplary data flow of an infrastructure automation based control system. The control system calculates the results from all sensing detectors, conducts data fusion, and exchanges information between RSUs and Vehicles. The control system comprises: a) Control Method Computation Module501; b) Data Fusion Module502; c) Communication Module (RSU)503; and d) Communication Module (OBU)504.

FIG.6illustrates an exemplary process of vehicle longitudinal control. As shown in the figure, vehicles are monitored by the RSUs. If related control thresholds (e.g., minimum headway, maximum speed, etc.) are reached, the necessary control algorithms is triggered. Then the vehicles follow the new control instructions to drive. If instructions are not confirmed, new instructions are sent to the vehicles.

FIG.7illustrates an exemplary process of vehicle latitudinal control. As shown in the figure, vehicles are monitored by the RSUs. If related control thresholds (e.g., lane keeping, lane changing, etc.) are reached, the necessary control algorithms are triggered. Then the vehicles follows the new control instructions to drive. If instructions are not confirmed, new instructions are sent to the vehicles.

FIG.8illustrates an exemplary process of vehicle fail safe control. As shown in the figure, vehicles are monitored by the RSUs. If an error occurs, the system sends the warning message to the driver to warn the driver to control the vehicle. If the driver does not make any response or the response time is not appropriate for driver to take the decision, the system sends the control thresholds to the vehicle. If related control thresholds (e.g., stop, hit the safety equipment, etc.) are reached, the necessary control algorithms is triggered. Then the vehicles follows the new control instructions to drive. If instructions are not confirmed, new instructions are sent to the vehicles.

FIG.9shows an exemplary physical component of a typical RSU, comprising a Communication Module, a Sensing Module, a Power Supply Unit, an Interface Module, and a Data Processing Module. The RSU may any of variety of module configurations. For example, for the sense module, a low cost RSU may only include a vehicle ID recognition unit for vehicle tracking, while a typical RSU includes various sensors such as LiDAR, cameras, and microwave radar.

FIG.10shows an exemplary internal data flow within a RSU. The RSU exchanges data with the vehicle OBUs, upper level TCU and the cloud. The data processing module includes two processors: external object calculating Module (EOCM) and AI processing unit. EOCM is for traffic object detection based on inputs from the sensing module and the AI processing unit focuses more on decision-making processes.

FIG.11show an exemplary structure of a TCC/TCU network. A macroscopic TCC, which may or may not collaborate with an external TOC, manages a certain number of regional TCCs in its coverage area. Similar, a regional TCC manages a certain number of corridor TCCs, a corridor TCC manages a certain number of segment TCUs, a segment TCU manages a certain number of point TCUs, and a point TCUs manages a certain number of RSUs. An RSU sends customized traffic information and control instructions to vehicles and receives information provided by vehicles. The network is supported by the services provided by the cloud.

FIG.12shows how an exemplary cloud system communicates with sensors of RSU, TCC/TCU (1201) and TOC through communication layers (1202). The cloud system contains cloud infrastructure (1204), platform (1205), and application service (1206). The application services also support the applications (1203).

FIG.13shows exemplary data collected from sensing module1301such as image data, video data, and vehicle status data. The data is divided into two groups by the data allocation module1302: large parallel data and advanced control data. The data allocation module1302decides how to assign the data1309with the computation resources1303, which are graphic processing units (GPUs)1304and central processing units (CPUs)1305. Processed data1310is sent to prediction1306, planning1307, and decision making modules1308. The prediction module provides results to the planning module1311, and the planning module provides results1312to the decision making module.

FIG.14shows how exemplary data collected from OBUs and RSUs together with control targets and traffic information from upper level IRIS TCC/TCC network1402are provided to a TCU. The lane management module of a TCU produces lane management and vehicle control instructions1403for a vehicle control module and lane control module.

FIG.15shows exemplary data flow for vehicle control in adverse weather. Table 1, below, shows approaches for measurement of adverse weather scenarios.

TABLE 1IRIS Measures for Adverse Weather ScenariosIRISHDMap + TOC +Normal autonomous vehicleRSU(Camera + Radar + Lidar)/OBU can(only sensors)greatly mitigate the impact of adverseCameraRadarLidarweather.Impact inVisibilityDetectingDetectingSolutionSolutionEnhance-adverseofdistancedistancefor degradefor degradement forweatherlines/signs/degraded.degraded.ofof distancevehicleobjectsvisibility.detection.control.degraded.Rain******HDMapRSU has aRSU canprovideswholecontrolinfo of lane/vision ofvehicleSnow*******line/sign/all vehiclesaccordinggeometry,on theto weatherwhichroad, so the(e.g., lowerFog************enhancechance ofthe speedRSU'scrash withon icyvision.otherroad).Sand-************vehicles arestormeliminated.Number of ″*″ means the degree of decrease.

FIG.16shows exemplary IRIS security measures, including network security and physical equipment security. Network security is enforced by firewalls1601and periodically complete system scans at various levels. These firewalls protect data transmission1605either between the system and an Internet1601or between data centers1603and local servers1604. For physical equipment security, the hardware is safely installed and secured by an identification tracker and possibly isolated.

InFIG.17, periodically, IRIS system components1704back up the data to local storage1703in the same Intranet1702through firewall1601. In some embodiments, it also uploads backup copy through firewall1601to the Cloud1701, logically locating in the Internet1702.

FIG.18shows an exemplary periodic IRIS system check for system failure. When failure happens, the system fail handover mechanism is activated. First, failure is detected and the failed node is recognized. The functions of failed node are handed over to shadow system and success feedback is sent back to an upper level system if nothing goes wrong. Meanwhile, a failed system/subsystem is restarted and/or recovered from a most recent backup. If successful, feedback is reported to an upper level system. When the failure is addressed, the functions are migrated back to the original system.

Exemplary hardware and parameters that find use in embodiments of the present technology include, but are not limited to the following:

OBU:

a) Communication module Technical SpecificationsStandard Conformance: IEEE 802.11p-2010Bandwidth: 10 MHzData Rates: 10 MbpsAntenna Diversity CDD Transmit DiversityEnvironmental Operating Ranges: −40° C. to +55° C.Frequency Band: 5 GHzDoppler Spread: 800 km/hDelay Spread: 1500 nsPower Supply: 12/24Vb) Data collection module Hardware technical SpecificationsIntuitive PC User Interface for functions such as configuration, trace, transmit, filter, log etc.High data transfer ratec) Software technical SpecificationsTachograph Driver alerts and remote analysis.Real-Time CAN BUS statistics.CO2 Emissions reporting.d) Vehicle control module Technical SpecificationsLow power consumptionReliable longitudinal and lateral vehicle control
RSU Designa) communication module which include three communication channels:Communication with vehicles including DSRC/4G/5G (e.g., MK5 V2X from Cohda Wireless)Communication with point TCUs including wired/wireless communication (e.g., Optical Fiber from Cablesys)Communication with cloud including wired/wireless communication with at least 20 M total bandwidthb) data Processing Module which include two processors:External Object Calculating Module (EOCM)Process Object detection using Data from the sensing module and other necessary regular calculation (e.g., Low power fully custom ARM/X86 based processor)AI processing UnitMachine learningDecision making/planning and prediction processingc) an interface Module:FPGA based Interface unitFPGA processor that acts like a bridge between the AI processors and the External Object Calculating Module processors and send instructions to the communication modules
The RSU Deploymenta. Deployment locationThe RSU deployment is based on function requirement and road type. An RSU is used for sensing, communicating, and controlling vehicles on the roadway to provide automation. Since the LIDAR and other sensors (like loop detectors) need different special location, some of them can be installed separately from the core processor of RSU.Two exemplary types of RSU location deployment type:i. Fixed location deployment. The location of this type of RSU are fixed, which is used for serving regular roadways with fixed traffic demand on the daily basis.ii. Mobile deployment. Mobile RSU can be moved and settled in new place and situation swiftly, is used to serve stochastic and unstable demand and special events, crashes, and others. When an event happens, those mobile RSU can be moved to the location and perform its functions.b. Method for coverage

The RSUs may be connected (e.g., wired) underground. RSUs are mounted on poles facing down so that they can work properly. The wings of poles are T-shaped. The roadway lanes that need CAVH functions are covered by sensing and communication devices of RSU. There are overlaps between coverage of RSUs to ensure the work and performance.c. Deployment Density

The density of deployment depends on the RSU type and requirement.

Usually, the minimum distance of two RSU depends on the RSU sensors with minimum covering range.d. Blind spot handlingThere may be blind sensing spots causing by vehicles blocking each other. The issue is common and especially serious when spacing between vehicles are close. A solution for this is to use the collaboration of different sensing technologies from both RSUs deployed on infrastructures and OBUs that are deployed on vehicles.This type of deployment is meant to improve traffic condition and control performance, under certain special conditions. Mobile RSU can be brought by agents to the deployment spot. In most cases, due to the temporary use of special RSUs, the poles for mounting are not always available. So, those RSU may be installed on temporary frames, buildings along the roads, or even overpasses that are location-appropriate.

Certain exemplary RSU configurations are shown inFIGS.19-22.FIG.19shows a sectional view of an exemplary RSU deployment.FIG.20shows an exemplary top view of an RSU deployment. In this road segment, sensing is covered by two types of RSU:901RSU A: camera groups, the most commonly used sensors for objects detection; and902RSU B: LIDAR groups, which makes3D representation of targets, providing higher accuracy. Cameras sensor group employ a range that is lower than LIDAR, e.g. in this particular case, below 150 m, so a spacing of 150 m along the roads for those camera groups. Other type of RSUs have less requirement on density (e.g., some of them like LIDAR or ultrasonic sensors involve distances that can be greater).

FIG.21shows an exemplary RSU lane management configuration for a freeway segment. The RSU sensing and communication covers each lane of the road segment to fulfill the lane management functions examples (showed in red arrows in FIG. including, but not limited to: 1) Lane changing from one lane to another; 2) Merging manipulations from an onramp; 3) Diverging manipulations from highway to offramp; 4) Weaving zone management to ensure safety; and 5) Revisable lane management.

FIG.22shows an exemplary lane management configuration for a typical urban intersection. The RSU sensing and communication covers each corner of the intersection to fulfill the lane management functions examples (showed in red in figure) including: 1) Lane changing from one lane to another; 2) Movement management (exclusive left turns in at this lane); 3) Lane closure management at this leg; and 4) Exclusive bicycle lane management.