ADVANCED PERCEPTION SYSTEM FOR VEHICLE AUTONOMY

An approach for a perception system for autonomous vehicle navigation is presented. The approach relies on low-cost electro-optical (EO) sensors for terrain classification, 3D environment modeling, and object/obstacle recognition. Stereo vision is used to generate real-time range maps which are populated into a hybrid probabilistic environment model. Textural and spectral cues are utilized for terrain classification and spatial contextual knowledge is proposed to augment object recognition performance. INTRODUCTION Current sensor suites demonstrated on autonomous vehicles are prohibitively expensive for mass deployment. In addition, current sensors (specifically LIDAR) provide obstacle detection, but lack robust object classification or perception capabilities sufficient for advanced decision making. An autonomous vehicle needs the ability to determine, for example, if an obstacle can be driven over/through, as in the case of a tall grass, brush, or a wooden barricade, or if it needs to circumnavigate the obstacle, as in the case of a concrete or rock barrier. Autonomous vehicles require enhanced sensing capability to fulfill current and future mission scenarios. Dismounted troops have virtually unlimited mobility. An autonomous support vehicle will therefore need to traverse any terrain within the operational envelope of the vehicle. This could include off-road terrain, grades, rough surfaces (rocky), vegetated (overgrown) paths, dirt roads, narrow paths, and stream/drainage (wet/dry) crossings. The Advanced Perception System (APS) is being developed to demonstrate enhanced perception capabilities utilizing low-cost sensors (electro-optical cameras). The program seeks to affirm the hypothesis that advanced analysis algorithms can be applied to data from economical sensors to achieve advanced perception capability. The initial program goal is to demonstrate obstacle/object/terrain classification providing data for use by path planning algorithms that is superior to that provided by current LIDAR systems. The sensor and data processing approach will parallel the human ability to achieve a high level of perception from only visual data. Specifically, the proposed solution will integrate: Texture analysis of Electro-optical (EO) imagery for object/region segmentation and classification Multispectral image analysis for object/region segmentation and classification Range estimation from EO imagery for region segmentation, object size and pose estimation Tiered classifiers coupled with context based reasoning for obstacle/object perception System performance will be evaluated and demonstrated in off-road and low-use dirt road environments. The program is sponsored by the Office of Naval Research, Code 30 in support of Marine Corps Enhanced Company Operations and the Small Unit Mobility Enhancement Technology (SUMET) Program. This paper describes the high-level system architectures and preliminary results. PROGRAM OBJECTIVES The APS development is part of a multi-year program to evaluate unmanned system technologies for small and distributed units. The APS is envisioned to provide sensory input for higher level path planning and cognitive reasoning systems. Several system-level objectives have been identified: Terrain Classification The APS should provide offroad terrain classification sufficient for traversability Proceedings of the 2010 Ground Vehicle Systems Engineering and Technology Symposium (GVSETS) Advanced Perception System for Vehicle Autonomy, Flannigan, et al. Page 2 of 6 analysis and path planning activities for varied mission plans and risk states. Object Classification The APS will require object classification for objects of interest, such as pedestrians and vehicles, that are important to high level behavior generation. Modularity The APS is platform agnostic and should provide interfaces that enable easy adaptation to varied hardware systems and missions. Low Cost A hardware (sensors, computational resources, communications) replication cost target of less than $30,000 has been identified to enable high volume application of the technology. SYSTEM ARCHITECTURE The APS hardware is focused on commercial-off-the-shelf (COTS) EO sensors in the visible and near-infrared spectrum. Future efforts will integrate other vehicle sensors such as GPS/IMU, wheel speed sensors etc. Current development is being conducted on Southwest Research Institute’s (SwRI’s) Mobile Autonomous Technology Research Initiative (MARTI) platform [1], shown in Figure 1, and will be transitioned to a militarily relevant vehicle for future demonstration milestones. A pair of color, 1.4 Megapixel, Gigabit Ethernet cameras is mounted to the vehicle roofline. These imagers are used for stereo disparity analysis for range calculation and for textural cues. A second multispectral camera is employed for color classification. Keeping with the low-cost COTS goals, the multispectral camera is composed of several (up to six) 0.36 Megapixel monochromatic imagers with individual optical filters, mounted in a compact assembly. Figure 2 shows a high-level conceptual architecture for the APS. Figure 1: SwRI's MARTI Autonomous Vehicle The software architecture makes use of the Robot Operating System (ROS) which is actively developed and maintained by Willow Garage [2]. ROS is an open-source framework for inter-process communication for robotic and sensor systems. ROS provides an architecture that fosters modularity and distributed processing which will be important as the APS becomes more sophisticated and is interfaced with other autonomy and hardware modules. The APS is executed on general purpose CPUs either resident within the MARTI vehicle or on portable systems. Future efforts will optimize the computational architecture for size, weight, and power consumption. Real-time performance for control of the autonomous platform is not yet required, and computational performance and future requirements are being characterized in the initial development phase.