Embedded Augmented Reality Training System for Dynamic Human-Robot Cooperation

In industrial production of high-wage countries, advanced automation technologies can partially compensate the lack of skilled workers, but human effectiveness and flexibility is still essential in many scenarios. Implementing the idea of mutual completion, direct human-robot cooperation appears suitable where strong forces are needed but human flexibility is indispensable. For this, the explicit spatial separation between robot and worker has to be given up. Prior to installation of sophisticated monitoring systems on the shop floor, advanced simulation methodologies have to be embedded directly into the production cell to design such cooperation scenarios safe and effective likewise. An immersive simulation system is presented which allows an optical see-through augmented reality (AR) configuration where the user is able to perceive the real tool in his hand. Alternatively, the system also supports a pure virtual reality (VR) mode where all objects’ visualization is artificial. Both variants accord in direct confrontation with a virtual robot and real-time physics simulation capabilities. A usability study with 40 subjects has been conducted, featuring robotically supported cast part blasting as experimental task. Results of user performance focussing on executing times and shooting accuracy indicate a tie between AR and VR and a surpassing overall usability in both configurations, but the users’ personal preference trends towards AR. 1.0 INTRODUCTION Due to demographic changes in high-wage countries, a significant lack of manufacturing specialists and skilled workers is foreseeable. Furthermore, constantly increasing pressure on costs, quality and timing combined with short product lifecycles and diversified product variants tightens selling conditions. Consequently, new manufacturing methods and appropriate simulation techniques are needed in order to strengthen competitiveness. Heavyweight goods handling in small-lot production is a good example where robots could ideally support human workers. This approach is particularly interesting for small and medium-sized enterprises, as demonstrated in the SMErobot initiative [1]. The sticking point is that the spatial separation between human and robot defined in international industrial norms like ISO 10218 [2] has to be given up. Therefore, besides safeguarding monitoring systems installed on the shop floor [3], new immersive simulation techniques are needed to minimize the risk of injury prior to start of production. Embedding the user directly into the virtual scene, advanced immersion could facilitate and accelerate safety assessments. Consequently, research efforts at the Institute of Industrial Engineering and Ergonomics at RWTH Aachen University feature advanced virtual and augmented reality technologies for more immediateness and realism. Report Documentation Page Form Approved OMB No. 0704-0188 Public reporting burden for the collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to a penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number. 1. REPORT DATE OCT 2009 2. REPORT TYPE N/A 3. DATES COVERED 4. TITLE AND SUBTITLE Embedded Augmented Reality Training System for Dynamic Human-Robot Cooperation 5a. CONTRACT NUMBER 5b. GRANT NUMBER 5c. PROGRAM ELEMENT NUMBER 6. AUTHOR(S) 5d. PROJECT NUMBER 5e. TASK NUMBER 5f. WORK UNIT NUMBER 7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) NATO 8. PERFORMING ORGANIZATION REPORT NUMBER 9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR’S ACRONYM(S) 11. SPONSOR/MONITOR’S REPORT NUMBER(S) 12. DISTRIBUTION/AVAILABILITY STATEMENT Approved for public release, distribution unlimited 13. SUPPLEMENTARY NOTES See also ADA562526. RTO-MP-HFM-169 Human Dimensions in Embedded Virtual Simulation (Les dimensions humaines dans la simulation virtuelle integree). , The original document contains color images. 14. ABSTRACT In industrial production of high-wage countries, advanced automation technologies can partially compensate the lack of skilled workers, but human effectiveness and flexibility is still essential in many scenarios. Implementing the idea of mutual completion, direct human-robot cooperation appears suitable where strong forces are needed but human flexibility is indispensable. For this, the explicit spatial separation between robot and worker has to be given up. Prior to installation of sophisticated monitoring systems on the shop floor, advanced simulation methodologies have to be embedded directly into the production cell to design such cooperation scenarios safe and effective likewise. An immersive simulation system is presented which allows an optical see-through augmented reality (AR) configuration where the user is able to perceive the real tool in his hand. Alternatively, the system also supports a pure virtual reality (VR) mode where all objects visualization is artificial. Both variants accord in direct confrontation with a virtual robot and real-time physics simulation capabilities. A usability study with 40 subjects has been conducted, featuring robotically supported cast part blasting as experimental task. Results of user performance focussing on executing times and shooting accuracy indicate a tie between AR and VR and a surpassing overall usability in both configurations, but the users personal preference trends towards AR. 15. SUBJECT TERMS 16. SECURITY CLASSIFICATION OF: 17. LIMITATION OF ABSTRACT SAR 18. NUMBER OF PAGES 14 19a. NAME OF RESPONSIBLE PERSON a. REPORT unclassified b. ABSTRACT unclassified c. THIS PAGE unclassified Standard Form 298 (Rev. 8-98) Prescribed by ANSI Std Z39-18 Embedded Augmented Reality Training System for Dynamic Human-Robot Cooperation 6 2 RTO-MP-HFM-169 2.0 VIRTUAL TECHNOLOGIES IN PROCESS SIMULATION AND ROBOTICS Desktop-PC-based 3D simulation environments are state of the art nowadays and cover most scenarios for industrial robotics in various use cases: from heavyweight goods handling to spot welding and spray painting, robots, fixtures and most equipment can be modelled and simulated [4]. This allows building up complete production lines including all challenges which come about like power-up phases, mutual interlocks, dynamic material allocation etc. Nevertheless, human workers in general and highly skilled workers in particular are still rarely taken into account. Representation with digital human models offers advanced analysis capabilities in terms of proportions, stress analysis, field of view etc. [5], but due to their high number of degrees of freedom, digital human models are cumbersome to handle, especially in interactive real-time scenarios. AR and VR allow direct (egocentric) confrontation of the user with the virtual objects. Humanoid robot interaction is a well-know area of application [6]. Robot manufacturers like ABB and KUKA [7] as well as third party researchers [8] have already caught up this track to support and to facilitate robot programming. In this context, an ongoing German research project with participation from industry as well as research labs called AVILUS focuses on further improvement of virtual and augmented reality technologies in product development and service [9]. Still, support for direct human-robot cooperation in terms of manufacturing is rarely featured. For AR, which is generally in regard to technology more challenging than VR for convincing results, desktop monitors with live video stream or images are most often used. Optical see-through (OST) headmounted displays (HMDs) still lack in usability and ergonomics because of their size, weight, resolution, and the hard-to-realize occlusion of real behind virtual objects. Nevertheless, they offer deep immersion and require less space and financial effort in comparison to more elaborate alternatives like CAVE systems, for example. Current developments concentrate on advanced visual combination of virtual and real objects with addressable focal planes [10], for example. Accurate and easy-to-use calibration routines for OST HMDs remains a challenging task; established methods are based on matching of virtual over real objects [11], newer approaches use cameras looking directly through the HMD optics to exploit both the intrinsic and extrinsic parameters [12]. 3.0 INDUSTRIAL USE CASE Industrial casting of massive metallic parts like crank cases is accompanied by undesirable disposition of sand relics. Cleaning is usually done through abrasive blasting with water, dissolvers or carbon dioxide pellets. The last-mentioned alternative is most recommendable since carbon dioxide is electrically insulating, chemically inert, nontoxic and inflammable [13]. Well-directed laboring is highly advisable for surface protection and economic pellet exhaustion. Typically, the pellets are shot salvo-wise with a specialized high-pressure pistol. In mass production, the cast parts are handled by highly-automated conveyor systems for fast processing. In small-lot production, however, operators depend on mobile handling devices like cranes and jack-up platforms. Frequent usage of these is cumbersome and dangerous through perpetual hooking and unhooking, clamping and unclamping etc. Direct human-robot cooperation can bring about a significant advantage through the idea of mutual completion: here, the robot could indefatigabl