# Unified Software Platform for Intelligent Home Service Robots
## 1 Introduction  
One of the main goals of robotic research is the commercial adoption of intelligent service robots--robots that can safely operate in various human environments without special infrastructure; communicate with untrained people using natural language and gestures; and handle various tasks that can help human lives. Thanks to the recent advancements of smartphones and motorized personal transportation devices, we have gained access to affordable processors, sensors and actuators, which has significantly lowered the hardware costs of intelligent mobile robots. In addition, the recent breakthroughs in the machine learning field have allowed the robots to robustly perceive the environment and communicate with people using natural language. As a result, we now have an increasing number of commercially available autonomous service robots that aim for home or enterprise markets [1; 2; 3; 4; 5].  
However, mainly due to the relative high cost and weight of the robotic arm, most of the current service robots either lack an arm [1; 2] or have less capable arms, mainly for gestures [3; 4]. This omission greatly limits the uses of the service robot, as the robot cannot physically interact with the environment by itself and needs continuous human support during its operation. Although there are mobile robots equipped with high degree of freedom (DOF) manipulators with high payload and accuracy [6; 7], they are not ideal for tight indoor environments, as their long, heavy and rigid manipulators increase the system size and weight significantly, require complex motion planning and pose possible safety risks.  
As a result, there came a new generation of home service robots designed with the goal of autonomous mobile manipulation in home environments. Toyota's Human Support Robot [5] is a relatively small and lightweight home service robot with a compliant low-DOF manipulator thatsimplifies the motion planning and lowers safety risks. The HSR has been adopted as a standard platform for two major international robotic competitions, the RoboCup@Home Domestic Standard Platform League (DSPL) and the World Robot Summit (WRS). By 2018, 33 universities in eight countries were using the HSR, which was on offer to the public since 2017 [8]. Stretch RE1 [9] is another commercial home service robot with a very small footprint and a compliant Cartesian manipulator that can handle various home service tasks in a tight home environment.  
In this paper, we present a unified, modular software framework that can be quickly adopted for different intelligent mobile manipulation robots. It consists of navigation, visual perception, manipulation, human-robot interaction and high level autonomy modules, and can handle general service tasks given by complex voice commands. The remainder of the paper proceeds as follows. Section 2 presents details of two robotic platforms, which are shown in Figure 1. we have used to test the software framework. Section 3 explains how the robot detects household objects, furniture and humans from sensory inputs. Section 4 describes motion generation and control methods for object pickup and furniture operation. Section 5 presents how the robot governs its autonomous behavior and communicates with human operators. Section 6 shows the experimental results from two international service robot competitions. Finally, we conclude with a discussion of potential future directions arising from this work.