# Semantic Task Planning for Service Robots in Open Worlds
## 6 Simulations and Results  
To evaluate the system, we have developed a simulation environment with GAZEBO [[http://gazebosim.org](http://gazebosim.org)]([http://gazebosim.org](http://gazebosim.org)) (accessed on 12 February 2021). As shown in Figure 4, the environment contains a domestic house and the model of Kejia. As shown in Figure 5, Kejia is equipped with a wheeled mobile base, a single 5-Degree-of-Freedom arm, a 2D laser range finder, an elevator, a pan-tilt, and an RGBD camera. Our system is implemented as nodes in ROS (Robot Operating System). It subscribes all necessary information (arm joint angles, robot pose, recognized objects, etc.) and publishes control messages (navigate to some location, turn on object recognition function to search some object, etc. ), which affect the behavior of the robot. The domestic house contains several rooms (bedroom, living room, kitchen, etc. ), furniture (bed, table, desk, cupboard, bookshelf, etc.) and some objects (cup, bottle, beer, coke, etc.). For all objects, there are two categories: specific and similar. The specific object has a unique label in the perception system, and this object is bound to this label. Similar objects (apples, same bowl, etc.) share a vision system label. For similar objects, their names in the knowledge base are different. The mapping between the vision system and knowledge base depends on location, size, and other properties. Figure 4: The domestic environment, which includes two bedrooms, one kitchen, one living room, and one study. In the experiment, we mainly consider three kinds of uncertainties: HRI uncertainties (vague or erroneous information from users), changing environment (object removed from a known position or object appears in an unexpected place), and execution errors (failed grab or navigation). The required objects and their positions may be unknown to the robot, so the robot needs to assume that there are objects needed in the place given by the user. Here are five scenarios to show how robots respond to these challenges. (A demo video is available at [[https://youtu.be/y4](https://youtu.be/y4) hJ1 OODIGO]([https://youtu.be/y4](https://youtu.be/y4) hJ1 OODIGO) (accessed on 12 February 2021).) **Scenario 1 (vague information): [PERSON] requested a coke from the kitchen. ** Initially, [PERSON] and the robot were in the study, the doors from the study to the kitchen are unlocked, the robot knew nothing about coke. The robot got a command _\"bring a coke from the kitchen for me\"_, then dependencies _root(ROOT-0, bring-1)_, _dobj(bring-1, coke-3)_, _nmod:from(bring-1, kitchen-6)_ and _nmod:for(kitchen-6, me-8)_ were generated, they assigned the _Action_, _Object_, _Source_ and _Goal_ of the action frame _action(robot, bring, coke, kitchen, jamie)_, where _\"me\"_ was replaced by the user'name. The robot got vague information: in the kitchen. The robot _first assumed_ there was coke in the kitchen table, then started searching in the kitchen by visiting the kitchen table first. There was one cup, one bottle, and one bowl on the kitchen table, but no coke. The robot made the _second assumption_: the coke was in the cupboard. It visited the cupboard and found two cookes and one beer, then bring one coke to [PERSON]. **Scenario 2 (error information): [PERSON] requested a cup from the living table. ** The robot got a command _\"get a cup from the living table for me\"_, then dependencies _root(ROOT-0, get-1)_, _dobj(get-1, cup-3)_, _nmod:from(get-1, table-7)_ and _nmod:for(table-7, me-9)_ were generated, they assigned the _Action_, _Object_, _Source_ and _Goal_ of the action frame _action(robot, get, cup, living_table, jamie)_, where _\"me\"_ was replaced by the user'name and _living_table_ generated based on a modifier _amod(table-7, living-6)_. [PERSON] and the robot were in the study, and [PERSON] asked the robot to go to the living table and brought a cup. Though the robot knew a cup on the kitchen table, it first tried to get a cup from the living table.