# Semantic Task Planning for Service Robots in Open Worlds
## 5 Planning, Execution and Monitor
### Execution and Monitor  
The components described in the previous sections are employed by the Plan Execution and Monitor component, which is the central coordination component for the command execution. The plan generation and execution follow the traditional \"planning-execution-monitor-replan\" loop. A simplified control flow for the execution of a planning task is shown in Figure 3.  
Figure 3: Plan generation, execution, and monitor. When a command is sent from a person, the NLP module will generate the corresponding _action frames_ and _modifiers_, and then construct the goal conditions. Before planning, parameters in action frames will be confirmed. Combined with the current world state, if the specific entity or location information cannot be confirmed, the HRI module will generate a query command to confirm this information and rewrite the goal and state based on the confirmed information. Then, domain knowledge, states, and goals are transmitted to the planner, then the ASP solver generates a plan. If the plan cannot be found, the planner generates feedback indicating the failure. A successful plan includes a series of actions bound variables and the preconditions and effects of these operations, which will be executed one by one. We use CLINGO [42] as our ASP solver. Each operation corresponding to the symbol planning operator is connected with a state machine that controls the robot's execution. These operations combine elementary or primitive skills into complex skills. The primitive skills are based on services provided by the robot such as inverse kinematics, motion planning, object recognition and location, face recognition, and so forth. For example, motion grasping uses a visual servo to perform object localization accurately. Action execution may fail due to the uncertainty of perception or the change of environment. To illustrate these changes, the preconditions and effects are verified by the _monitor_. The monitoring program detects changes in the environment and the results of action execution and decides how to make decisions. There are two kinds of environment changes: one is useful information for the current task, and the other is ineffective. Both will be imported into the knowledge database. We divide the useful information into two kinds: the disappearance of preconditions of the actions in the plan; the other is the emergence of new objects or states that meet the task's requirements. In our system, both of them are all seen as conflicts. The former represents that the current plan will not complete the task, while the latter indicates that there may be a better solution to complete the task. They will make the robot re-plan and find a new solution. If new information shows there are no required objects in the knowledge base, the robot will make the new assumptions(assume some required object is somewhere), the same location is only allowed one assumption in a task. In previous work [11], any environment change will trigger a re-plan. We adjust to re-plan when the above two changes occur. The algorithm in [1] re-plans when the unexpected result is detected after the action but ignores subsequent actions (some changes may satisfy current action, but make the conditions of some subsequent actions unsatisfied). Our algorithm's advantage is that it detects whether the environment changes conflict with the current action and detects whether there is a conflict with subsequent actions. As shown in Algorithm 2, according to the action parameters (\(a\)) and the action model (\(D\)), it instantiates the preconditions (\(P\)) of the action, then detects whether these preconditions conflict with environmental changes (line 5-7). In our system, a conflict means that any two states cannot exist simultaneously. These are pre-defined, for example, an object cannot exist in two places simultaneously, and the hand cannot be opened and closed. If the action is successful, the next action will be performed.