combining user cues with robot control