The FAA does not explicitly say that you have to be holding your controller while your drone is in the air, however, if anything goes wrong up there, you could face fines and be charged with reckless endangerment. Think of a car on the road, in many states it is not illegal to remove your hands from the steering wheel, but you are still 100% responsible for what the car does on the road.