Allow disabled physics in the control manager