Turns out, removing the comment post type support disables comments altogether for new posts. Oops!