Decouple material object loading/deletion from material system setup

This also fixes a crash on loading a map passed on the command line.