[Startup] Rework startup a bit to put more of our config into git