Only run compatibility check once per server