Block new HTTP requests while mutating the config