Cleanup registry when a container fails to get created.