Rename noop option to dry_run

Also, only allow using it in static mode.