Only require password when required