You can refuse access, but doing so is often a bad idea: appraisers will almost invariably appraise high when denied access, to prevent people from denying access to reduce thrir taxes. You will almost certainly end up with a higher appraisal and tax burden if you deny access than if you allow it.