Allow skipping backref validation, fixes #465