Simplify code at a few places

Remove some duplicate checks,
and unnecessary errors.
Because we are going to anchor the loaded annotations, and for that, we need the mapping data. (And we might not have scanned before, since we have options in place to skip the initial scanning.)