Don't build the gem every default rake run.
Support disabling audit for tests tagged inaccessible.

[Finishes #52922087]