After changing the way annotations are fetched, properly clean up
and implement proper ACL.