Supporting custom annotation readers

Implementation for #372
I don't think this needs a qualifier.