Big refactoring to convert variant annotations to Annotation objects, which can be either strings or doubles