WIP
Build the annotations dictionary but not set it yet.