Only write non-empty annotations