The "Annotations" button will toggle annotations immediately for boards.
However, annotations do not show up until after the first rule is applied.