Decrease dependence of SecondaryToolbar on UI buttons

Implements the proposal in #3818.