Allow Ctrl-click as well as middle-click to assign/deassign all