Documentation states clearly what   ```~=``` does, and being an operator that is never used with integers or enums, I don't think it behaves surprisingly, simply because of the fact that if you don't know what it does, you won't use it.
If you see it in use, it takes a moment to check the ddoc and see what it does.
```-=``` would be much more suprising in this sense, as it does not conform to what it does on enums.
I believe there should be a method to unset a flag; an alternative would be to use a named method ```auto ref unset(E flag)``` but it really doesn't fit with the native feel of Flags.
```&= ~```, besides involving two operators which is already quite ugly, is doubly ugly in the version that takes an enum as a parameter, as it forces you (instead of only allowing it) to rely on the very unsafety of enums that Flags is meant to supersede.