Modify internal is_swappable trait to be a bit cleaner (still does not work with g++)

Remove some false warnings from clang in various tests.

Also, please note that is_swappable is basically broken for G++. It
works under clang, but under it is *always* true_type. This is something
I cannot fix.