Make some code more idiomatic

Nice point on bswap! I also started to think some `unsafe` is acceptable if it is for speed and it has safe fallback. (I checked if llvm optimizes safe routine well, but it doesn't even with `--opt-level=3`.)