Prefer direct buffer by default

- Because it's faster in most cases thanks to unsafe