[WIP] mbyte/fileio: general cleanup, try to simplify encoding detection + conversion

Not sure what will be the end result of this PR, but I needed a place to hack and perhaps others can chime in with their thoughts. For now it's just general cleanup work to hopefully make the code less inscrutable. This way it'll be easier to see whether future changes are correct and don't implode the quantum universe that is encodings.

Plans:

- cleanup encoding support (`mbyte.c`)
- support iconv again (in PR #1370, but the same commit is also present here)
- split up `readfile()`, this will be quite tough and probably also difficult to test
  - base the encoding detection more on functions in `mbyte.c` instead of duplicating a lot of work inside of `readfile()`
  - ???

Ping @splinterofchaos who has shown interest in some of this and has done some serious work on cleaning up `readfile()`.