minor, don't report illegal characters on the buffer boundary

Signed-off-by: Sebastian Freundt <freundt@ga-group.nl>
fix, PEBKAC, use correct boundaries for unicode ranges in gencls(1)

Signed-off-by: Sebastian Freundt <freundt@ga-group.nl>
fix, PEBKAC, offset 3-byte utf8 sequences properly

Signed-off-by: Sebastian Freundt <freundt@ga-group.nl>
test, adapt regression tests to fixed terms binary

Signed-off-by: Sebastian Freundt <freundt@ga-group.nl>
hygiene, use read() instead of (buffered) fread()

Signed-off-by: Sebastian Freundt <freundt@ga-group.nl>
minor, always return the number of processed bytes

Signed-off-by: Sebastian Freundt <freundt@ga-group.nl>
minor, rewrite co_snarf() coru to evaluate the number of processed bytes

Signed-off-by: Sebastian Freundt <freundt@ga-group.nl>
minor, fly over utf8 4-byte sequences (with an error message)

Signed-off-by: Sebastian Freundt <freundt@ga-group.nl>
minor, only reset buffer offset in co_snarf() when stuff has been processed

Signed-off-by: Sebastian Freundt <freundt@ga-group.nl>
minor, leave the top 4 bytes in global buffer free to allow reading beyond the end of buffer

which saves us some extra checks in the utf8 multi-byte collectors.

Signed-off-by: Sebastian Freundt <freundt@ga-group.nl>
chore, assert that we can only process what we've got

Signed-off-by: Sebastian Freundt <freundt@ga-group.nl>
Revert "minor, leave the top 4 bytes in global buffer free to allow reading beyond the end of buffer"

This reverts commit 9269dec9caf791462bb6cc2565cc8a745a61e88b.

It's not safe.
minor, check before reading beyond the end-of-buffer

Signed-off-by: Sebastian Freundt <freundt@ga-group.nl>
Merge branch 'tool/terms'

* tool/terms:
  minor, check before reading beyond the end-of-buffer
  Revert "minor, leave the top 4 bytes in global buffer free to allow reading beyond the end of buffer"
  chore, assert that we can only process what we've got
  minor, leave the top 4 bytes in global buffer free to allow reading beyond the end of buffer
  minor, only reset buffer offset in co_snarf() when stuff has been processed
  minor, fly over utf8 4-byte sequences (with an error message)
  minor, rewrite co_snarf() coru to evaluate the number of processed bytes
  minor, always return the number of processed bytes
  hygiene, use read() instead of (buffered) fread()
  test, adapt regression tests to fixed terms binary
  fix, PEBKAC, offset 3-byte utf8 sequences properly
  fix, PEBKAC, use correct boundaries for unicode ranges in gencls(1)
  minor, don't report illegal characters on the buffer boundary