Fixed some more signed-unsigned-comparison warnings.
plugins: checksum: Use glib for filesize and reading bytes.

- Fixes support for large files.
- Fixes filenames not being passed in the filename encoding.
- Adds missing checks for errors from openssl's SHA functions.