Fix out-of-bounds read in COFF and P9 file recognition

(In check_bytes, that is.)
r_bin_reload: initialize len_bytes, avoid unpredictable allocation

The problem here is that r_io_desc_read checks out_sz, the argument
&len_bytes is passed as, and uses *out_sz as the number of bytes to
allocate unless its UT64_MAX.  So, instead of passing an uninitialized
value, initialized len_bytes to UT64_MAX.