Actually enforce maximum download size. The old approach would only
enforce it when the content is known before-hand: for example by a
Content-Length header. If this was not the case, the crawler would
happily download hundreds of megabytes. A maximum size is now propagated
to the load() function in the Page class that reads up to max_bytes
bytes from the stream, replacing EntityUtils.toByteArray because that
does not supported partial reads. The result of this is reflected in the
"truncated" flag in the Page class. If bytes were available after
reading, Page.isTruncated() wil return true.