Don't allow getutxo answer to go over the size of the send buffer.

Limit the possible answer size for getutxo.

This does not adjust MAX_INV_SZ because it's unnecessary for the mix, additionally, requesting normal outputs even up to MAX_INV_SZ does not run out of space in the send buffer. To make a giant response you need a giant output. So there might be some use case for doing a big query, at any rate, if the send buffer is too big then that's a general issue we should fix elsewhere.

In the end I didn't truncate the answer. I just return false which hits Misbehaving(). The reason is, again, no real app will ever hit this case. You need to deliberately pick a giant output and a giant number of repeated queries. So Misbehaving is a reasonable solution.
How so? Like I said, requesting 50,000 outputs of a normal output would not even trigger problems. Requesting 50,000 outputs of the biggest output on the testnet doesn't even kill my node, it just bloats up and then shrinks a few seconds later. dhill had to run many times in parallel to get the effect of him running out of RAM.

So I don't see why shrinking the send buffer would cause problems for any normal app, not even if we halved it.