Have recv fail if we have a bogus client address

Signed-off-by: Adam C. Emerson <aemerson@linuxbox.com>