runtime: improve mmap return value checking for netbsd/openbsd

Rather than just checking for ENOMEM, check for a return value of less
than 4096, so that we catch other errors such as EACCES and EINVAL.

R=golang-dev, rsc
CC=golang-dev
https://codereview.appspot.com/7942043
runtime: correct return value checks for mmap on darwin/freebsd

On Darwin and FreeBSD, the mmap syscall return value is returned
unmodified. This means that the return value will either be a
valid address or a positive error number.

Also check return value from mmap in SysReserve - the callers of
SysReserve expect nil to be returned if the allocation failed.

R=golang-dev, rsc
CC=golang-dev
https://codereview.appspot.com/7871043