[GH #935] honor user-rlimits for returned sysmem_amount (untested)

We need to check for RLIMIT_DATA and then RLIMIT_AS,
first hard, then soft limits.