Restrict program memory with ptrace.

It intercepts brk/mmap/mmap2 calls to get heap size and return -ENOMEM in
syscalls when threshold exceeded.