Util.set_memory_limit, with a C stub to call setrlimit