[lib] Use setrlimit() instead of prlimit().

Use setrlimit() to disable coredumps for compatibility reasons.