Fix memory fd handling and leaks