WIP: Fix refcounting problems.

See comments:
https://github.com/numpy/numpy/pull/3524#discussion-diff-5671274
https://github.com/numpy/numpy/pull/3524#discussion-diff-5671342
https://github.com/numpy/numpy/pull/3524#discussion-diff-5671399