Properly clean up destroyed record arrays

This build upon #2361 which cleaned up filtered record arrays, but did not clean up
record arrays coming from .all and .find