Fix NPE/AIOOBE when building a bucket which has not been collected.

Close #5048