MAINT: Fix missing decrefs and resolve circular ref in test