Conditional freeing based on reference count