Title: Why do Python objects without circular references still get removed by garbage collection?
Tags: python,performance,garbage-collection,reference-counting

Question: If a function creates a lot of objects in a loop that are not referenced elsewhere, they will be removed immediately due to Python's reference counting.
If I store the objects in a list this time, the objects will be removed by garbage collection when the function exits. The objects do not have a circular reference. Why are they not removed by ReferenceCounting when the function ends and the list is removed?
As an example I have the following program with  two scenarios where I call the run function once with the parameter with_append = True  and the other time with with_append = False.
In the first scenario, the garbage collection jumps in and the whole programme takes much longer to finish. In the second scenario, there is no garbage collection active. You can see this by running the program with py-spy and using the native option.
Below is the example programme and the output for both cases.
import time

COUNT = 10000000


class User:
    def __init__(self, name):
        self.name = name


def run(with_append):
    l = []
    for i in range(COUNT):
        u = User(f"user {i}")
        if with_append:
            l.append(u)


ts = time.time()
run(with_append=True)
print("function completed - ", time.time() - ts)

Scenario 1
Call run with param: with_append = True
Output of: python demo.py
function completed -  10.940133094787598

Output of: py-spy top --native -- python demo.py
Total Samples 1000
GIL: 100.00%, Active: 100.00%, Threads: 1

  %Own   %Total  OwnTime  TotalTime  Function (filename)                                                                                                                                                  
 53.00%  53.00%    4.51s     4.51s   gc_collect_main (libpython3.10.so.1.0)
 38.00% 100.00%    4.41s    10.00s   run (demo3.py)
  9.00%  10.00%    1.07s     3.13s   __init__ (demo3.py)
  0.00%   0.00%   0.010s    0.010s   0x7f91a5b2a746 (libc-2.31.so)
  0.00% 100.00%   0.000s    10.00s   <module> (demo3.py)
  0.00%  53.00%   0.000s     4.51s   gc_collect_with_callback (libpython3.10.so.1.0)

Half of the time is spend in gc_collect_main.
Scenario 2
Call run with param: with_append = False
Output of: python demo.py
function completed -  4.351471424102783

Output of: py-spy top --native -- python demo.py
Total Samples 400
GIL: 100.00%, Active: 100.00%, Threads: 1

  %Own   %Total  OwnTime  TotalTime  Function (filename)                                                                                                                                                  
 85.00% 100.00%    3.38s     4.00s   run (demo3.py)
 13.00%  14.00%   0.560s    0.600s   __init__ (demo3.py)
  1.00%   1.00%   0.020s    0.020s   unicode_dealloc (libpython3.10.so.1.0)
  1.00%   1.00%   0.020s    0.020s   0x7fe05fca5742 (libc-2.31.so)
  0.00%   0.00%   0.010s    0.010s   0x7fe05fca5746 (libc-2.31.so)
  0.00%   0.00%   0.010s    0.010s   0x7fe05fca5724 (libc-2.31.so)
  0.00% 100.00%   0.000s     4.00s   <module> (demo3.py)

Why is reference counting not used to free memory in both cases? And why did garbage collection take so much more time to remove the objects?

Answer: Whenever you reassign u, the reference count for the old User object drops to zero, making it eligible for garbage collection, so before the loop ends it's able to collect the User object contained by u on the fly. So it waits until the end of the function to perform garbage collection on the list instead of doing it during the execution, because the reference count for the list does not reach zero until the list falls out of scope.