> First, it is not equivalent - next() cannot apply to range() output, for example - you will need to do some iter() games and watch out for iteration order side effects if your values are iterators vs. lists.

It uses generator/iteration semantics instead of list semantics. If you wrap the whole thing with a decorator like function that does:

def scan_wrapper(f, x0, x):
return list(scan(f, iter(x), x0)

You get the exact same semantics. For most cases (including the one you cited), the alternate semantics are actually better, more flexible, and avoid requiring a list to be built in the first place.

No problem with iteration order side effects either unless your f() somehow invalidates your iterable... and you still have some potential exposure there in your original implementation.

> Second, it is ~10% faster...
> It potentially uses less memory...

Yeah, I think you are understating it to say the least. Not only are you using less memory, but you are saving having to rejuggle/resize the list all the time.

> (unless Python 3 gained TCO when I wasn't looking. Did it?)

I guess in a way it sort of did for the case of yield from: 'The iterator is run to exhaustion, during which time it yields and receives values directly to or from the caller of the generator containing the yield from expression (the "delegating generator").'

So, even without full on TCO (which is still possible... I'm not sure if they did it with yield from), you at least have direct pass through from the generator to the caller. Because of iterator semantics, that should mean that each of the generators gets created on an as needed basis and the previous generator should get destroyed right thereafter. It is possible though that it isn't quite doing it right, in which case I'll concede that I'm still allocating an N deep generator stack, but that is still likely to be more memory efficient because it isn't having to reallocate/resize/copy increasingly larger lists throughout the execution.

> recursive solutions without TCO are rarely good enough to replace iteration.

As I mentioned, you can do the recursive solution as well, and it has the advantage of working with old Python. Still simpler and still far more efficient (here it is with extra wrapping to keep the semantics the same):

def scan(f, x0, x):
def scan_helper():
yield x0
for x1 in x:
x0 = f(x0, x)
yield x0
return list(scan_helper())

> It is mostly space-efficient in general.

You say that like when doing statistical analysis space-efficiency isn't a concern...

> I think it is more idiomatic, though - and also Python2 compatible - to just replace references to 'r' with yield in my code, than using the recursive definition you gave above - which is more idiomatic in functional languages, but less in Python (and harder to debug in any language than the iterative version)

I was actually mostly getting at using yield instead of list append. I was just trying to express it as tersely as possible, which unsurprisingly became Python 3 and a functional style mechanism.

While I agree that often there is a struggle to understand functional programming, I think in this case it is very idiomatic Python (particularly since they defined "yield from" specifically for cases like this), and the code is very simple, readable, and easier to verify for correctness.