[Edit: quietbritishjim was faster than me, and makes more or less the same points. You can safely ignore this post.]

First let's have a look at this:

>>> [ x() for x in [lambda: i for i in range(5)] ]
[4, 4, 4, 4, 4]

Variable i in the function body is a nonlocal or global, taken from the nearest enclosing scope where i is bound. When a function (lambdas are functions just like any other) is executed, a nonlocal or global variable gets the value it has in its scope at the time of the function call. Step by step:

>>> import inspect

>>> i = 0
>>> a = lambda: i
>>> inspect.getclosurevars(a)
ClosureVars(nonlocals={}, globals={'i': 0}, builtins={}, unbound=set())

We see that variable i in a refers to the global i, which currently has value 0. Obviously a() returns 0:

>>> a()
0

Next we create a new lambda, with the same definition as before, but we first increment i.

>>> i = 1
>>> b = lambda: i
>>> inspect.getclosurevars(b)
ClosureVars(nonlocals={}, globals={'i': 1}, builtins={}, unbound=set())

So i in b refers to the global i (as before), which now has value 1. Unsurprisingly b() returns 1:

>>> b()
1

But variable i in a also refers to the global i which is now equal to 1:

>>> inspect.getclosurevars(a)
ClosureVars(nonlocals={}, globals={'i': 1}, builtins={}, unbound=set())

And sure enough, a() now returns 1:

>>> a()
1

We can set i to any value, and both a() and b() will then return that value.

>>> i = 'gotcha'
>>> a()
'gotcha'
>>> b()
'gotcha'

Going back to your example, the i used in the lambdas refers to a nonlocal instead of a global (the i used in the list comprehension), but that doesn't make a difference for the topic at hand. What does matter is that the current value of i is used, which is 4 after the for loop in the list comprehension.

Now to the second example.

>>> i = 0
>>> a = lambda i=i: i

i is now a local, since it's a parameter to the function. It's a bit unclear because i is used in two different meanings here; let's rename one of the uses to j. This is exactly equivalent:

>>> a = lambda j=i: j
>>> inspect.getclosurevars(a)
ClosureVars(nonlocals={}, globals={}, builtins={}, unbound=set())

This time there are no closure variables. j in the function body does not anymore refer to anything outside of the function scope. Instead a is now a function with one parameter, which returns the value of that parameter:

>>> a(10)
10
>>> a('hello')
'hello'

What's of course more important here is that we have a default value for that parameter. Default parameter values in Python are evaluated when the function is defined, not when it is executed. i was 0 when we defined a, so a will always have 0 as default parameter. We can see that default value:

>>> a.__defaults__
(0,)

Add a second function:

>>> i = 1
>>> b = lambda i=i: i
>>> b.__defaults__
(1,)

Now b is also a function taking one parameter, but this time that parameter has default value 1. Meanwhile the default value of function a has not changed:

>>> a.__defaults__
(0,)

So now we can predict that a() will return its default parameter value (which is 0), and b() will return its default parameter (which is 1). Let's check.

>>> a()
0
>>> b()
1

So the big difference is that your first example uses nonlocal/global variables which are evaluated when the function is called, while your second example uses default parameter values which are evaluated when the function is defined.

Note: you can change default parameter values though, if you really really want:

>>> a = lambda i=5: i
>>> a()
5
>>> a.__defaults__ = ("don't actually do this",)
>>> a()
"don't actually do this"