Don't close the SPDY peer too early.

This was occasionally causing tests to fail because the test was
observing the connection close.