it's admittedly rare that a SetDeadline would fail, but nonetheless this defer should happen right after we're sure we've established a connection, before we set and check the deadline (or we could orphan a connection)