# Singularity analysis via the iterated kernel method

Stephen Melczer and Marni Mishna†

Department of Mathematics, Simon Fraser University, Burnaby, Canada

**Abstract** We provide exact and asymptotic counting formulas for five singular lattice path models in the quarter plane. Furthermore, we prove that these models have a non D-finite generating function.

**Résumé** Nous présentons des résultats énumératives pour les cinq modèles de marches dans le quart de plan dites “singulière”. Nous prouvons que ces modèles sont non-holonome.

**Keywords:** Lattice path enumeration, D-finite, generating function, singularities

## 1 Introduction

The study of lattice path models restricted to the quarter plane has led to some useful innovations in enumeration, including applications of boundary value methods [8, 9, 12], powerful and widely applicable variants of the kernel method [5, 7, 14] and some original computer algebra approaches [3]. In addition to purely enumerative results, there are also results on asymptotic enumeration and the nature of the generating function: either rational, algebraic, transcendental D-finite<sup>(i)</sup>, or other.

A key observation of Bousquet-Mélou and Mishna was that lattice path models with small steps restricted to the quarter plane appeared to be naturally partitioned according to the nature of their generating functions: specifically, they conjectured a test for whether or not the generating function of a given model would satisfy algebraic or linear differential equations. This property is often correlated to other, more combinatorial, qualities of a model. Of the 79 non-isomorphic models, 23 are well studied with D-finite generating functions, 51 are highly suspected to be non D-finite – Bostan et al. proved the excursion (walks returning to the origin) generating functions are not D-finite [4], and Kurkova and Raschel proved that the trivariate generating functions marking endpoint [12] are not D-finite. The remaining 5 models are called singular, and resist both these strategies. Two of these models had been previously considered [14],

<footnote>†This work was supported by NSERC via a Discovery Grant, a USRA fellowship, and a Michael Smith Foreign Study Supplement. SM was also partially supported by funding from the Office for Science and Technology at the Embassy of France in Canada. MM is grateful to LaBRI (U. Bordeaux) that housed her during the writing of this work.</footnote>

<footnote>‡Email: srm9@sfu.ca, mmishna@sfu.ca</footnote>

<footnote>(i) A univariate series is D-finite if it satisfies a linear differential equation with polynomial coefficients. Additionally, a series is D-finite if and only if its analytic continuation is D-finite.</footnote>
where their univariate generating functions were proven to be non D-finite. We apply this strategy, a relatively direct application of the iterated kernel method, a technique modelled on the work of Janse van Rensburg et al. in [10], and Bousquet-Mélou and Petkovsec [6] to the final 3 models.

Specifically, the present work proves that the remaining cases are not D-finite (and we note a small correction to the cases already proven). From the method, it is straightforward to determine explicit generating function expressions and asymptotic counting information, in the process identifying many potential singularities for each generating function. In each case, we show that the number of singularities is infinite, and far enough away from the dominating pole that they do not affect the first order asymptotics. The key difficulty here, as was the case in [14], is the justification that these singularities are true poles and are not somehow canceled by some quirk of the expression. This is significant because a D-finite function has a finite number of singularities, and so such a demonstration is a proof of non D-finiteness of the generating function. In the course of our proofs we revisit some older theorems on polynomials that a reader faced with a similar problem may find useful.

In summary, for each of the five singular models we take a unified approach to prove formulas for asymptotic enumeration and determine an explicit expression for the generating function, information which cannot be determined using other known methods. In addition, we prove that the (univariate) counting generating functions are not D-finite. This extended abstract presents the main results, and we we refer the reader to [13] for a completed manuscript.

## 1.1 The family of singular models

A lattice path model is defined by a set of vectors – the allowable directions in which one can move along the lattice of non-negative integers. We are particularly interested in models which permit only “small” steps, that is, the steps are contained in $\{0, +1, -1\}^2$. We use the notation $NW \equiv (-1, 1)$, $N \equiv (0, 1)$, $NE \equiv (1, 1)$, etc. The family of singular models consists of the following five models, each given with two different representations:

$$A = \text{\textup{\textbf{\Large $\overrightarrow{\rightleftharpoons}$}}\text{}} = \{\text{NW}, \text{NE}, \text{SE}\}$$

$$B = \text{\textup{\textbf{\Large $\overrightarrow{\rightleftharpoons}$}}\text{}} = \{\text{NW}, N, E, SE\}$$

$$C = \text{\textup{\textbf{\Large $\overrightarrow{\rightleftharpoons}$}}\text{}} = \{\text{NW}, N, NE, E, SE\}$$

$$D = \text{\textup{\textbf{\Large $\overrightarrow{\rightleftharpoons}$}}\text{}} = \{\text{NW}, N, SE\}$$

$$\mathcal{E} = \text{\textup{\textbf{\Large $\overrightarrow{\rightleftharpoons}$}}\text{}} = \{\text{NW}, N, NE, SE\}$$

Models A and D are the two models considered by Mishna and Rechnitzer, and their strategy, known as the iterated kernel method, applies to all of these models. Note that the present work corrects an error found in [14], which does not substantially change the stated results but which does imply some additional manipulation for the proof.

For each model $S \in \{A, B, C, D, \mathcal{E}\}$ we address the following:

1. What is the number $S_n$ of walks of length $n$ in the model?

2. How does $S_n$ grow asymptotically when $n$ is large?

3. Is the generating function $S(t) = \sum_n S_n t^n$ D-finite?

The next section describes how to obtain generating function expressions. This is followed by the asymptotic analysis and non D-finiteness proofs for the symmetric models. We conclude with a summary of the analysis of the asymmetric models.
# 2 An explicit expression for the generating function

## 2.1 The functional equation and its kernel

Our central mathematical object is the multivariate generating function $S_{x,y}(t) = \sum_{i,j,n} s_{ij}(n) x^i y^j t^n$, where $s_{ij}(n)$ counts the number of walks of length $n$ ending at the point $(i, j)$ with steps from $S$. (Throughout, $S$ is our generic step set.) Our goal is to determine properties of $S(t) \equiv S_{1,1}(t)$, the generating function for the number of walks in the plane.

For each of the five step sets, we associate a polynomial called the kernel; for the step set $S$, define

$$K_S(x, y) = xy - txy \sum_{(i,j) \in S} x^i y^j.$$

As we restrict ourselves to small steps, the inventory of the steps has the following form

$$\sum_{(i,j) \in S} x^i y^j = xP_1(y) + P_0(y) + \frac{1}{x}P_{-1}(y) = yQ_1(x) + Q_0(x) + \frac{1}{y}Q_{-1}(x). \quad (1)$$

Thus, $K_S(x, y)$ can be regarded as a quadratic in $y$ (respectively $x$) whose coefficients contain $t$, $x$ and the $Q_i(y)$ (resp. $t$, $y$, and $P_i(x)$):

$$K_S(x, y) = -txQ_1(x)y^2 + (x - xtQ_0(x))y - xtQ_{-1}(x). \quad (2)$$

When the model is clear, we omit the subscript $S$. One common property of the singular models is that they contain the steps NW and SE, and at least one other step – this prevents degeneracy in the quadratic.

Each model admits a functional equation for $S_{x,y}(t)$. We apply the common decomposition that a walk is either the empty walk, or a shorter walk followed by a single step. Taking into account the restrictions on walk location, as well as the fact that substituting $x = 0$ (respectively $y = 0$) into the function $S_{x,y}(t)$ gives the generating function of walks ending on the $y$-axis (respectively $x$-axis), we obtain, as many others have before us, the functional equation

$$K(x, y)S_{x,y}(t) = xy + K(x, 0)S_{x,0}(t) + K(0, y)S_{0,y}(t). \quad (3)$$

We are interested in the solutions to the kernel equation of the form:

$$K(x, Y_+(x; t)) = K(x, Y_-(x; t)) = K(X_+(y; t), y) = K(X_-(y; t), y) = 0, \quad (4)$$

and these algebraic functions are easily determined since the kernel is a quadratic:

$$Y_{\pm}(x; t) = \frac{(1 - tQ_0(x)) \mp \sqrt{(Q_0(x)^2 - 4Q_1(x)Q_{-1}(x))t^2 - 2Q_0(x)t + 1}}{2tQ_1(x)} \quad (5)$$

$$X_{\pm}(y; t) = \frac{(1 - tP_0(y)) \mp \sqrt{(P_0(y)^2 - 4P_1(y)P_{-1}(y))t^2 - 2P_0(y)t + 1}}{2tP_1(y)}. \quad (6)$$

There are other function pairs which annihilate the kernel, as we shall see. Remark that the boundary value method begins as we have, with the functional equation (3), but ultimately uses a different parametrization to represent the roots of the kernel, and from there a very different means to get access to the generating function.

The generating function has a natural expression in terms of iterated compositions of the $Y$ and $X$, hence the name iterated kernel method.
## 2.2 What makes this family special?

Consider the lowest order terms of the roots of the kernel as a power series in $t$. They are

$$Y_+ = P_{-1}(x)t + O(t^2) \quad \text{and} \quad Y_- = \frac{1}{tP_1(x)} - \frac{P_0(x)}{P_1(x)} + O(t),$$

where $P_r(x) = \sum_{(i,r) \in S} x^i$. Of the 56 (conjectured) non D-finite models only 5 models, precisely the singular family we are studying, have a lowest order term with a positive power in $x$ and $t$, implying that the infinite sum obtained by the iterated kernel method converges. This prevents the method from being applied to a broader range of models in this context.

## 2.3 Fast enumeration

Using the series expressions (8) and (9) below, we can generate the first $N$ terms of $S_{1,0}(t)$ and $S_{1,1}(t)$ for each model with $\tilde{O}(N^3)$ bit-complexity (where the notation $\tilde{O}(\cdot)$ suppresses logarithmic factors), which is an order of magnitude faster than the $\tilde{O}(N^4)$ bit-complexity of the naive generation algorithm. The key lies in utilizing a linear recurrence for $1/Y_n$ that the iterated kernel method generates for each model (see Table 1). The cost of generating $S_{1,0}(t)$ and $S_{1,1}(t)$ is then dominated by the inversion of the $1/Y_n$, which have summands whose bit-size grows linearly, giving the cubic complexity.

Thus, although the generating function is not D-finite, and hence the coefficients do not satisfy a nice fixed length linear recurrence, we are able to generate the terms in a relatively efficient manner.

# 3 Symmetric models: A, B, C

## 3.1 An explicit generating function expression

We focus first on the three models A, B and C, as these models are symmetric about the line $x = y$. As such, these models benefit from the relation $S_{x,0} = S_{0,x}$, and Equation (3) can be rewritten as

$$K(x, y)S_{x,y}(t) = xy + K(0, y)S_{0,y}(t) + K(0, x)S_{0,x}(t). \quad (7)$$

Our iterates satisfy $Y_{n+1}(x) = Y_+(Y_n(x))$, $Y_0(x) = x$, and consequently, $K(Y_n, Y_{n+1}) = 0$ for all $n$ by substituting $x = Y_n(x)$ into the kernel relation $K(x, Y_+(x)) = 0$. Thus, when we make this substitution into Equation (7) we find for each $n$:

$$0 = Y_n(x)Y_{n+1}(x) + K(0, Y_{n+1}(x))S_{0, Y_{n+1}(x)}(t) + K(0, Y_n(x))S_{0, Y_n(x)}(t).$$

We can determine an expression for $K(0, x)S_{0,x}(t)$ by taking an alternating sum of these equations since all of the $K(0, Y_n(x))S_{0, Y_n(x)}(t)$ terms are canceled for $n > 0$ in a telescoping sum:

$$0 = \sum_{n=0}^{\infty} (-1)^n \left( Y_n(x)Y_{n+1}(x) + K(0, Y_{n+1}(x))S_{0, Y_{n+1}(x)}(t) + K(0, Y_n(x))S_{0, Y_n(x)}(t) \right) \\ = K(0, x)S_{0,x}(t) + \sum_{n=0}^{\infty} (-1)^n Y_n(x)Y_{n+1}(x).$$
We rearrange this and evaluate at $x = 1$ to express the counting generating function for walks returning to the axis:

$$S_{0,1}(t) = \frac{1}{t} \sum_{n=0}^{\infty} (-1)^n Y_n(1) Y_{n+1}(1), \qquad (8)$$

as $K(0, 1) = -t$ for each case considered here. This converges as a power series because in each of these cases $Y_n(x) = O(t^n)$.

Furthermore, substituting $x = 1$ and $y = 1$ into Equation (7) gives the full counting generating function

$$S(t) = \frac{1 - 2tS_{0,1}(t)}{1 - t|S|} = \frac{1}{1 - t|S|} \left( 1 - 2 \sum_{n} (-1)^n Y_n(1) Y_{n+1}(1) \right). \qquad (9)$$

We address the robustness of this expression as a complex function in Theorem 2, after we are able to determine an explicit expression for $Y_n(1)$ as a rational function of $Y_1(1)$.

## 3.2 Asymptotic Enumeration

In each of these cases, the singularity at $\frac{1}{|S|}$ is dominant.

**Theorem 1** For each model $S$ in $\{A, B, C\}$, the sum $R(t) := \sum_n (-1)^n Y_n(1) Y_{n+1}(1)$ is convergent at $t = \frac{1}{|S|}$. The radius of convergence is bounded below by $t = \frac{1}{p_0+2\sqrt{p_1p_2-1}}$ where $p_i = P_i(1) = |\{(i, r) : -1 \le r \le 1, (i, r) \in S\}|$. The dominant singularity for each model $S$ in $\{A, B, C\}$ is a simple pole at $\sigma$, where $\frac{1}{\sigma} = |S|$, the number of permitted directions in the model. As a consequence, the number of walks $S_n = [t^n]S(t)$ grows asymptotically like

$$S_n \sim \kappa_S \left(\frac{1}{\sigma}\right)^n + O\left((p_0 + 2\sqrt{p_1p_2-1})^n\right),$$

where each $\kappa_S$ is the constant $R(\sigma)$, which can be calculated to arbitrary precision using Equation (8).

The strategy is to relate the quarter-plane problem to a half-plane problem, which has an explicit general solution given in [1]. We use the expressions that we find for the generating functions to demonstrate the convergence; the results are summarized below.

**Model** Asymptotic estimate for number of walks of length $n$

$$A \qquad A_n \sim \kappa_A 3^n + O\left((2\sqrt{2})^n\right) \qquad \kappa_A = 0.17317888...$$

$$B \qquad B_n \sim \kappa_B 4^n + O\left((1 + 2\sqrt{2})^n\right) \qquad \kappa_B = 0.15194581...$$

$$C \qquad C_n \sim \kappa_C 5^n + O\left((1 + 2\sqrt{3})^n\right) \qquad \kappa_C = 0.38220125...$$

## 3.3 Towards the Non D-Finiteness of $A(t)$, $B(t)$ and $C(t)$

The set of D-finite functions are closed under algebraic substitution. Thus, to prove that the generating functions $A(t)$, $B(t)$ and $C(t)$ are not D-finite, it is equivalent to consider these functions evaluated at $t = q/(1 + q^2)$. These turn out to be easier to analyze as the transformation concentrates the singularities around the unit circle. As such, we shall re-interpret the notation we have introduced this far to be functions of $q$ directly.
<figure><img src="image_1.png" /><figcaption>Fig. 1: Plots of the singularities of $Y_{20}(1)|_{t=\frac{q}{1+q^2}}$ for the three symmetric models.</figcaption></figure>

For each model the $Y_n(1)$ terms contribute singularities. A quick glance at an example is very suggestive – see Figure 1 for the singularities of $Y_{20}(1)$ in the $q$-plane for the three different models. The main difficulty is proving that there is no cancellation; that is, that the singularities in the figure are indeed present in the generating function. To prove this we follow these steps:

**Step 1** Determine an explicit expression for $Y_n(1)$;

**Step 2** Determine a polynomial $\sigma_n(q)$ whose set of roots contains the poles of $Y_n(1)$;

**Step 3** Determine a region where there are roots of $\sigma_n(q)$ that are truly poles of $Y_n(1)$;

**Step 4** Show that there is no point $\rho$ in that region that is a root of both $\sigma_n(q)$ and $\sigma_k(q)$ for different $n$ and $k$;

**Step 5** Demonstrate that $S_{1,0}(q/(1+q^2))$ has an infinite number of singularities and consequently, it is not D-finite. It follows that $S(t)$ is not D-finite, by closure under algebraic substitution and the expression in Equation (9).

### 3.3.1 Step 1: An explicit expression for $Y_n$

In this section we find an explicit, non-iterated expression for the functions $Y_n$. We follow the method of [14] very closely, with the exception that we make the variable substitution earlier in the process. As such, we repeat, that we view all functions as functions of $q$ in this section. From the variable substitution $t = q/(1+q^2)$ in Equation (3), we re-solve the kernel to ensure control over the choice of the branch in the solution. Here are the kernels:

$$K_A(x, y) = -q(x^2 + 1)y^2 + x(1 + q^2)y - qx^2$$

$$K_B(x, y) = -q(x + 1)y^2 + x(-qx + 1 + q^2)y - qx^2$$

$$K_C(x, y) = -q(1 + x + x^2)y^2 + x(-qx + 1 + q^2)y - qx^2.$$

We denote this generically as $K(x, y) = a_2y^2 + a_1y + a_0$, adapting the $a_i$ to each particular model. Each is solved as before to get our initial solutions to $K(x, Y(x)) = 0$. Great care is taken here to ensure that
the branch as written remains analytic at 0:

$$Y^{A}_{\pm 1}(x; q) = \frac{x}{2q(1+x^2)} \cdot \left(1 + q^2 \mp \sqrt{1 - 2(2x^2 + 1)q^2 + q^4}\right)$$

$$Y^{B}_{\pm 1}(x; q) = \frac{x}{2q(1+x)} \cdot \left(1 - qx + q^2 \mp \sqrt{q^4 - 2q^3x + (x^2 - 4x - 2)q^2 - 2qx + 1}\right)$$

$$Y^{C}_{\pm 1}(x; q) = \frac{x}{2q(1+x+x^2)} \cdot \left(1 - qx + q^2 \mp \sqrt{q^4 - 2q^3x - (3x^2 + 4x + 2)q^2 - 2qx + 1}\right).$$

We define the sequence of iterates {$Y_n(x)$}$_{(n)}$ as before: $Y_{n+1}(x) = Y_+(Y_n(x); q)$, $Y_1(x) = Y_+(x; q)$.

For each of these models, examining the coefficients of y in the kernel implies

$$\frac{1}{Y_{-}(x; q)} + \frac{1}{Y_{+}(x; q)} = \frac{Y_{-}(x; q) + Y_{+}(x; q)}{Y_{-}(x; q) \cdot Y_{+}(x; q)} = \frac{-a_{1}/a_{2}}{a_{0}/a_{2}} = -\frac{a_{1}}{a_{2}}. \qquad (10)$$

The iterates compose nicely, since for each model $Y_{-}(Y_{+}(x)) = Y_{+}(Y_{-}(x)) = x$.

It turns out to be easier to work with the reciprocal of $Y_n$, so we define $\bar{Y}_n = \frac{1}{Y_n(1)}$, and view this as a function of $q$. Equation (10) then converts into a recurrence after the substitution $x = Y_{n-1}(x)$. Specifically, this gives a linear recurrence for the reciprocal function, $\frac{1}{\bar{Y}_n(x)}$; we are interested in these evaluated at $x = 1$, and the resulting recurrences and their solutions in terms of $\bar{Y}_1$ are summarized below:

$$\begin{array}{lll}
\mathcal{S} & \textbf{Recurrence} & \bar{Y}_n^S(q) \\
\mathcal{A} & \bar{Y}_n = (q + \frac{1}{q})\bar{Y}_{n-1} - \bar{Y}_{n-2} & \frac{(q^2 - q^{2n}) + q(q^{2n} - 1)\bar{Y}_1}{q^n(q^2 - 1)} \\
\mathcal{B}, \mathcal{C} & \bar{Y}_n = (q + \frac{1}{q})\bar{Y}_{n-1} - \bar{Y}_{n-2} - 1 & \frac{q(q-1)(q^{2n}-1)\bar{Y}_1^{B,C} + (q-q^n)(2q^{n+1}-q^n+q^2-2q)}{q^n(q+1)(q-1)^2}
\end{array}$$

<caption>Tab. 1: The recurrences and their solutions for models A, B, and C.</caption>

Following the same procedure as above, we obtain a generic expression for $S_{1,0}(t)$, the generating function for the number of walks which return to the axis for model S, which can be applied to all three symmetric walks:

$$S_{1,0} \left( \frac{q}{1+q^2} \right) = (q+1/q) \sum_{n=0}^{\infty} (-1)^n Y_n^S(1) Y_{n+1}^S(1; q). \qquad (11)$$

Our careful choice of branches now implies that this is a formal power series. (Remark, this was not the case in [14].) Our expression is robust – an application of the ratio test implies the sum converges everywhere, except possibly on the unit circle and at the poles of the $Y_n$.

**Proposition 2** For $S \in \{A, B, C\}$ the sum $(q+1/q) \sum_{n=0}^{\infty} (-1)^n Y_n^S(1) Y_{n+1}^S(1)$ is convergent for all $q \in \mathbb{C}$ with $|q| \neq 1$, except possibly at the set of points defined by the singularities of the $Y_n^S(1)$ for all $n$.
### 3.3.2 Step 2: The singularities of $Y_n^S(1)$

In order to argue about the singularities, we find a family of polynomials $\sigma_n(q)$ that the roots of $\bar{Y}_n$ satisfy. The polynomials in Table 2 are obtained by manipulating the explicit expressions given above. Unfortunately, extraneous roots are introduced during the algebraic manipulation when an equation is squared to remove the square root present. In fact, the extraneous roots are exactly those which correspond to a negative sign in front of the square root. If one defines $\bar{Y}_{-1} = \bar{Y}_{-}$ and $\bar{Y}_{-n} = \bar{Y}_{-1} \circ \bar{Y}_{-(n-1)}$ for $n > 1$, then using the argument above one can check that $\bar{Y}_{-n}$ satisfies the same recurrence relation as $\bar{Y}_n$, up to a reversal of the sign in front of the square root. Thus, we see that the set of roots of $\sigma_n(q)$ is simply the union of the sets of roots of $\bar{Y}_n$ and $\bar{Y}_{-n}$.

$$S \quad \sigma_n(q)$$

$$A \quad \alpha_n(q) = q^{4n} + q^{2n+2} - 4q^{2n} + q^{2n-2} + 1$$

$$B \quad \beta_n(q) = (q^{2n-1} + (q^3 - 2q^2 - 2q + 1)q^{n-2} + 1)(q^{2n+1} + (q^3 - 2q^2 - 2q + 1)q^{n-1} + 1)$$

$$C \quad \gamma_n(q) = q^2(1 + q^2 - q)(1 + q^{4n}) + q(q^2 - 3q + 1)(q + 1)^2(q^n + q^{3n}) \\ + q^{2n}(1 - q^2 - 4q + 14q^3 - 4q^5 - q^4 + q^6)$$

**Tab. 2:** The singularities of $Y_n^S$ in the $q$-plane satisfy the polynomial $\sigma_n(q)$

Furthermore, we can show that these roots are dense around the unit circle using the results of Beraha, Kahane, and Weiss – specifically, a weakened statement of the Main Theorem of [2].

**Corollary 3** *The roots of the families of polynomials $\{\alpha_n(q)\}$, $\{\beta_n(q)\}$, and $\{\gamma_n(q)\}$ are dense around the unit circle.*

In addition, we can show that $\alpha_n(q)$ and $\gamma_n(q)$ have no roots on the unit circle, except possibly $q = \pm 1$, and if $q$ is a root of $\beta_n(q)$ on the unit circle not equal to 1 then

$$\arg q \in [\pi - \arccos(\sqrt{2} - \frac{1}{2}), \pi) \cup [-\pi, -\pi + \arccos(\sqrt{2} - \frac{1}{2})) \approx [2.7, 3.1) \cup [-2.7, -3.1).$$

This is complimented by the fact that Rouche's Theorem implies the roots converge to the unit circle as $n$ approaches infinity.

### 3.3.3 Step 3: Verify that $Y_n(1)$ has some singularities

At this point we have not yet completely established that the $Y_n(1)$ actually have singularities. Theoretically, it is possible that all the roots were added in our manipulations to determine $\sigma_n(q)$ for the different models. As we mentioned above, the roots of $\sigma_n(q)$ are either singularities of $Y_n(1)$ or singularities of $\bar{Y}_n(1)$. Thus, we prove Lemma 4 which describes at least some region where we are certain to find roots of $\bar{Y}_n$. Experimentally, it seems that the roots are evenly partitioned so that roots outside the unit circle belong to $\bar{Y}_n$ and those inside the unit circle belong to $\bar{Y}_{-n}$, but we do not prove this.

**Lemma 4** If $\arg(q) \in (-\pi/2, -3\pi/8) \cup (3\pi/8, \pi/2)$ then $\bar{Y}_n^S = \bar{Y}_{-n}^S|_{q \to 1/q}$ for all $n$, for $S \in \{A, B, C\}$. Consequently (using Corollary 3), for an infinite number of $n$ each of $Y_n^A, Y_n^B$, and $Y_n^C$ admit at least one singularity in the complex $q$-plane in that region.

The proof requires only basic manipulations of the formulas. Thus, we can ensure an infinite source of potential singularities for these generating functions.
### 3.3.4 Step 4: The singularities are distinct

We characterize the roots which are shared between $Y_n$ and $Y_k$ for the three models.

For each of the three models we find the roots of the numerators of our explicit expressions in Table 1 as quadratics in $q^n$. This determines functions $r_1(q)$ and $r_2(q)$, independent of $n$, such that $q_c^n = r_1(q_c)$ or $q_c^n = r_2(q_c)$ at any pole $q_c$ of $Y_n$. Furthermore, suppose $q_c$ is also a pole of $Y_k$ for $k \neq n$ so that $q_c^k = r_1(q_c)$ or $q_c^k = r_2(q_c)$. It is immediate that if $q_c^k = r_1(q_c) = q_c^n$ or $q_c^k = r_2(q_c) = q_c^n$ then $q_c$ must be on the unit circle. Otherwise, we may assume without loss of generality that $q_c^n = r_1(q_c)$ and $q_c^k = r_2(q_c)$. Each model can be considered separately to prove the following result:

**Proposition 5** For models A and C, if $q_c$ is a pole of $Y_n$ which lies off the unit circle then it is not a pole of $Y_k$ for $k \neq n$. For model B, if $q_c$ is a pole of $Y_n$ off the unit circle then it is not a pole of $Y_k$ for $|k-n| > 1$.

### 3.3.5 Step 5: The generating function is not D-finite

Now we tie up all the arguments.

**Theorem 6** The generating functions $A(t)$, $B(t)$, and $C(t)$ of walks in the quarter plane with steps from A, B, and C, respectively, are not D-finite.

**Proof:** As argued above, it is sufficient to prove the non D-finiteness of $S_{1,0}(t)$ evaluated at $t = q/(1+q^2)$ for each model. In Lemma 4 we have shown that $Y_n^A(1)$, $Y_n^B(1)$, and $Y_n^C(1)$ admit singularities off the unit circle for an infinite number of $n$. One can show that the singularities are not canceled in the two summands of (11) containing $Y_n^A$ and $Y_n^C$ (respectively the four summands containing $Y_{n-1}^B$, $Y_n^B$, and $Y_{n+1}^B$) so that, as they lie off the unit circle where the remaining summands are analytic by Proposition 5, they give an infinite number of singularities of $A_{1,0}$, $B_{1,0}$, and $C_{1,0}$. $\square$

## 4 Asymmetric models

The asymmetric models are not substantially different, but when we iterate, we have more functions to keep track of, and eventually cancel. Aside from some irritating bookkeeping, there is no main obstacle to following the strategy of the symmetric models.

### 4.1 An explicit generating function expression

To obtain the generating function expressions we follow the same path as in the symmetric case: we generate a sequence of equations, each which annihilates the kernel. This opens up the possibility of a telescoping sum expression from which we can find an expression for the generating function of walks returning to the axis in terms of iterates of two functions. An explicit expression for these iterates is obtained by solving some very simple recurrences. We complete these steps for the asymmetric models in this section.

As before, we begin with the main functional equation (3), make the substitution $t = q/(1+q^2)$, and re-arrange to get the kernel equations:

$$ \begin{aligned} (xy(1+q^2) - qy^2 - qxy^2 - qx^2) D_{x,y}(t) &= xy(1+q^2) - qx^2 D_{x,0}(t) - qy^2 D_{0,y}(t) \\ (xy(1+q^2) - qy^2 - qxy^2 - qx^2y^2 - qx^2) E_{x,y}(t) &= xy(1+q^2) - qx^2 E_{x,0}(t) - qy^2 E_{0,y}(t), \end{aligned} $$ 
with kernels 
$$ \begin{aligned} K_D(x, y) &= -q(1+x)y^2 + (q^2+1)xy - qx^2 \\ K_E(x, y) &= -q(1+x+x^2)y^2 + (1+q^2)xy - qx^2. \end{aligned} $$
As there is no longer an $x = y$ symmetry, we solve the kernels as functions of both $x$ and $y$; that is, we find $Y(x)$ satisfying $K(x, Y(x)) = 0$ and also $X(y)$ satisfying $K(X(y), y) = 0$. We have some choice over how we split the solutions over different branches. One such choice of branches is:

$$X^D_{\pm}(y; q) = \frac{y}{2q} \cdot \left(1 - qy + q^2 \mp \sqrt{q^4 - 2q^3y + (y^2 - 2)q^2 - 2qy + 1}\right)$$

$$Y^D_{\pm}(x; q) = \frac{x}{2q(1+x)} \cdot \left(1 + q^2 \mp \sqrt{q^4 - 4q^2x - 2q^2 + 1}\right).$$

$$X^E_{\pm}(y; q) = \frac{y}{2q(1+y^2)} \cdot \left(1 - qy + q^2 \mp \sqrt{q^4 - 2q^3y - (3y^2 + 2)q^2 - 2qy + 1}\right)$$

$$Y^E_{\pm}(x; q) = \frac{x}{2q(1+x+x^2)} \cdot \left(1 + q^2 \mp \sqrt{q^4 - 2(2x^2 + 2x + 1)q^2 + 1}\right).$$

Next, as we described in the introductory summary, we repeatedly alternate the substitution of the $X$ and $Y$ and create two related sequences of functions:

$$\chi_n(x) = X_+(Y_+(\chi_{n-1}(x); q); q), \quad \chi_0(x) = x \quad \text{and} \quad \Upsilon_n(y) = Y_+(X_+(\Upsilon_{n-1}(x); q); q), \quad \Upsilon_0(y) = y.$$

Simple substitutions yield the kernel relations $K(\chi_n(x), Y_+(\chi_n(x))) = K(X_+(\Upsilon(y)), \Upsilon(y)) = 0$, amongst others. As before, we generate an infinite list of relations by substituting $x = \chi_n(x)$, $y = Y_+(\chi_n(x))$, and then a second infinite list using the substitutions $x = X_+(\Upsilon(y))$, $y = \Upsilon(y)$. Again, we form a telescoping sum, and after some manipulation this results in an expression for the generating functions returning to the axis. For $S \in \{D, E\}$ we have:

$$S_{x,0} \left( \frac{q}{1+q^2} \right) = \frac{q}{1+q^2} \sum_{n \ge 0} \chi_n(x) \cdot \underbrace{(Y_+(\chi_n(x)) - Y_+(\chi_{n-1}(x)))}_{\Delta_L(x;q)} \quad (12)$$

$$S_{0,y} \left( \frac{q}{1+q^2} \right) = \frac{q}{1+q^2} \sum_{n \ge 0} X_+(\Upsilon_n(y)) \cdot \underbrace{(\Upsilon_n(y) - \Upsilon_{n+1}(y))}_{\Delta_R(y;q)}. \quad (13)$$

The two models have identical structure in their generating function, and differ only in their respective functions $X_1$ and $Y_1$. Our greatest challenge at this point is keeping track of the various parts:

$$\chi'_{0}(x) = Y_{1}(x), \quad \chi'_{n}(x) = Y_{1}(\chi_{n}(x)), \quad \Upsilon'_{n}(y) = X_{1}(\Upsilon_{n}(y)), \quad \Upsilon'_{0}(y; q) = X_{1}(y)$$

$$\Delta_{L,0} = \chi'_{0}(x), \quad \Delta_{L,n}(x) = \chi'_{n}(x) - \chi'_{n-1}(x), \quad \Delta_{R,0} = \Upsilon_{0}(y), \quad \Delta_{R,n}(y) = \Upsilon_{n}(y) - \Upsilon_{n+1}(y)$$

For each model, we isolate the left and right hand sides: $L_x^S(q) = qx^2S_{x,0}(q/(1+q^2))$ and $R_y^S(q) = qy^2S_{0,y}(q/(1+q^2))$ so that

$$K_S(x, y)S_{x,y}(q/(1+q^2)) = xy(1+q^2) - L_x^S(q) - R_y^S(q).$$

Similar to previous cases, we can use the coefficients of $K_S(x, y)$ and the two identities $Y_{\pm 1}(X_{\mp 1}(y)) = y$ and $X_{\pm 1}(Y_{\mp 1}(x)) = x$ to form the paired up recurrences for the multiplicative inverses of these functions. Here we use the notation that $\bar{F} = \frac{1}{F}$:

$$\bar{\chi}_{\pm n} = (q + 1/q)\bar{\chi}'_{\pm(n-1)} - \bar{\chi}_{\pm(n-1)} - 1,$$

$$\bar{\Upsilon}_{\pm n} = (q + 1/q)\bar{\Upsilon}'_{\pm(n-1)} - \bar{\Upsilon}_{\pm(n-1)}^{S},$$

$$\bar{\chi}'_{\pm n} = (q + 1/q)\bar{\chi}'_{\pm n} - \bar{\chi}_{\pm(n-1)}$$

$$\bar{\Upsilon}'_{\pm n}^{S} = (q + 1/q)\bar{\Upsilon}_{\pm n}^{S} - \bar{\Upsilon}'_{\pm(n-1)} - 1.$$
These recurrences are easily solved, as before, as closed form expressions. Again, we can verify that we have suitable expressions for the analytic continuation of the generating functions for $\mathcal{D}$ and $\mathcal{E}$. These expressions are well suited to exact enumeration, as in the symmetric case, and they also yield asymptotic enumeration formulas.

**Theorem 7 (Mishna and Rechnitzer [14]; Proposition 16)** If $D_n$ denotes the number of walks with steps from $\mathcal{D}$ staying in the positive quarter plane, then $D_n \sim \kappa_D \frac{3^n}{\sqrt{n}}(1+o(1))$ where $0 < \kappa_D \le \sqrt{\frac{3}{\pi}}$.

**Theorem 8** The function $E(t)$ has a simple singularity at $t = 1/4$, where it has a residue of value $\kappa_E \in [\frac{122}{525}, \frac{7}{10}]$. The number, $E_n$, of walks taking steps in $\mathcal{E}$ and staying in the positive quarter plane grows asymptotically as $E_n = \kappa_E \cdot 4^n + O((1+2\sqrt{2})^n)$.

Computational evidence given by calculating the series for $E_{1,0}(1/4)$ and $E_{0,1}(1/4)$ to a large number of terms implies that the value of the growth constant is approximately 0.2636, which is consistent with the growth of computationally generated values of $E_n$ for large $n$.

## 4.2 The generating functions $D(t)$ and $E(t)$ are not D-finite

The additional sums that arise in our expressions for $D(t)$ and $E(t)$ do not change our fundamental strategy, demonstrating an infinite set of singularities, but here we actually have a simpler argument in hand. In both cases the series $L_1(q)$ has an infinite source of singularities on the imaginary axis, at each of which $R_1(q)$ is analytic.

**Lemma 9** Both of the functions $\bar{\chi}_n^D(q)$ and $\bar{\chi}_n^E(q)$ have a root on the imaginary axis between $i$ and $2i$, when $n$ is even.

**Proof:** First, we show that the roots of $\bar{\chi}_n^D(q)$ satisfy the polynomial

$$\omega_n^D = (q^{4n+2} + q^{2n+4} - 4q^{2n+2} + q^{2n} + q^2)^2$$

and that the roots of $\bar{\chi}_n^E(q)$ satisfy the polynomial

$$\omega_n^E = q^2 (q^4 - q^2 + 1) (q^{8n} + 1) + 2q^2 (q^4 - 4q^2 + 1) (q^{6n} + q^{2n}) + (q^8 - 10q^6 + 24q^4 - 10q^2 + 1)q^{4n}.$$

The rest is straightforward manipulation as in [14].

<figure><img src="image_2.png" /></figure>

A sampling of these singularities is given in Figure 2.

<figure><img src="image_3.png" /><figcaption>**Fig. 2:** The modulus of the purely imaginary singularities of $D(t)$ and $E(t)$, coming from $\chi_{2n}$ for $n = 1, \dots, 100$.</figcaption></figure>

**Theorem 10** Neither the generating function $D(t)$ nor the generating function $E(t)$ for walks in the quarter plane with steps from $\mathcal{D}$ and $\mathcal{E}$, respectively, are D-finite.
# 5 Conclusion

This work addresses a family of lattice path models that have resisted other powerful approaches. There should also be other models, with larger step sizes or in higher dimensions, to which this method may be suitable. We intend to seek them out and to try to apply this method, ideally automating as much as is possible. Finally, we are always in a search to understand the combinatorial nature of D-finite functions. Are there properties inherent to these classes from which we should be able immediately to predict the nature of the generating function? Where does the intuition lie?

# 6 Acknowledgments

We are indebted to Alin Bostan, Mireille Bousquet-Mélou, Manuel Kauers, Pierre Lairez, and Andrew Rechnitzer for some key discussions.

# References

[1] C. Banderier, and Ph. Flajolet. Basic Analytic Combinatorics of Lattice Paths. *Theoret. Comput. Sci.*, 281(1-2):3780, 2002.

[2] S. Beraha, J. Kahane, and N. J. Weiss. Limits of zeroes of recursively defined polynomials. *Proc. Nat. Acad. Sci. U.S.A.*, 72(11):4209, 1975.

[3] A. Bostan and M. Kauers. Automatic classification of restricted lattice walks. In *21st International Conference on Formal Power Series and Algebraic Combinatorics (FPSAC 2009)*, Discrete Math. Theor. Comput. Sci. Proc., AK, pages 201–215. Assoc. Discrete Math. Theor. Comput. Sci., Nancy, 2009.

[4] A. Bostan, K. Raschel, and B. Salvy. Non-D-finite excursions in the quarter plane. Pre-print, http://arxiv.org/abs/1205.3300.

[5] M. Bouquet-Mélou. Walks in the quarter plane: Kreweras' algebraic model. *Annals of Applied Probability*, 15(2):2005, 2005.

[6] M. Bousquet-Mélou and M. Petkovsec. Walks confined in a quadrant are not always D-finite. *Theoret. Comput. Sci.*, 307:257-276, 2003.

[7] M. Bousquet-Mélou and M. Mishna. Walks with small steps in the quarter plane. In *Algorithmic probability and combinatorics*, volume 520 of *Contemp. Math.*, pages 1–39. Amer. Math. Soc., Providence, RI, 2010.

[8] G. Fayolle, R. Iasnogorodski, and V. A. Malyshev. *Random walks in the quarter-plane: algebraic methods, boundary value problems and applications*. Applications of mathematics. Springer, 1999.

[9] G. Fayolle and K. Raschel. On the holonomy or algebraicity of generating functions counting lattice walks in the quarter-plane. *Markov Process. Related Fields*, 16(3):485–496, 2010.

[10] E. J. Janse van Rensburg, T. Prellberg, and A. Rechnitzer. Partially directed paths in a wedge. *J. Combin. Theory Ser. A*, 115(4):623–650, 2008.

[11] S. Johnson, M. Mishna, and K. Yeats. Towards a combinatorial understanding of lattice path asymptotics. Manuscript in progress, 2013.

[12] I. Kurkova and K. Raschel. Explicit expression for the generating function counting Gessel's walks. *Adv. in Appl. Math.*, 47(3):414–433, 2011.

[13] S. Melczer and M. Mishna. Singularity analysis via the iterated kernel method. Preprint. http://arxiv.org/abs/arXiv:1303.3236.

[14] M. Mishna and A. Rechnitzer. Two non-holonomic lattice walks in the quarter plane. *Theoret. Comput. Sci.*, 410(38-40):3616–3630, 2009.