# Coefficientwise Hankel-total positivity of the Schett polynomials

Bishal Deb, Alan D. Sokal

<figure><img src="image_1.png" /></figure>

## To cite this version:

Bishal Deb, Alan D. Sokal. Coefficientwise Hankel-total positivity of the Schett polynomials. 2023. [https://hal.science/hal-04297025v1](https://hal.science/hal-04297025v1)

HAL Id: hal-04297025

[https://hal.science/hal-04297025v1](https://hal.science/hal-04297025v1)

Preprint submitted on 21 Nov 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
# Coefficientwise Hankel-total positivity of the Schett polynomials

Bishal Deb*<fnref n="1" />,<fnref n="2" /> and Alan D. Sokal†<fnref n="1" />,<fnref n="3" />

**Abstract.** We prove the coefficientwise Hankel-total positivity of the even and odd subsequences of Schett polynomials $X_n(x, y, z)$.

**Keywords:** Schett polynomials, Jacobian elliptic functions, total positivity, production matrix, exponential Riordan array, checkerboard exponential Riordan array

## 1 Introduction

A half-century ago, Schett [17, 18] introduced implicitly, and then Dumont [5, 6] introduced explicitly, the sequence of polynomials

$$X_n(x,y,z) = \left(yz \frac{\partial}{\partial x} + xz \frac{\partial}{\partial y} + xy \frac{\partial}{\partial z}\right)^n x, \quad (1.1)$$

which we shall call the *Schett polynomials* initialized in $x$; clearly they are homogeneous polynomials of degree $n+1$ with nonnegative integer coefficients, and are symmetric in $y \leftrightarrow z$. Schett [17, 18] showed that the polynomials $X_n(x, y, z)$ unify and generalize the Taylor coefficients of the Jacobian elliptic functions sn, cn, dn.

It is easy to see that for even (resp. odd) $n$, the polynomial $X_n(x, y, z)$ is divisible by $x$ (resp. by $yz$) and that the quotient is a polynomial in $x^2, y^2, z^2$. We therefore define the reduced Schett polynomials

$$\hat{X}_n(x^2, y^2, z^2) = \begin{cases} \frac{1}{x} X_n(x, y, z) & \text{for } n \text{ even} \\ \frac{1}{yz} X_n(x, y, z) & \text{for } n \text{ odd} \end{cases} \quad (1.2)$$

Then $\hat{X}_{2k}$ and $\hat{X}_{2k+1}$ are homogeneous polynomials of degree $k$ in $x^2, y^2, z^2$. It is also not difficult to show that these two polynomials coincide when $x = 0$: that is, $\hat{X}_{2k}(0, y^2, z^2) = \hat{X}_{2k+1}(0, y^2, z^2)$.

<footnote n="1">¹University College London, London, UK</footnote>

<footnote n="2">²Sorbonne Université and Université Paris Cité, CNRS, Laboratoire de Probabilités, Statistique et Modélisation, Paris, France</footnote>

<footnote n="3">³New York University, New York, NY, USA</footnote>

<footnote>*bishal@gonitsora.com. Partially supported by the DIMERS project ANR-18-CE40-0033.</footnote>

<footnote>†sokal@nyu.edu</footnote>
## 1.1 Total positivity and production matrices

The main purpose of the present paper is to prove the coefficientwise Hankel-total positivity of the even and odd subsequences of Schett polynomials. Recall first that a finite or infinite matrix of real numbers is called totally positive (TP) if all its minors are nonnegative. Background information on totally positive matrices can be found in [11, 10, 15, 7]; they have applications to many areas of pure and applied mathematics.<fnref n="1" /> In particular, it is known [9, Théorème 9] [15, section 4.6] that an infinite Hankel matrix $(a_{i+j})_{i,j\ge 0}$ of real numbers is totally positive if and only if the underlying sequence $(a_n)_{n\ge 0}$ is a Stieltjes moment sequence, i.e. the moments of a positive measure on $[0, \infty)$.

However, this is only the beginning of the story, because our main interest [19] is not with sequences and matrices of real numbers, but rather with sequences and matrices of polynomials (with integer or real coefficients) in one or more indeterminates $x$: in applications they will typically be generating polynomials that enumerate some combinatorial objects with respect to one or more statistics. We equip the polynomial ring $\mathbb{R}[x]$ with the coefficientwise partial order: that is, we say that $P$ is nonnegative (and write $P \succeq 0$) in case $P$ is a polynomial with nonnegative coefficients. We then say that a matrix with entries in $\mathbb{R}[x]$ is coefficientwise totally positive if all its minors are polynomials with nonnegative coefficients; and we say that a sequence $a = (a_n)_{n\ge 0}$ with entries in $\mathbb{R}[x]$ is coefficientwise Hankel-totally positive if its associated infinite Hankel matrix $H_\infty(a) = (a_{i+j})_{i,j\ge 0}$ is coefficientwise totally positive. Most generally, we can consider sequences and matrices with entries in an arbitrary partially ordered commutative ring; total positivity and Hankel-total positivity are then defined in the obvious way. Coefficientwise Hankel-total positivity of a sequence of polynomials $(P_n(x))_{n\ge 0}$ implies the pointwise Hankel-total positivity (i.e. the Stieltjes moment property) for all $x \ge 0$, but it is vastly stronger.

Our first main theorem is:

**Theorem 1.1 (Coefficientwise Hankel-total positivity of the Schett polynomials).**

(a) The sequence $(\hat{X}_{2n}(x^2, y^2, z^2))_{n\ge 0}$ of even reduced Schett polynomials is coefficientwise Hankel-totally positive in $x, y, z$.

(b) The sequence $(\hat{X}_{2n+1}(x^2, y^2, z^2))_{n\ge 0}$ of odd reduced Schett polynomials is coefficientwise Hankel-totally positive in $x, y, z$.

We remark that the $z=0$ specialization of Theorem 1.1(a) follows from a continued fraction due to Stieltjes [21, p. H17] and Rogers [16, p. 77] together with some general theory [19].<fnref n="2" /> But the general case of Theorem 1.1 is considerably more subtle.

<footnote n="1">¹See [20, footnote 4] for many references.</footnote>

<footnote n="2">²See also Flajolet and Françon [8, Theorem 1] and Milne [13, Theorems 3.2 and 3.11] for modern presentations of Rogers' elegant proof of the continued fraction.</footnote>
Our proof of Theorem 1.1 will be based on the method of production matrices [3, 4].
Let $P = (p_{ij})_{i,j \ge 0}$ be an infinite matrix with entries in a commutative ring $R$; we assume
that $P$ is either row-finite (i.e. has only finitely many nonzero entries in each row) or
column-finite. Now define an infinite matrix $A = (a_{nk})_{n,k \ge 0}$ by $a_{nk} = (P^n)_{0k}$. We call $P$
the **production matrix** and $A$ the **output matrix**, and we write $A = \mathcal{O}(P)$. The two key
facts here are the following [14, section 9.2]: if $R$ is a partially ordered commutative ring
and $P$ is totally positive, then $\mathcal{O}(P)$ is totally positive [14, Theorem 9.4], and the zeroth
column of $\mathcal{O}(P)$ is Hankel-totally positive [14, Theorem 9.7].

We will prove Theorem 1.1 by exhibiting production matrices that generate the even or odd reduced Schett polynomials as the zeroth column of their output matrix, and then proving the total positivity of those production matrices. The production matrices will be quadridiagonal unit-lower-Hessenberg matrices: that is, they will have a superdiagonal with all entries equal to 1, and then a diagonal, a subdiagonal, and a second subdiagonal; all other entries are equal to 0. In the language of [14], these are the production matrices for a 2-branched J-fraction. Similar but simpler quadridiagonal production matrices arose in [1]. In particular, we will prove the following two results:

**Proposition 1.2** (Production matrices for even and odd Schett polynomials).

(a) The quadridiagonal production matrix $P(x, y, z) = (p_{ij})_{i,j \ge 0}$ defined by

$$p_{n,n+1} = 1 \tag{1.3a}$$

$$p_{n,n} = (2n)^2 x^2 + (2n+1)^2 (y^2 + z^2) \tag{1.3b}$$

$$p_{n,n-1} = (2n)^2(2n-1) \left[ (2n-1)x^2(y^2+z^2) + (2n+1)y^2z^2 \right] \tag{1.3c}$$

$$p_{n,n-2} = (2n)^2(2n-2)^2(2n-1)(2n-3)x^2y^2z^2 \tag{1.3d}$$

$$p_{n,k} = 0 \quad \text{if } k < n-2 \text{ or } k > n+1 \tag{1.3e}$$

generates the even reduced Schett polynomials $(\hat{X}_{2n}(x^2, y^2, z^2))_{n \ge 0}$ as the zeroth column of its output matrix $\mathcal{O}(P)$.

(b) The quadridiagonal production matrix $Q(x, y, z) = (q_{ij})_{i,j \ge 0}$ defined by

$$q_{n,n+1} = 1 \tag{1.4a}$$

$$q_{n,n} = (2n+2)^2 x^2 + (2n+1)^2 (y^2 + z^2) \tag{1.4b}$$

$$q_{n,n-1} = (2n)^2(2n+1) \left[ (2n+1)x^2(y^2+z^2) + (2n-1)y^2z^2 \right] \tag{1.4c}$$

$$q_{n,n-2} = (2n)^2(2n-2)^2(2n+1)(2n-1)x^2y^2z^2 \tag{1.4d}$$

$$q_{n,k} = 0 \quad \text{if } k < n-2 \text{ or } k > n+1 \tag{1.4e}$$

generates the odd reduced Schett polynomials $(\hat{X}_{2n+1}(x^2, y^2, z^2))_{n \ge 0}$ as the zeroth column of its output matrix $\mathcal{O}(Q)$.
**Proposition 1.3** (Total positivity of the production matrices).

(a) The quadridiagonal production matrix $P(x, y, z)$ defined in (1.3) is coefficientwise totally positive in $x, y, z$.

(b) The quadridiagonal production matrix $Q(x, y, z)$ defined in (1.4) is coefficientwise totally positive in $x, y, z$.

We note that the production matrices $P(x, y, z)$ and $Q(x, y, z)$ coincide when $x = 0$; this is consistent with the fact that $\hat{X}_{2n}(0, y^2, z^2) = \hat{X}_{2n+1}(0, y^2, z^2)$.

Combining Propositions 1.2 and 1.3 with the general theory of totally positive production matrices [14, Theorem 9.7] proves Theorem 1.1.

## 1.2 Exponential generating functions of the output matrices

We can also give exponential generating functions for the output matrices $O(P)$ and $O(Q)$, using the theory of exponential Riordan arrays. Recall first that if $F(t)$ and $G(t)$ are formal power series with coefficients in a commutative ring $R$, with $G(0) = 0$, then the exponential Riordan array $\mathcal{R}[F, G]$ is the infinite lower-triangular matrix with entries

$$ \mathcal{R}[F, G]_{nk} = \frac{n!}{k!} [t^n] F(t) G(t)^k \in R. \quad (1.5) $$

If the ring $R$ contains the rationals (as it will here), this says that the exponential generating function of the $k$th column of $\mathcal{R}[F, G]$ is $F(t)G(t)^k/k!$. Note also that if $F$ is even and $G$ is odd, then $\mathcal{R}[F, G]_{nk}$ is nonvanishing only when $n$ and $k$ have the same parity; that is, only the even-even and odd-odd submatrices of $\mathcal{R}[F, G]$ are nonvanishing. In this situation we call $\mathcal{R}[F, G]$ a *checkerboard exponential Riordan array*.

We will show that the output matrix $O(P)$ is the even-even submatrix of a particular checkerboard exponential Riordan array. Clearly $F$ must be the exponential generating function of the zeroth column of $O(P)$, which by Proposition 1.2(a) are the even reduced Schett polynomials $\hat{X}_{2n}(x^2, y^2, z^2)$, alternated with zero entries. Hence we must have

$$ F(t) = \sum_{n=0}^{\infty} \hat{X}_{2n}(x^2, y^2, z^2) \frac{t^{2n}}{(2n)!} = \mathrm{Dn}(t; x, y, z) \mathrm{En}(t; x, y, z), \quad (1.6) $$

where $\mathrm{Dn}$ and $\mathrm{En}$ are Dumont's hyperelliptic functions [6, section 3], and the final equality is [6, eq. (3.11)]. For $G$, it turns out that we will have $G(t) = \mathrm{Sn}(t; x, y, z)$, where $\mathrm{Sn}$ is another of Dumont's hyperelliptic functions. We will therefore prove:

**Proposition 1.4** (Exponential generating function for the output matrix $O(P)$).

Let the matrix $P$ be as in Proposition 1.2(a), and let $F(t)$ and $G(t)$ be as above. Then the output matrix $O(P)$ is the even-even submatrix of the exponential Riordan array $\mathcal{R}[F, G]$:

$$ O(P)_{nk} = \frac{(2n)!}{(2k)!} [t^{2n}] F(t) G(t)^{2k}. \quad (1.7) $$
We can do something analogous for the output matrix $\mathcal{O}(Q)$ associated to the odd Schett polynomials, but this time it will be most convenient to find it as the odd-odd submatrix of a checkerboard exponential Riordan array. We will again have $G(t) = \text{Sn}(t; x, y, z)$. Since the zeroth column of $\mathcal{O}(Q)$, which by Proposition 1.2(b) are the odd reduced Schett polynomials $\hat{X}_{2n+1}(x^2, y^2, z^2)$, will now appear in the $k=1$ column of $\mathcal{R}[F, G]$ in odd rows starting at $n=1$, we must have

$$F(t)G(t) = \sum_{n=0}^{\infty} \hat{X}_{2n+1}(x^2, y^2, z^2) \frac{t^{2n+1}}{(2n+1)!} = \text{Sn}(t; x, y, z) \text{Cn}(t; x, y, z), \quad (1.8)$$

where $\text{Sn}$ and $\text{Cn}$ are Dumont's hyperelliptic functions, and the final equality is again [6, eq. (3.11)]. Therefore $F(t) = \text{Cn}(t; x, y, z)$. We will then prove:

**Proposition 1.5** (Exponential generating function for the output matrix $\mathcal{O}(Q)$).

Let the matrix $Q$ be as in Proposition 1.2(b), and let $F(t)$ and $G(t)$ be as above. Then the output matrix $\mathcal{O}(Q)$ is the odd-odd submatrix of the exponential Riordan array $\mathcal{R}[F, G]$:

$$\mathcal{O}(Q)_{nk} = \frac{(2n+1)!}{(2k+1)!} [t^{2n+1}] F(t)G(t)^{2k+1}. \quad (1.9)$$

## 2 Proof of Propositions 1.2, 1.4 and 1.5

We shall prove Proposition 1.4 (resp. 1.5) by considering the checkerboard exponential Riordan array defined there and showing that the production matrix of its even-even (resp. odd-odd) submatrix is indeed what is written in Proposition 1.2(a) [resp. 1.2(b)]. Since the zeroth column of this submatrix is, by construction, the sequence of even (resp. odd) reduced Schett polynomials, this will also prove Proposition 1.2.

The production matrix of an exponential Riordan array $\mathcal{R}[F, G]$ is well known [20, Theorem 2.19] to be $p_{nk} = (n!/k!) (z_{n-k} + k a_{n-k+1})$, where $A(s) = \sum_{n=0}^{\infty} a_n s^n$ and $Z(s) = \sum_{n=0}^{\infty} z_n s^n$ satisfy

$$G'(t) = A(G(t)), \quad \frac{F'(t)}{F(t)} = Z(G(t)). \quad (2.1)$$

Here we will give the analogous result for the even-even and odd-odd submatrices of a checkerboard exponential Riordan array.

First we define some new series:

$$B(s) \stackrel{\text{def}}{=} A(s)^2, \quad C(s) \stackrel{\text{def}}{=} 2A(s)Z(s), \quad D(s) \stackrel{\text{def}}{=} Z(s)^2 + A(s)Z'(s). \quad (2.2)$$

Since $F$ is even and $G$ is odd, it follows that $A$ is even and $Z$ is odd; then $B$ and $D$ are even, while $C$ is odd, and we can write

$$B(t) = \sum_{m=0}^{\infty} b_m t^{2m}, \quad C(t) = \sum_{m=0}^{\infty} c_m t^{2m+1}, \quad D(t) = \sum_{m=0}^{\infty} d_m t^{2m}. \quad (2.3)$$
We then have:

**Theorem 2.1** (Production matrices for submatrices of a checkerboard exponential Riordan array). Let $\mathcal{R}[F, G]$ be a checkerboard exponential Riordan array with invertible diagonal entries and $F(0) = 1$, and let $\mathcal{R}[F, G]^{ee}$ and $\mathcal{R}[F, G]^{oo}$ be its even-even and odd-odd submatrices. Then:

(a) The production matrix of $\mathcal{R}[F, G]^{ee}$ has entries

$$p_{nk} = \frac{(2n)!}{(2k)!} \left[ 2k(n+k)b_{n-k+1} + 2k c_{n-k} + d_{n-k} \right]. \qquad (2.4)$$

(b) The production matrix of $\mathcal{R}[F, G]^{oo}$ has entries

$$p_{nk} = \frac{(2n+1)!}{(2k+1)!} \left[ (2k+1)(n+k+1)b_{n-k+1} + (2k+1)c_{n-k} + d_{n-k} \right]. \qquad (2.5)$$

The proof will be based on the following simple but powerful result, which we discovered very recently and which to our surprise is apparently new; it is inspired by the integration-by-parts arguments employed by Stieltjes [21], Rogers [16] and others [8, 13] to deduce classical J-fractions.

**Proposition 2.2** (Exponential-generating-function method for production matrices). Let $A = (a_{nk})_{n,k \ge 0}$ and $P = (p_{nk})_{n,k \ge 0}$ be row-finite matrices with entries in a commutative ring $\mathbb{R}$ containing the rationals, with $a_{0k} = \delta_{k0}$. Fix integers $r \ge 1$ and $0 \le s < r$, and define the column exponential generating functions of $A$:

$$A_k^{[r,s]}(t) = \sum_{n=0}^{\infty} a_{nk} \frac{t^{rn+s}}{(rn+s)!}. \qquad (2.6)$$

Then the following are equivalent:

(a) For all $k \ge 0$,

$$\frac{d^r}{dt^r} A_k^{[r,s]}(t) = \sum_{n=0}^{\infty} p_{nk} A_n^{[r,s]}(t). \qquad (2.7)$$

(b) $A = \mathcal{O}(P)$.

**PROOF.** On both sides of (2.7), the only powers of $t$ that can occur are $t^{rm+s}$ with $m \ge 0$; for the left-hand side we have here used the hypothesis that $0 \le s < r$. So take the coefficient of $t^{rm+s}/(rm+s)!$ on both sides of (2.7):

$$\left[ \frac{t^{rm+s}}{(rm+s)!} \right] \frac{d^r}{dt^r} A_k^{[r,s]}(t) = \left[ \frac{t^{r(m+1)+s}}{(r(m+1)+s)!} \right] A_k^{[r,s]}(t) = a_{m+1,k} \qquad (2.8)$$
while

$$ \left[ \frac{t^{rm+s}}{(rm+s)!} \right] \sum_{n=0}^{\infty} p_{nk} A_{n}^{[r,s]}(t) = \sum_{n=0}^{\infty} p_{nk} a_{mn}. \qquad (2.9) $$

But $a_{m+1,k} = \sum_{n=0}^{\infty} a_{mn} p_{nk}$ is precisely the recurrence stating that $A = \mathcal{O}(P)$. $\square$

Note that, to prove that $A = \mathcal{O}(P)$, it suffices to verify the identities (2.7) for one pair $(r,s)$. On the other hand, if $A = \mathcal{O}(P)$, then the identities (2.7) hold for all pairs $(r,s)$ satisfying $0 \le s < r$.

We remark also that (a) $\implies$ (b) holds even when $s \ge r$; but in this case the hypothesis (2.7) is unlikely to hold, because there are terms that will occur on the left-hand side (unless they happen to vanish) that cannot occur on the right-hand side, namely $t^{rm+s}$ with $m < 0$. We will now see an example of this.

**SKETCH OF PROOF OF THEOREM 2.1.** We apply Proposition 2.2 with $r = 2$, taking $s = 0$ for part (a) and $s = 1$ for part (b). That is, we will compute the production matrix of the matrix with elements $a_{nk} = \mathcal{R}[F,G]_{2n+s,2k+s}$. Here we can take in principle any integer $s \ge 0$, but we will see at the end why only $s = 0,1$ give a valid final result.

For any $k \in \mathbb{Z}$, let $H_k(t)$ be the exponential generating function of column $2k+s$ of $\mathcal{R}[F,G]$,

$$ H_k(t) \stackrel{\text{def}}{=} \frac{F(t) G(t)^{2k+s}}{(2k+s)!}, \qquad (2.10) $$

where by convention we take $H_k(t) \stackrel{\text{def}}{=} 0$ if $2k+s < 0$. A straightforward computation (about 1 page long) using (2.1)–(2.3) shows that

$$ H_k''(t) = \sum_{n=-1}^{\infty} p_{nk} H_n(t) \qquad (2.11) $$

where

$$ p_{nk} = \frac{(2n+s)!}{(2k+s)!} \left[ (2k+s)(n+k+s) b_{n-k+1} + (2k+s) c_{n-k} + d_{n-k} \right]. \qquad (2.12) $$

In general the term $n = -1$ can contribute to the sum (2.11) when $k = 0$, via $b_{n-k+1} = b_0$ (note that $H_{-1}$ is nontrivial if $s \ge 2$); but its coefficient is $s(s-1)$ and hence vanishes for $s \in \{0,1\}$ (and in any case $H_{-1} = 0$ for $s \in \{0,1\}$). Therefore, for $s \in \{0,1\}$ we can apply Proposition 2.2 with $r = 2$. $\square$

We also have another proof of Theorem 2.1; we will publish it in the fuller version of this work.
We now apply Theorem 2.1 to the checkerboard exponential Riordan arrays defined in Propositions 1.4 and 1.5. From $G(t) = \text{Sn}(t; x, y, z)$ and the differential equation satisfied by $\text{Sn}$ [6, eq. (3.7)], we have immediately

$$A(s) = [(1 + x^2 s^2)(1 + y^2 s^2)(1 + z^2 s^2)]^{1/2}. \qquad (2.13)$$

For Proposition 1.4 we have $F(t) = \text{Dn}(t; x, y, z) \text{En}(t; x, y, z)$; using [6, eq. (3.3)/(3.4)] we get

$$Z(s) = s A(s) \left[ \frac{y^2}{1 + y^2 s^2} + \frac{z^2}{1 + z^2 s^2} \right]. \qquad (2.14)$$

For Proposition 1.5 we have $F(t) = \text{Cn}(t; x, y, z)$; using [6, eq. (3.3)/(3.4)] we get

$$Z(s) = s A(s) \frac{x^2}{1 + x^2 s^2}. \qquad (2.15)$$

From this we compute, in each case, the series $B, C, D$ and the coefficients $b_m, c_m, d_m$; this computation is straightforward and we leave it to the reader. Substituting these coefficients into Theorem 2.1 yields the production matrices of Proposition 1.2.

We remark that when $x = y$ and $z = 0$, these two checkerboard exponential Riordan arrays coincide; this array was found earlier in [2, eq. (6.24)].

### 3 Proof of Proposition 1.3

We begin by observing that the production matrices $P$ and $Q$ defined in Proposition 1.2 have a nice factorization. Let $T(y, z) \stackrel{\text{def}}{=} P(0, y, z) = Q(0, y, z)$ be the production matrix at $x = 0$: it is tridiagonal with matrix elements

$$T_{n,n+1} = 1 \qquad (3.1a)$$

$$T_{n,n} = (2n + 1)^2 (y^2 + z^2) \qquad (3.1b)$$

$$T_{n,n-1} = 4n^2(4n^2 - 1)y^2z^2 \qquad (3.1c)$$

And let $L(x)$ be the lower-bidiagonal matrix with entries

$$L_{n,n} = 1 \qquad (3.2a)$$

$$L_{n,n-1} = 4n^2 x^2 \qquad (3.2b)$$

A simple computation shows:

**Proposition 3.1 (Factorization of the production matrices).**

(a) $P(x, y, z) = L(x) T(y, z)$.

(b) $Q(x, y, z) = T(y, z) L(x)$.
Since a nonnegative bid diagonal matrix is totally positive, to prove Proposition 1.3 it suffices to show that $T(y,z)$ is coefficientwise totally positive. This turns out to be surprisingly difficult; but it is contained within a tridiagonal special case of [1, Theorem C.1], which asserts the coefficientwise total positivity of a class of quadridiagonal lower-Hessenberg matrices, defined as follows. Let

$$ \mathbf{P} \stackrel{\text{def}}{=} L_1 L_2 U + L_1 D_1 + L_2 D_2 \quad (3.3) $$

where

$$ L_1 = \alpha I + \zeta L, \quad L_2 = \beta I + \eta L \quad (3.4) $$

and

* $L$ is the lower-bidiagonal matrix with the sequence $a_0, a_1, \dots$ on the diagonal, the sequence $b_1, b_2, \dots$ on the subdiagonal, and zeroes elsewhere;

* $U$ is the upper-bidiagonal matrix with the sequence $c_1, c_2, \dots$ on the superdiagonal, the sequence $d_0, d_1, \dots$ on the diagonal, and zeroes elsewhere;

* $D_1$ is the diagonal matrix with entries $e_0, e_1, \dots$;

* $D_2$ is the diagonal matrix with entries $f_0, f_1, \dots$;

and $\alpha, \beta, \zeta, \eta, \mathbf{a} = (a_n)_{n \ge 0}, \mathbf{b} = (b_n)_{n \ge 1}, \mathbf{c} = (c_n)_{n \ge 1}, \mathbf{d} = (d_n)_{n \ge 0}, \mathbf{e} = (e_n)_{n \ge 0}, \mathbf{f} = (f_n)_{n \ge 0}$ are all indeterminates. We then have:

**Theorem 3.2.** [1, Theorem C.1] *The matrix P defined by (3.3)/(3.4) is totally positive, coefficientwise in the indeterminates $\alpha, \beta, \zeta, \eta, \mathbf{a}, \mathbf{b}, \mathbf{c}, \mathbf{d}, \mathbf{e}, \mathbf{f}$.*

The quadridiagonal matrices $P$ and $Q$ of Proposition 1.2 are not (as far as we can tell) covered by Theorem 3.2, but the tridiagonal matrix $T$ is. Taking $\mathbf{d} = \mathbf{0}$ makes the matrix $\mathbf{P}$ tridiagonal; the further specializations

$$ \begin{gathered} \alpha = \beta = 1, \quad \zeta = y^2, \quad \eta = z^2, \\ a_n = 0, \quad b_n = (2n)(2n+1), \quad c_n = 1, \quad e_n = (2n+1)z^2, \quad f_n = (2n+1)y^2 \end{gathered} \quad (3.5) $$

then yield $T$.

**SKETCH OF PROOF OF THEOREM 3.2.** We first let $\mathbf{Q} = \mathbf{P}|_{\mathbf{f}=0} = L_1(L_2U + D_1)$. Next let $\mathbf{p}_k, \mathbf{q}_k$ and $\mathbf{\ell}_k$ denote the $k$-th columns of the matrices $\mathbf{P}, \mathbf{Q}$ and $L_2$, respectively.

We use induction to prove that for any fixed pair of integers $0 \le k \le m+1$, the matrix $(\mathbf{q}_0, \mathbf{q}_1, \dots, \mathbf{q}_{k-1}, \mathbf{p}_k, \dots, \mathbf{p}_m)$ is totally positive. We do this in the following steps:

* Step 1: The matrix $\mathbf{Q}$ is totally positive: this follows from the fact that bid diagonal matrices with non-negative entries are totally positive, together with the tridiagonal comparison theorem [19] [22, Proposition 3.1] [1, Proposition 2.6]. This establishes the base case $k = m+1$ of our induction.
* Step 2: The matrix $(q_0, \dots, q_{k-1}, \ell_k)$ is totally positive: When $d = 0$, this can easily be shown by direct consideration of the minors. The general case is significantly more difficult.

* Step 3: If the matrix $(p_{k+1}, \dots, p_m)$ is totally positive, then so is $(q_0, \dots, q_{k-1}, \ell_k, p_{k+1}, \dots, p_m)$: We prove this by induction where the base case $k = m$ is Step 2. When $k < m$, we let $t_{k+1} = (p_{k+1}|_{b_{k+1}=0})|_{f_{k+1} \to \zeta b_{k+1} c_{k+1}}$ and let $(t_{n,k+1})_{n \ge 0} = t_{k+1}$. Next let $\tilde{p}_{k+1}$ be the same as $t_{k+1}$ except that the entry $t_{k,k+1}$ is made equal to 0. It is not difficult to show that the matrix $(\tilde{p}_{k+1}, p_{k+2}, \dots, p_m)$ is totally positive. Next, we notice that the matrix $S = (q_0, \dots, q_{k-1}, \ell_k, \tilde{p}_{k+1}, p_{k+2}, \dots, p_m)$ consists of two totally positive blocks which overlap in a single row. Using [11, p. 398], we get that $S$ is totally positive. Finally, we obtain the desired result by right-multiplying $S$ with the upper-bidiagonal matrix that has 1 on the diagonal, $(\alpha + \zeta a_k) c_{k+1}$ in position $(k, k+1)$ and zeros elsewhere.

* Step 4 (Induction step): If the matrix $(q_0, q_1, \dots, q_k, p_{k+1}, \dots, p_m)$ is totally positive, then so is $(q_0, q_1, \dots, q_{k-1}, p_k, \dots, p_m)$: This follows from noticing $p_k = q_k + f_k \ell_k$ and then using the column-linearity of determinants.

The details of this proof can be found in [1, Appendix C]. □

# 4 Remarks

In a longer version of this work, we intend to treat the following generalizations:

1) We can define Schett polynomials in any number of variables, not just three. Fix an integer $m \ge 1$, and define the Schett operator in $m+1$ variables $x_0, x_1, \dots, x_m$ by

$$ \mathcal{D}_{m+1} \stackrel{\text{def}}{=} \sum_{i=0}^{m} \left( \prod_{\substack{0 \le j \le m \\ j \ne i}} x_j \right) \frac{\partial}{\partial x_i} . \qquad (4.1) $$

Then define the *Schett polynomials in $m+1$ variables* initialized in $x_0$ by

$$ X_n^{[m+1]}(x_0, x_1, \dots, x_m) \stackrel{\text{def}}{=} (\mathcal{D}_{m+1})^n x_0 . \qquad (4.2) $$

These polynomials are clearly symmetric in $x_1, \dots, x_m$. Since $x_0$ plays a special role, we write $x = x_0$ and $y = (y_1, \dots, y_m) = (x_1, \dots, x_m)$. It turns out that if we define $a_i = 1/y_i$ and make an appropriate rescaling, then the Schett polynomial in $m$ variables is obtained from the one in $m+1$ variables by specializing to $a_m = 0$. This allows us to take $m \to \infty$ and define a symmetric-function generalization of the Schett polynomials.
For this generalization we have obtained analogues of Propositions 1.2, 1.4 and 1.5 for the case $x = 0$, but the coefficientwise total positivity of this production matrix is at present a conjecture. Furthermore, we have not yet been able to find a production matrix for the case $x \neq 0$.

2) Schett [17, Theorem III/1] showed that $X_n(1,1,1) = n!$. It was therefore natural to interpret $X_n(x,y,z)$ as enumerating permutations of $[n] \stackrel{\text{def}}{=} \{1, \dots, n\}$ with respect to some bivariate statistic, and Dumont [5] identified the statistic: he showed that

$$X_n(x,y,z) = \begin{cases} x D_n(x,y,z) & \text{for } n \text{ even} \\ y D_n(x,y,z) & \text{for } n \text{ odd} \end{cases} \qquad (4.3)$$

where

$$D_n(x,y,z) \stackrel{\text{def}}{=} \sum_{\sigma \in \mathfrak{S}_n} (x^2)^{\text{cpeakodd}(\sigma)} (y^2)^{\text{cpeakeven}(\sigma)} z^{\text{cdrise}(\sigma)} + \text{cdfall}(\sigma) + \text{fix}(\sigma); \qquad (4.4)$$

here these statistics count, respectively, odd cycle peaks, even cycle peaks, cycle double rises, cycle double falls, and fixed points in the permutation $\sigma$. (We remark that the $y \leftrightarrow z$ symmetry is quite mysterious in this combinatorial interpretation.) A different combinatorial interpretation of the Schett polynomials, in terms of vertices of even and odd degree at even and odd levels of an increasing tree, was given by Lin and Ma [12, Theorem 1.6]. It will be interesting to find combinatorial interpretations for the higher-order Schett polynomials.

3) We can generalize (4.3)/(4.4) by introducing a factor $\lambda^{\text{cyc}(\sigma)}$, where $\text{cyc}(\sigma)$ denotes the number of cycles in the permutation $\sigma$. For these *Schett polynomials with cycle-counting*, we have found generalizations of (1.3)/(1.4) that empirically generate the correct zeroth-column sequences, but we have not yet been able to *prove* that they do so. On the other hand, we *have* proven the corresponding generalization of Proposition 1.3: these production matrices are totally positive, coefficientwise in $x, y, z, \lambda$, as a consequence of the $d=0$ case of Theorem 3.2.

## References

[1] B. Deb, A. Dyachenko, M. Pétréolle, and A. D. Sokal. “Lattice paths and branched continued fractions, III: Generalizations of the Laguerre, rook and Lah polynomials”. 2023. arXiv:23??.?????

[2] B. Deb and A. D. Sokal. “Continued fractions for cycle-alternating permutations”. 2023. arXiv:2304.06545. Link.

[3] E. Deutsch, L. Ferrari, and S. Rinaldi. “Production matrices”. *Advances in Applied Mathematics* 34.1 (2005), pp. 101–122. doi.

[4] E. Deutsch, L. Ferrari, and S. Rinaldi. “Production matrices and Riordan arrays”. *Annals of Combinatorics* 13.1 (2009), pp. 65–85. doi.
[5] D. Dumont. "A combinatorial interpretation for the Schett recurrence on the Jacobian elliptic functions". Mathematics of Computation 33 (1979), pp. 1293-1297. doi.

[6] D. Dumont. "Une approche combinatoire des fonctions elliptiques de Jacobi". Advances in Mathematics 41.1 (1981), pp. 1-39. doi.

[7] S. M. Fallat and C. R. Johnson. Totally Nonnegative Matrices. Princeton University Press, 2011.

[8] P. Flajolet and J. Françon. "Elliptic functions, continued fractions and doubled permutations". European Journal of Combinatorics 10.3 (1989), pp. 235-241. doi.

[9] F. R. Gantmakher and M. G. Krein. "Sur les matrices complètement non négatives et oscillatoires". Compositio Mathematica 4 (1937), pp. 445-476. Link.

[10] F. R. Gantmakher and M. G. Krein. Oscillation Matrices and Kernels and Small Vibrations of Mechanical Systems. Based on the second Russian edition, 1950. AMS Chelsea Pub., 2002.

[11] S. Karlin. Total Positivity. Stanford University Press, 1968.

[12] Z. Lin and J. Ma. "A symmetry on weakly increasing trees and multiset Schett polynomials". 2021. arXiv:2104.10539. Link.

[13] S. C. Milne. "Infinite families of exact sums of squares formulas, Jacobi elliptic functions, continued fractions, and Schur functions". Ramanujan Journal 6 (2002), pp. 7-149. doi.

[14] M. Pétréolle, A. D. Sokal, and B.-X. Zhu. "Lattice paths and branched continued fractions: An infinite sequence of generalizations of the Stieltjes-Rogers and Thron-Rogers polynomials, with coefficientwise Hankel-total positivity". To appear in the *Memoirs of the American Mathematical Society*. 2018. arXiv:1807.03271. Link.

[15] A. Pinkus. Totally Positive Matrices. Cambridge University Press, 2010.

[16] L. J. Rogers. "On the representation of certain asymptotic series as convergent continued fractions". Proceedings of the London Mathematical Society (series 2) 4.1 (1907), pp. 72-89. doi.

[17] A. Schett. "Properties of the Taylor series expansion coefficients of the Jacobian elliptic functions". Mathematics of Computation 30 (1976), pp. 143-147. doi.

[18] A. Schett. "Recurrence formula of the Taylor series expansion coefficients of the Jacobian elliptic functions". Mathematics of Computation 31 (1977), pp. 1003-1005. doi.

[19] A. D. Sokal. "Coefficientwise Hankel-total positivity". Monograph in preparation.

[20] A. D. Sokal. "Total positivity of some polynomial matrices that enumerate labeled trees and forests I: Forests of rooted labeled trees". Monatshefte für Mathematik 200.2 (2022), pp. 389-452. doi.

[21] T. J. Stieltjes. "Sur la réduction en fraction continue d'une série procédant selon les puisances descendantes d'une variable". Annales de la Faculté des Sciences de Toulouse : Mathématiques (1889), H1-H17. Link.

[22] B.-X. Zhu. "Stieltjes moment properties and continued fractions from combinatorial triangles". Advances in Applied Mathematics 130 (2021), 102232. doi.