# Selberg integrals and Catalan-Pfaffian Hankel determinants

Masao Ishikawa<fnref n="1" />* and Jiang Zeng<fnref n="2" />†

**Abstract.** In our previous works "Pfaffian decomposition and a Pfaffian analogue of $q$-Catalan Hankel determinants" (by M.Ishikawa, H. Tagawa and J. Zeng, *J. Combin. Theory Ser. A*, **120**, 2013, 1263–1284) we have proposed several ways to evaluate certain Catalan-Hankel Pfaffians and also formulated several conjectures. In this work we propose a new approach to compute these Catalan-Hankel Pfaffians using Selberg's integral as well as their $q$-analogues. In particular, this approach permits us to settle most of the conjectures in our previous paper.

**Résumé.** Dans nos travaux précédents "Pfaffian decomposition and a Pfaffian analogue of $q$-Catalan Hankel determinants" (by M.Ishikawa, H. Tagawa and J. Zeng, em J. Combin. Theory Ser. A, **120**, 2013, 1263–1284) nous avons proposé plusieurs méthodes pour évaluer certains Catalan-Pfaffian déterminants de Hankel et avons aussi formulé plusieurs conjectures. Dans ce travail nous proposons une nouvelle approche pour calculer ces Catalan-Pfaffian déterminants de Hankel en utilisant l'intégrale de Selberg ainsi que leurs $q$-analogues. En particulier, cette approche nous permet de confirmer la plus part de nos conjectures précédentes.

**Keywords:** Hankel determinants, Pfaffians, hyperpfaffians, Orthogonal polynomials,

## 1 Introduction

In Ishikawa et al. (2013) the three authors presented several open problems concerning Pfaffian analogue of several Hankel determinants. Ishikawa and Koutschan (2012) partially settled Conjecture 6.2 in Ishikawa et al. (2013) by a computer proof using Zeilberger’s Holonomic Ansatz for Pfaffians. In this paper we settle most of the conjectures except Conjecture 6.3 in Ishikawa et al. (2013). Furthermore we give another proof of Theorem 3.1 in Ishikawa et al. (2013) by reducing it to the $k = 2$ case of Askey’s $q$-Selberg’s integral formula via de Bruijn’s formula. We believe that our new proof gives a simpler and essentially insightful method to Pfaffian analogues of several Hankel determinants.

We say a matrix $A = (a_{i,j})_{i,j \ge 1}$ (or $A = (a_{i,j})_{1 \le i,j \le n}$) is skew-symmetric if it satisfies $a_{j,i} = -a_{i,j}$ for $i, j \ge 1$. A skew-symmetric matrix is completely determined by its upper triangular entries so that

<footnote n="1">¹Department of Mathematics, University of the Ryukyus, Nishihara, Okinawa 901-0213, Japan,</footnote>

<footnote n="2">²Institut Camille Jordan, Université Claude Bernard Lyon 1, France,</footnote>

<footnote>*Email: ishikawa@edu.u-ryukyu.ac.jp, Partially supported by Grant-in-Aid for Scientific Research (C) 25400018.</footnote>

<footnote>†Email: zeng@math.univ-lyon1.fr.</footnote>
we identify a skew-symmetric matrix $A = (a_{i,j})_{i,j \ge 1}$ (resp. $A = (a_{i,j})_{1 \le i,j \le n}$) with the upper triangular matrix $A = (a_{i,j})_{1 \le i < j}$ (resp. $A = (a_{i,j})_{1 \le i < j \le n}$). Let

$$ \mathfrak{E}_{2n} = \left\{ \begin{pmatrix} 1 & 2 & \cdots & 2n \\ \sigma_1 & \sigma_2 & \cdots & \sigma_{2n} \end{pmatrix} \in \mathfrak{S}_{2n} \middle| \sigma_{2i-1} < \sigma_{2i} \text{ for } i = 1, \dots, n \right\} $$

For instance $\mathfrak{E}_4$ has the following 6 permutations: (1, 2, 3, 4), (1, 3, 2, 4), (1, 4, 2, 3), (2, 3, 1, 4), (2, 4, 1, 3), (2, 3, 1, 4). This implies

$$ \mathrm{Pf}(a_{ij})_{1 \le i,j \le 4} = a_{12}a_{34} - a_{13}a_{24} + a_{14}a_{23}. $$

A hyperpfaffian is a generalization of a Pfaffian, and first defined by Barvinok Barvinok (1995). Here we adopt the definition by Matsumoto Matsumoto (2008), which is a special case of the definition by Barvinok.

**Definition 1.1** Let $m$ and $n$ be positive integers, and let $B = (B(i_1, \dots, i_{2m}))_{1 \le i_1 < \dots < i_{2m} \le 2n}$ be an array which satisfies

$$ B(i_{\tau_1(1)}, i_{\tau_1(2)}, \dots, i_{\tau_m(2m-1)}, i_{\tau_m(2m)}) = \operatorname{sgn}(\tau_1) \cdots \operatorname{sgn}(\tau_m) B(i_1, \dots, i_{2m}) $$

for all $(\tau_1, \dots, \tau_m) \in (\mathfrak{S}_2)^m$. The hyperpfaffian $\mathrm{Pf}^{[2m]}(B)$ of $B$ is defined by

$$ \mathrm{Pf}^{[2m]}(B) = \frac{1}{n!} \sum_{\sigma_1, \dots, \sigma_m \in \mathfrak{E}_{2n}} \operatorname{sgn}(\sigma_1 \cdots \sigma_m) \times \prod_{i=1}^{n} B(\sigma_1(2i-1), \sigma_1(2i), \dots, \sigma_m(2i-1), \sigma_m(2i)). $$

Throughout this paper we use the standard notation for $q$-series (see Andrews et al. (2000); Gasper and Rahman (2004)):

$$ (a; q)_{\infty} = \prod_{k=0}^{\infty} (1 - aq^k), \qquad (a; q)_n = \frac{(a; q)_{\infty}}{(aq^n; q)_{\infty}} $$

for any integer $n$. Usually $(a; q)_n$ is called the *q-shifted factorial*, and we frequently use the compact notation:

$$ (a_1, a_2, \dots, a_r; q)_{\infty} = (a_1; q)_{\infty} (a_2; q)_{\infty} \cdots (a_r; q)_{\infty}, $$

$$ (a_1, a_2, \dots, a_r; q)_n = (a_1; q)_n (a_2; q)_n \cdots (a_r; q)_n. $$

The $r+1\phi_r$ basic hypergeometric series is defined by

$$ r+1\phi_r \begin{bmatrix} a_1, a_2, \dots, a_{r+1}; q, z \\ b_1, \dots, b_r \end{bmatrix} = \sum_{n=0}^{\infty} \frac{(a_1, a_2, \dots, a_{r+1}; q)_n}{(q, b_1, \dots, b_r; q)_n} z^n. $$
## 2 Minor summation formula of Pfaffians

Let $A = (a_{ij})_{i,j \ge 1}$ be an array. When $I = \{i_1, \dots, i_r\}$ is a row index set, $J = \{j_1, \dots, j_r\}$ is a column idenx set, let $A_{I}^{I} = A_{j_1, \dots, j_r}^{i_1, \dots, i_r}$ denote the $r \times r$ minor of $A$ obtained by choosing the rows in $I$ and the columns in $J$. We use the notation $[n] = \{1, \dots, n\}$ for a positive integer $n$. For example, if $A = (a_{ij})_{i,j \ge 1}$, then we have

$$A_{2,3,5}^{1,2,4} = \begin{pmatrix} a_{12} & a_{13} & a_{15} \\ a_{22} & a_{23} & a_{25} \\ a_{42} & a_{43} & a_{45} \end{pmatrix}.$$

Further, if $A$ is a skew-symmetric matrix, then we write $A_I$ for $A_I^I$ in short. For later use we cite the minor summation formula of Pfaffians here:

**Theorem 2.1** (Ishikawa and Wakayama (1995, 2006)) Let $n \le N$ be positive integers and assume $n$ is even. Let $H = (h_{i,j})_{1 \le i \le n, 1 \le j \le N}$ be an $n \times N$ rectangular matrix, and let $A = (\alpha_{i,j})_{1 \le i,j \le N}$ be a skew symmetric matrix of size $N$. Then we have

$$\sum_{\substack{I \subseteq [N] \\ \#I = n}} \mathrm{Pf}(A_I) \det(H_I^{[n]}) = \mathrm{Pf}(Q), \qquad (2.1)$$

where the skew symmetric matrix $Q$ is defined by $Q = (Q_{i,j}) = HAH^T$ whose entries may be written in the form

$$Q_{i,j} = \sum_{1 \le k < l \le N} \alpha_{k,l} \det(H_{k,l}^{i,j}), \quad (1 \le i, j \le n). \qquad (2.2)$$

When $n$ is odd, we can immediately derive a similar formula from the case where $n$ is even. Matsumoto Matsumoto (2008) gave the following hyperpfaffian analogue of Theorem 2.1.

**Theorem 2.2** (Matsumoto (2008)) Let $m, n$ and $N$ be positive integers such that $2n \le N$. Let $H(s) = (h_{ij}(s))_{1 \le i \le 2n, 1 \le j \le N}$ be $2n \times N$ rectangular matrices for $1 \le s \le 2m$, and let $A = (\alpha_{i,j})_{1 \le i,j \le N}$ be a skew symmetric matrix of size $N$. Then we have

$$\sum_{\#I=2n} \mathrm{Pf}(A_I) \prod_{s=1}^{m} \det \left( H(s)_{I}^{[2n]} \right) = \mathrm{Pf}^{[2m]}(Q),$$

where the array $Q = (Q_{i_1, \dots, i_{2m}})_{1 \le i_1, \dots, i_{2m} \le 2n}$ is defined by

$$Q_{i_1, \dots, i_{2m}} = \sum_{1 \le k < l \le N} a_{k,l} \prod_{s=1}^{m} \det(H(s)_{k,l}^{i_{2s}-1, i_{2s}}).$$

We cite the following proposition from Ishikawa and Wakayama (1995, 2006) to compute certain Pfaffians in the following sections.

**Proposition 2.3** Let $\{\alpha_k\}_{k \ge 1}$ be any sequence, and let $n$ be a positive integer. Let $B = (b_{i,j})_{i,j \ge 1}$ be the skew-symmetric matrix defined by

$$b_{i,j} = \begin{cases} \alpha_i & \text{if } j = i + 1 \text{ for } i \ge 1, \\ -\alpha_j & \text{if } i = j + 1 \text{ for } j \ge 1, \\ 0 & \text{otherwise.} \end{cases} \qquad (2.3)$$
If $I = (i_1, \dots, i_{2n})$ is an index set such that $1 \le i_1 < \dots < i_{2n}$, then

$$ \mathrm{Pf}(B_I) = \begin{cases} \prod_{k=1}^{n} \alpha_{i_{2k}-1} & \text{if } i_{2k} = i_{2k-1} + 1 \text{ for } k = 1, \dots, n, \\ 0 & \text{otherwise.} \end{cases} \qquad (2.4) $$

### 3 De Bruijn's formula and Hankel Pfaffians

The $q$-Jackson integral from 0 to $a$ is defined by

$$ \int_0^a f(x) \, d_q x = (1-q)a \sum_{n=0}^{\infty} f(aq^n)q^n, $$

which is absolutely convergent when $|q| < 1$. More generally, the $q$-integral on $[a, b]$ is defined by

$$ \int_a^b f(x) \, d_q x = \int_0^b f(x) \, d_q x - \int_0^a f(x) \, d_q x. $$

Let $\omega$ be the measure on an interval $[0, a]$ defined by a given weight function $w(x)$ such that $\omega(d_q x) = w(x) d_q x$. The moment $\mu_n(q)$ of the measure $\omega$ is defined by

$$ \mu_n(q) = \int_a^b x^n \omega(d_q x). $$

A sequence of polynomials $p_n(x)$ ($n = 0, 1, \dots$) is called an *orthogonal polynomial sequence with respect to* the measure $\omega$ if it satisfies the following two conditions:

(i) $\deg p_n(x) = n$,

(ii) $\int_a^b p_m(x)p_n(x)\omega(d_q x) = K_n\delta_{m,n}$ holds for any integers $m, n \ge 0$, where $K_n > 0$ is a constant.

The following proposition is usually called de Bruijn's formula:

**Proposition 3.1** Let $n$ be a positive integer, and let $\phi_i(x)$ and $\psi_i(x)$ be functions on $[0, a]$ for $1 \le i \le 2n$. Then

$$ \int \dotsi \int_0 \int_{x_1 < \dotsi < x_n \le a} \det (\phi_i(x_j) |\psi_i(x_j)|) \, d_q \mu(x_1) \dotsi d_q \mu(x_n) = \mathrm{Pf} (Q_{i,j})_{1 \le i,j \le 2n}, \quad (3.1) $$

where

$$ Q_{i,j} = \int_0^a \{\phi_i(x)\psi_j(x) - \phi_j(x)\psi_i(x)\} \, d_q \mu(x) \quad (3.2) $$

and $(\phi_i(x_j) |\psi_i(x_j)|)$ denotes the $2n \times 2n$ matrix whose $i$th row is

$$ (\phi_i(x_1), \psi_i(x_1), \dots, \phi_i(x_n), \psi_i(x_n)) $$

for $1 \le i \le 2n$.
In fact, Proposition 3.1 is a corollary of the following proposition, which is a hyperpfaffian version of de Bruijn’s formula.

**Proposition 3.2** Let $m$ and $n$ be positive integers. Let $\phi_{s,i}(x)$ and $\psi_{s,i}(x)$ be functions on $[0, a]$ for $1 \le i \le 2n$, $1 \le s \le m$. Then we have

$$ \int \dotsi \int_{0 \le x_1 < \dotsi < x_n \le a} \prod_{s=1}^{m} \det (\phi_{s,i}(x_j) | \psi_{s,i}(x_j)) \omega(d_q x) \\ = \mathrm{Pf}^{[2m]} (Q_{i_1, \dotsi, i_{2m}})_{1 \le i_1, \dotsi, i_{2m} \le 2n}, \qquad (3.3) $$

where

$$ Q_{i_1, \dotsi, i_{2m}} = \int_0^a \prod_{s=1}^m \{ \phi_{s, i_{2s-1}}(x) \psi_{s, i_{2s}}(x) - \phi_{s, i_{2s}}(x) \psi_{s, i_{2s-1}}(x) \} \omega(d_q x) \qquad (3.4) $$

for $1 \le i_1, \dotsi, i_{2m} \le 2n$.

**Corollary 3.3** Let $\omega(d_q x) = w(x)d_q x$ be a measure on $[0, a]$, and let $\mu_i = \int_0^a x^i \omega(d_q x)$ be the $i$th moment of $\omega$. Then we have

$$ \mathrm{Pf} \left( (q^{i-1} - q^{j-1}) \mu_{i+j+r-2} \right)_{1 \le i < j \le 2n} \\ = \frac{q^{\binom{n}{2}} (1-q)^n}{n!} \int_{[0,a]^n} \prod_i x_i^{r+1} \prod_{i<j} (x_i - x_j)^2 \prod_{i<j} (qx_i - x_j)(x_i - qx_j) \omega(d_q x). \qquad (3.5) $$

**Proof.** If one sets $\varphi_i(x) = q^{i-1}x^{i-1}$ and $\psi_i(x) = x^{i+r-1}$ in (3.2), then one obtains

$$ Q_{i,j} = (q^{i-1} - q^{j-1}) \int_0^1 x^{i+j+r-2} \omega(d_q x) = (q^{i-1} - q^{j-1}) \mu_{i+j+r-2}. $$

On the other hand, if one substitutes $\varphi_i(x)$ and $\psi_i(x)$ as above in (3.1), then one also gets

$$ \det (\phi_i(x_j) | \psi_i(x_j))_{1 \le i \le 2n, 1 \le j \le n} = \det (q^{i-1} x_j^{i-1} | x_j^{i-1})_{1 \le i \le 2n, 1 \le j \le n} \\ = q^{\binom{n}{2}} (1-q)^n (x_1 \dotsi x_n)^{r+1} \prod_{i<j} (x_i - x_j)^2 \prod_{i<j} (qx_i - x_j)(x_i - qx_j), $$

by using the Vandermonde determinant $\det(a_j^{i-1}) = \prod_{i<j} (a_j - a_i)$. Hence one concludes that

$$ \mathrm{Pf} \left( (q^{i-1} - q^{j-1}) \mu_{i+j+r-2} \right)_{1 \le i < j \le 2n} \\ = q^{\binom{n}{2}} (1-q)^n \int \dotsi \int_{0 \le x_1 < \dotsi < x_n \le a} \prod_i x_i^{r+1} \prod_{i<j} (x_i - x_j)^2 \\ \times \prod_{i<j} (qx_i - x_j)(x_i - qx_j) \omega(d_q x) $$

from (3.1). One sees that (3.5) is an easy consequence of this identity.
If we let $q \to 1$ in Corollary 3.3, then we obtain the following corollary:

**Corollary 3.4** Let $\psi(dx) = \psi'(x)dx$ be a measure on an interval $[0, a]$, and let $\mu_i = \int_0^a x^i \psi(dx)$ denote the $i$th moment. Then we have

$$ \mathrm{Pf}((j-i)\mu_{i+j+r-2})_{1\le i<j\le 2n} = \frac{1}{n!} \int_{[0,a]^n} \prod_i x_i^{r+1} \prod_{i<j} (x_i - x_j)^4 \psi(dx). \quad (3.6) $$

If we set $\phi_{s,i}(x) = ix^{i-1}$ and $\psi_{s,i}(x) = x^{i+r_s-1}$ in Proposition 3.2 as in the proof of Corollary 3.3 then we obtain the following corollary:

**Corollary 3.5** Let $\psi(dx) = \psi'(x)dx$ be a measure on an interval $[0, a]$, and let $\mu_i = \int_0^a x^i \psi(dx)$ denote the $i$th moment. Then we have

$$ \begin{aligned} & \mathrm{Pf}^{[2m]}\left(\prod_{s=1}^{m}(i_{2s}-i_{2s-1}) \cdot \mu_{i_1+\cdots+i_{2m}+r}\right)_{0\le i<j\le 2n-1} \\ & = \frac{1}{n!} \int_{[a,b]^n} \prod_i x_i^{r+m} \prod_{i<j} (x_i - x_j)^{4m} \psi(dx). \end{aligned} \quad (3.7) $$

# 4 Selberg-Askey integral formula

In this section we give a scketch of another proof of (Ishikawa et al., 2013, Theorem 3.1).

**Theorem 4.1** For integers $n \ge 1$ and $r \ge 0$, we have

$$ \begin{aligned} & \mathrm{Pf}\left( (q^{i-1} - q^{j-1}) \frac{(aq; q)_{i+j+r-2}}{(abq^2; q)_{i+j+r-2}} \right)_{1\le i,j\le 2n} \\ & = a^{n(n-1)} q^{n(n-1)(4n+1)/3+n(n-1)r} \prod_{k=1}^{n-1} (bq; q)_{2k} \prod_{k=1}^{n} \frac{(q; q)_{2k-1}(aq; q)_{2k+r-1}}{(abq^2; q)_{2(k+n)+r-3}}. \end{aligned} \quad (4.1) $$

Let $\omega$ be the measure on $[0, 1]$ defined by

$$ \int_0^1 f(x) \omega(d_q x) = \frac{(aq; q)_\infty}{(abq^2; q)_\infty} \sum_{k=0}^\infty \frac{(bq; q)_k}{(q; q)_k} (aq)^k f(q^k) \quad (4.2) $$

which implies

$$ w(x) = \frac{1}{1-q} \cdot \frac{(aq, bq; q)_\infty}{(abq^2, q; q)_\infty} \cdot \frac{(qx; q)_\infty}{(bqx; q)_\infty} x^{\alpha+1}, $$

where $a = q^\alpha$. The $n$th moment is given by

$$ \mu_n = \int_0^1 x^n \omega(d_q x) = \frac{(aq; q)_n}{(abq^2; q)_n} \quad (n = 0, 1, 2, \dots), \quad (4.3) $$

which is the moment of the Little q-Jacobi polynomials Gasper and Rahman (2004); Koekoek et al. (2010)

$$ p_n(x; a, b; q) = \frac{(aq; q)_n}{(abq^{n+1}; q)_n} (-1)^n q^{\binom{n}{2}} {}_2\phi_1 \left[ \begin{array}{c} q^{-n}, abq^{n+1} \\ aq \\ \hline ; q, xq \end{array} \right]. \quad (4.4) $$
The q-gamma function is defined on $\mathbb{C} \setminus \mathbb{Z}_{<0}$ by

$$ \Gamma_q(a) = \frac{(q; q)_{\infty}}{(q^a; q)_{\infty}} (1 - q)^{1-a} $$

First we obtain

$$ \begin{aligned} & \mathrm{Pf}((q^{i-1} - q^{j-1}) \frac{(aq; q)_{i+j+r-2}}{(abq^2; q)_{i+j+r-2}})_{1 \le i < j \le 2n} \\ &= C \int_{[0,1]^n} \prod_{i<j} \prod_{l=0}^{1} (x_i - q^l x_j) (x_i - q^{-l} x_j) \prod_i x_i^{\alpha+r+1} \frac{(qx_i; q)_{\infty}}{(bqx_i; q)_{\infty}} d_q x \end{aligned} $$

from (3.5) where $C = \frac{q^{n(n-1)}}{n!} \left\{ \frac{(aq,bq;q)_{\infty}}{(abq^2,q;q)_{\infty}} \right\}^n$. The following identity was conjectured by (Askey, 1980, Conjecture 1), and proved by Habsieger Habsieger (1987, 1988) and Kadell (Kadell, 1988, Theorem 2; $l=m=0$) independently:

$$ \int_{[0,1]^n} \prod_{i<j} t_i^{2k} (q^{1-k} t_j / t_i; q)_{2k} \prod_{i=1}^{n} t_i^{x_i-1} \frac{(t_i q; q)_{\infty}}{(t_i q^y; q)_{\infty}} d_q t = q^{kx(\frac{n}{2})+2k^2(\frac{n}{3})} S_n(x, y; q), \quad (4.5) $$

where

$$ S_n(x, y; q) = \prod_{j=1}^{n} \frac{\Gamma_q(x + (j-1)k)\Gamma_q(y + (j-1)k)\Gamma_q(jk + 1)}{\Gamma_q(x + y + (n + j - 2)k)\Gamma_q(k + 1)}. \quad (4.6) $$

Habsieger (1987) showed that (4.5) implies the following variation”

**Theorem 4.2 (Habsieger (1987)) (4.5) implies**

$$ \begin{aligned} & \int_{[0,1]^n} \prod_{i<j} \prod_{l=0}^{k-1} (t_j - q^l t_i) (t_j - q^{-l} t_i) \prod_i t_i^{x_i-1} \frac{(t_i q; q)_{\infty}}{(t_i q^y; q)_{\infty}} d_q t \\ &= n! q^{kx(\frac{n}{2})+2k^2(\frac{n}{3})} \frac{S_n(x, y; q)}{\Gamma_{q^k}(n+1)}. \end{aligned} \quad (4.7) $$

If one combines (3.5) with this result then one sees that (4.1) follows from (4.7) by using

$$ \frac{S_n(x, y; q)}{\Gamma_{q^k}(n+1)} = \frac{(1-q)^n}{(q; q)_{k-1}^n} \prod_{j=1}^{n} \frac{(q^{x+y+(n+j-2)k}, q; q)_{\infty}(q; q)_{jk-1}}{(q^{x+(j-1)k}, q^{y+(j-1)k}; q)_{\infty}}. $$

# 5 Al-Salam and Carlitz I,II

In this section we use the standard q-exponential functions:

$$ e_q(x) = \sum_{n=0}^{\infty} \frac{x^n}{(q; q)_n} = \frac{1}{(x; q)_{\infty}}, \qquad E_q(x) = \sum_{n=0}^{\infty} \frac{q^{\frac{n(n-1)}{2}} x^n}{(q; q)_n} = (-x; q)_{\infty}. $$
Al-Salam and Carlitz Al-Salam and Carlitz (1965); Chihara (1978) defined the sequences {$U_n^{(a)}(y; q)$} ($a < 0$) and {$V_n^{(a)}(x; q)$} of orthogonal polynomials by

$$
\begin{align*}
\rho_a(x; q)e_q(xy) &= \sum_{n=0}^{\infty} U_n^{(a)}(y; q) \frac{x^n}{(q; q)_n}, \\
\frac{1}{\rho_a(x; q)}E_q(-xy) &= \sum_{n=0}^{\infty} V_n^{(a)}(y; q) \frac{(-1)^n q^{\frac{n(n-1)}{2}} x^n}{(q; q)_n},
\end{align*}
$$

where

$$
\rho_a(x; q) = (x; q)_{\infty}(ax; q)_{\infty} = E_q(-x)E_q(-ax).
$$

These sequences {$U_n^{(a)}(y; q)$} and {$V_n^{(a)}(x; q)$} are called the Al-Salam and Carlitz I polynomials and the Al-Salam and Carlitz II polynomials, respectively. The orthogonality relations of these polynomials are given by

$$
\int_a^1 U_m^{(a)}(x;q) U_n^{(a)}(x;q) w_U^{(a)}(x;q) dq x = (1-q)(-a)^n q^{\frac{n(n-1)}{2}} (q;q)_n \delta_{m,n}, \quad (5.1)
$$

$$
\int_1^\infty V_m^{(a)}(x;q) V_n^{(a)}(x;q) w_V^{(a)}(x;q) dq x = (1-q) a^n q^{-n^2} (q;q)_n \delta_{m,n}, \quad (5.2)
$$

where the weight functions $w_U^{(a)}(x; q)$ and $w_V^{(a)}(x; q)$ are defined by

$$
\begin{align*}
w_U^{(a)}(x;q) &= \frac{(qx;q)_{\infty} \binom{qx}{a; q)}{(q;q)_{\infty} (aq;q)_{\infty} \binom{q}{a; q)}; \\
w_V^{(a)}(x;q) &= \frac{(q;q)_{\infty} (aq;q)_{\infty} \binom{q}{a; q)}{(x;q)'_{\infty} \binom{a}{a; q)}.
\end{align*}
$$

(See Al-Salam and Carlitz (1965); Chihara (1978).) Here $(x;q)'_{\infty}$ denotes the product except the term which equals 0. Note that these Jackson integrals are given by

$$
\int_a^1 f(x) d_q x = (1-q) \left\{ \sum_{n=0}^{\infty} f(q^n) q^n - a \sum_{n=0}^{\infty} f(aq^n) q^n \right\}, \\
\int_1^\infty f(x) d_q x = (1-q) \sum_{n=0}^\infty f(q^{-n}) q^{-n}.
$$

The *n*th moments of the above measures are given by

$$
\begin{align*}
\int_a^1 x^n w_U^{(a)}(x;q) d_q x &= (1-q) F_n^{(a)}(a;q), \\
\int_1^\infty x^n w_V^{(a)}(x;q) d_q x &= (1-q) G_n^{(a)}(a;q),
\end{align*}
$$
where

$$F_n^{(a)}(a; q) = \sum_{k=0}^{n} \begin{bmatrix} n \\ k \end{bmatrix}_q a^k,$$

$$G_n^{(a)}(a; q) = \sum_{k=0}^{n} \begin{bmatrix} n \\ k \end{bmatrix}_q a^{k(k-n)}.$$

Here $\begin{bmatrix} n \\ k \end{bmatrix}_q = \frac{(q;q)_n}{(q;q)_k (q;q)_{n-k}}$ denotes the q-binomial coefficient. The purpose of this section is to prove the following theorem in which (5.3) and (5.4) were stated in (Ishikawa et al., 2013, Conjecture 6.1). However, our conjectures in (Ishikawa et al., 2013, Conjecture 6.1) had some mistakes in the power of $q$, and they are corrected in the following theorem.

**Theorem 5.1** Let $F_n(a; q)$ and $G_n(a; q)$ be as above. Then we have

$$\mathrm{Pf}\left( (q^{i-1} - q^{j-1}) F_{i+j-3}(a; q) \right)_{1 \le i,j \le 2n} = a^{n(n-1)} q^{\frac{1}{6}n(n-1)(4n-5)} \prod_{k=1}^{n} (q; q)_{2k-1}, \quad (5.3)$$

$$\mathrm{Pf}\left( (q^{i-1} - q^{j-1}) F_{i+j-2}(a; q) \right)_{1 \le i,j \le 2n} = a^{n(n-1)} q^{\frac{1}{6}n(n-1)(4n+1)} \prod_{k=1}^{n} (q; q)_{2k-1} \sum_{k=0}^{n} q^{(n-k)(n-k-1)} \begin{bmatrix} n \\ k \end{bmatrix}_{q^2} a^k. \quad (5.4)$$

$$\mathrm{Pf}\left( (q^{i-1} - q^{j-1}) G_{i+j-3}(a; q) \right)_{1 \le i,j \le 2n} = a^{n(n-1)} q^{-\frac{n(n-1)(4n-5)}{3}} \prod_{k=1}^{n} (q; q)_{2k-1}, \quad (5.5)$$

$$\mathrm{Pf}\left( (q^{i-1} - q^{j-1}) G_{i+j-2}(a; q) \right)_{1 \le i,j \le 2n} = a^{n(n-1)} q^{-\frac{1}{3}n(n-1)(2n-1)} \prod_{k=1}^{n} (q; q)_{2k-1} \sum_{k=0}^{n} \begin{bmatrix} n \\ k \end{bmatrix}_{q^2} a^k. \quad (5.6)$$

In this section we prove this theorem. For that purpose we use

$$\mathrm{Pf}\left( (q^{i-1} - q^{j-1}) F_{i+j-r-2}^{(a)}(a; q) \right)_{1 \le i < j \le 2n} = \frac{1}{n!} q^{n(n-1)} (1-q)^n \\ \times \int_{[a,1]^n} \prod_{i<j} \prod_{l=0}^{1} (x_i - q^l x_j) (x_i - q^{-l} x_j) \prod_{i=1}^{n} x_i^{r+1} w_U^{(a)}(x_i; q) d_q \boldsymbol{x}, \quad (5.7)$$

$$\mathrm{Pf}\left( (q^{i-1} - q^{j-1}) G_{i+j+r-2}^{(a)}(a; q) \right)_{1 \le i < j \le 2n} = \frac{1}{n!} q^{n(n-1)} (1-q)^n \\ \times \int_{[1, \infty)^n} \prod_{i<j} \prod_{l=0}^{1} (x_i - q^l x_j) (x_i - q^{-l} x_j) \prod_{i=1}^{n} x_i^{r+1} w_V^{(a)}(x_i; q) d_q \boldsymbol{x}. \quad (5.8)$$

which is a consequence of (3.5). Here we only need the case where $r = -1, 0$. Next, let $\tau_i$ denote the $q$-shift operator in the $i$th variable, i.e.,

$$\tau_i f(x_1, \dots, x_n) = f(x_1, \dots, x_{i-1}, qx_i, x_{i+1}, \dots, x_n).$$
Let $M_1$ denote the Macdonald operator defined by

$$M_1 := \sum_{i=1}^{n} A_i(t)\tau_i, \quad A_i(t) := \prod_{\substack{j=1 \\ j \neq i}}^{n} \frac{tx_i - x_j}{x_i - x_j},$$

which acts on the ring of the symmetric polynomials of $n$ variables $x = (x_1, \dots, x_n)$. Further we set

$$E_k := \sum_{i=1}^{n} x^k A_i(t) \frac{\partial}{\partial q x_i}, \quad \frac{\partial}{\partial q x_i} := \frac{1 - \tau_i}{(1 - q)x_i},$$

and we let $\tilde{M}_1$ denote the operator obtained by replacing $q$ and $t$ by $q^{-1}$ and $t^{-1}$, respectively, in $M_1$. Let $U_{\lambda}^{(a)}(x; q, t)$ denote the symmetric polynomial in the variables $x = (x_1, \dots, x_n)$, which is defined by

$$\mathcal{H}U_{\lambda}^{(a)}(x; q, t) = \tilde{e}(\lambda)U_{\lambda}^{(a)}(x; q, t).$$

Here $\mathcal{H}$ denotes the linear operator defined by

$$\mathcal{H} = \tilde{M}_1 - (1 + a)[E_0, \tilde{M}_1] + a[E_0, [E_0, \tilde{M}_1]],$$

and $\tilde{e}(\lambda) = \sum_{i=1}^{n} q^{-\lambda_i} t^{-n+i}$. Define the symmetric polynomials $V_{\lambda}^{(a)}(x; q, t)$ by

$$V_{\lambda}^{(a)}(x; q, t) = U_{\lambda}^{(a)}(x; q^{-1}, t^{-1}).$$

Baker and Forrester (2000) proved

$$\int_{[a,1]^n} \Delta_k^2(\boldsymbol{x}) \prod_{i=1}^n w_U^{(a)}(x_i; q) d_q \boldsymbol{x} \\ = (1-q)^n (-a)^{\frac{kn(n-1)}{2}} q^{k^2 \binom{n}{3} - \frac{k(k-1)}{2} \binom{n}{2}} \prod_{i=1}^n \frac{(q;q)_{ki}}{(q;q)_k}, \qquad (5.9)$$

$$\int_{[1, \infty]^n} \Delta_k^2(\boldsymbol{x}) \prod_{i=1}^n w_V^{(a)}(x_i; q) \, d_q \boldsymbol{x} \\ = (1-q)^n a^{\frac{kn(n-1)}{2}} q^{-2k^2 \binom{n}{3} - k^2 \binom{n}{2}} \prod_{i=1}^n \frac{(q;q)_{ki}}{(q;q)_k}, \qquad (5.10)$$

where

$$\Delta_k^2(\boldsymbol{x}) = \prod_{i<j} \prod_{l=-k+1}^{k} (x_i - q^l x_j).$$

Further they proved the orthogonality relations

$$\int_{[a,1]^n} U_{\lambda}^{(a)}(\boldsymbol{x}; q, t) U_{\mu}^{(a)}(\boldsymbol{x}; q, t) \Delta_k^2(\boldsymbol{x}) \prod_{i=1}^n w_U^{(a)}(x_i; q) d_q \boldsymbol{x} = 0, \qquad (5.11)$$

$$\int_{[1, \infty]^n} V_{\lambda}^{(a)}(\boldsymbol{x}; q, t) V_{\mu}^{(a)}(\boldsymbol{x}; q, t) \Delta_k^2(\boldsymbol{x}) \prod_{i=1}^n w_V^{(a)}(x_i; q) d_q \boldsymbol{x} = 0, \qquad (5.12)$$
when $\lambda \neq \mu$. We can derive (5.3) from (5.7) and (5.9), and also (5.5) from (5.8) and (5.10). But, here we have no space to state the details. To prove (5.4) and (5.6), we use the $r=0$ case of (5.7) and (5.8), then expand the product $\prod_{i=1}^n x_i = e_n(x)$ by the symmetric polynomials $U_\lambda^{(a)}(x; q, t)$ or $V_\lambda^{(a)}(x; q, t)$, and use the orthogonality relations (5.11) or (5.12).

**Acknowledgement.** We are indebted to an anonymous reviewer of FPSAC'14 for providing insightful comments on the relation of our paper with Sinclair (2012) in which we may expect nice applications of our Pfaffian and hyperpfaffian identities.

## References

W. Al-Salam and L. Carlitz. Some orthogonal $q$-polynomials. *Math. Nachr.*, 30:47–61, 1965.

G. E. Andrews, R. Askey, and R. Roy. *Special Functions*. Cambridge Univ. Press, New York, first edition, 2000. ISBN: 0-521-78988-5.

R. Askey. Some basic hypergeometric extensions of integrals of Selberg and Andrews. *Comm. Math. Phys.*, 11:203–951, 1980.

T. H. Baker and P. J. Forrester. Multivariable Al-Salam & Carlitz polynomials associated with the type A $q$-Dunkl kernel. *Math. Nachr.*, 212:5–35, 2000.

A. I. Barvinok. New algorithms for $k$-matroid intersection and matroid $k$-parity problems. *Math. Program*, 69:449–470, 1995.

D. M. Bressoud. *Proofs and Confirmations: The Story of the Alternating-Sign Matrix Conjecture*. Cambridge University Press, Cambridge, first edition, 1999. ISBN: 0-521-66170-6.

T. Chihara. *An Introduction to Orthogonal Polynomials*. Gordon and Breach, New York, 1978.

N. G. de Bruijn. On some multiple integrals involving determinants. *J. Indian Math. Soc.*, 19:133–151, 1955.

G. Gasper and M. Rahman. *Basic Hypergeometric Series*. Cambridge Univ. Press, Cambridge, second edition, 2004. ISBN: 9-780-52183357-8.

L. Habsieger. Une $q$-intégrale de Selberg et Askey. *Publ. I.R.M.A. Strasboug*, 334:25–45, 1987.

L. Habsieger. Une $q$-intégrale de Selberg et Askey. *SIAM J. Math. Anal.*, 19:1475–1489, 1988.

M. Ishikawa and C. Koutschan. Zeilberger's holonomic ansatz for Pfaffians. *CoRR abs*, 1201.5253, 2012. arxiv:1011.5941.

M. Ishikawa and M. Wakayama. Minor summation formula of Pfaffians. *Linear and Multilinear Alg.*, 39: 285–305, 1995.

M. Ishikawa and M. Wakayama. Applications of minor summation formula, III: Plücker relations, lattice paths and Pfaffian identities. *J. Combin. Theory Ser. A*, 113:113–155, 2006.
M. Ishikawa and J. Zeng. A Pfaffian analogue of the Hankel determinants and the Selberg integrals. *RIMS Kokyuroku*, 1795:189–203, 2012.

M. Ishikawa, H. Tagawa, and J. Zeng. A *q*-analogue of Catalan Hankel determinants. *RIMS Kôkyûroku Bessatsu*, B11:19–42, 2009. available at arXiv:1009.2004.

M. Ishikawa, H. Tagawa, and J. Zeng. A generalization of the Mehta-Wang determinant and Askey-Wilson polynomials. available at arXiv:1210.5305, 2012.

M. Ishikawa, H. Tagawa, and J. Zeng. Pfaffian decomposition and a Pfaffian analogue of *q*-Catalan Hankel determinants. available at arXiv:1011.5941, 2013.

K. W. J. Kadell. A proof of Askey's conjectured *q*-analogue of Selberg's integral and a conjecture of Morris. *SIAM J. Math. Anal.*, 19:969–986, 1988.

R. Koekoek, P. A. Lesky, and R. F. Swarttouw. *Hypergeometric Orthogonal Polynomials and Their analogues*. Springer-Verlag, Berlin-New York, first edition, 2010. ISBN: 978-3-642-05014-5.

J. Luque and J. Thibon. Pfaffian and hafnian identities in shuffle algebras. *Adv. Appl. Math.*, 29:620–646, 2002.

J. Luque and J. Thibon. Hankel hyperdeterminants and Selberg integrals. *J. Phys. A: Math. Gen.*, 36: 5267–5292, 2003.

S. Matsumoto. Hyperdeterminantal expressions for Jack functions of rectangular shapes. *Journal of Algebra*, 320:612–632, 2008.

C. D. Sinclair. Ensemble averages when $\beta$ is a square integer. *Monatsh. Math.*, 166:121–144, 2012.

R. Stanley. *Enumerative combinatorics, Volume I*. Cambridge University Press, second edition edition, 2011.

J. Stembridge. Nonintersecting paths, Pfaffians, and plane partitions. *Adv. Math.*, 83:96–131, 1990.