<figure><img src="image_1.png" /></figure>

HAL
open science

A representation-theoretic proof of the branching rule
for Macdonald polynomials

Yi Sun

► To cite this version:

Yi Sun. A representation-theoretic proof of the branching rule for Macdonald polynomials. 27th International Conference on Formal Power Series and Algebraic Combinatorics (FPSAC 2015), Jul 2015, Daejeon, South Korea. pp.853-864, 10.46298/dmtcs.2493 . hal-01337768

HAL Id: hal-01337768

https://hal.science/hal-01337768

Submitted on 27 Jun 2016

**HAL** is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# A representation-theoretic proof of the branching rule for Macdonald polynomials

Yi Sun¹†

¹Massachusetts Institute of Technology, Cambridge, MA 02139, USA

**Abstract.** We give a new representation-theoretic proof of the branching rule for Macdonald polynomials using the Etingof-Kirillov Jr. expression for Macdonald polynomials as traces of intertwiners of $U_q(gl_n)$. In the Gelfand-Tsetlin basis, we show that diagonal matrix elements of such intertwiners are given by application of Macdonald's operators to a simple kernel. An essential ingredient in the proof is a map between spherical parts of double affine Hecke algebras of different ranks based upon the Dunkl-Kasatani conjecture.

**Résumé.** Nous donnons une nouvelle preuve représentation-théorique de la règle de branchement pour les polynômes de Macdonald en utilisant l'expression Etingof-Kirillov Jr. pour les polynômes de Macdonald comme des traces de intertwiners de $U_q(gl_n)$. Dans la base de Gelfand-Tsetlin, nous montrons que les éléments de matrice diagonaux de ces intertwiners sont donnés par action des opérateurs de Macdonald à un noyau simple. Un ingrédient essentiel dans la preuve est une application entre les parties sphériques des algèbres de Hecke double affines de rangs différents basés sur la conjecture Dunkl-Kasatani.

**Keywords:** Macdonald polynomials, quantum groups, Gelfand-Tsetlin basis, double affine Hecke algebras

## 1 Introduction

The Macdonald polynomials $P_\lambda(x; q, t)$ are a two-parameter family of symmetric polynomials indexed by partitions $\lambda$ which form an orthogonal basis for the ring of symmetric functions with respect to a $(q, t)$-deformation of the standard inner product. They were originally introduced by Macdonald (see [Mac95]) as a generalization of many known families of special functions, including Schur functions, Jack and Hall-Littlewood polynomials, and Heckman-Opdam hypergeometric functions. Macdonald proved a branching rule for the $P_\lambda(x; q, t)$ and conjectured three additional symmetry, evaluation, and norm identities collectively known as Macdonald's conjectures. These conjectures were proven by Cherednik using techniques from double affine Hecke algebras. Etingof and Kirillov Jr. realized the Macdonald polynomials in [EK94] in terms of traces of intertwiners of the quantum group $U_q(gl_n)$; using this interpretation, they gave new proofs of Macdonald's conjectures in [EK96].

The purpose of this work is to give a representation-theoretic proof and interpretation of Macdonald's branching rule from the perspective of quantum groups. We give a new expression for diagonal matrix

<footnote>†Email: yisun@math.mit.edu. Supported by a NSF graduate research fellowship (NSF Grant #1122374).</footnote>
elements of $U_q(\mathrm{gl}_n)$-intertwiners in the Gelfand-Tsetlin basis as the application of Macdonald's difference operators to a simple kernel. We then show that the resulting summation expression for $P_\lambda(x; q, t)$ becomes Macdonald's branching rule after a summation by parts procedure. A key ingredient is the construction of a map $\mathrm{Res}_l(q^2)$ between spherical parts of double affine Hecke algebras of different ranks. Our construction makes use of the Dunkl-Kasatani conjecture and is compatible with Cherednik's $SL_2(\mathbb{Z})$-action on spherical DAHA.

In the remainder of this extended abstract, we state our results on matrix elements of quantum group intertwiners and maps between spherical double affine Hecke algebras and then explain in more detail how they may be used to give a new proof of Macdonald's branching rule. Full details, background, and a complete list of references can be found in [Sun14b].

## 1.1 Macdonald polynomials

Let $\rho = (\frac{n-1}{2}, \dots, \frac{1-n}{2})$ and let $e_r$ denote the elementary symmetric polynomial. For a partition $\lambda$, the Macdonald polynomial $P_\lambda(x; q^2, t^2)$ is the joint polynomial eigenfunction with leading term $x^\lambda$ and eigenvalue $e_r(q^{2\lambda}t^{2\rho})$ of the operators

$$D_{n,x}^r(q^2, t^2) = t^{r(r-n)} \sum_{|I|=r} \prod_{i \in I, j \notin I} \frac{t^2 x_i - x_j}{x_i - x_j} T_{q^2, I},$$

where $T_{q^2, I} = \prod_{i \in I} T_{q^2, i}$ and $T_{q^2, i}f(x_1, \dots, x_n) = f(x_1, \dots, q^2 x_i, \dots, x_n)$. An integral signature $\lambda$ is a sequence $\lambda = (\lambda_1 \ge \dots \ge \lambda_n)$ with $\lambda_i - \lambda_j \in \mathbb{Z}$. We extend the definition of Macdonald polynomials to arbitrary signatures by setting $P_{(\lambda_1+c, \dots, \lambda_n+c)}(x; q^2, t^2) = (x_1 \cdots x_n)^c P_\lambda(x; q^2, t^2)$.

Say that integral signatures $\mu = (\mu_1 \ge \dots \ge \mu_{n-1})$ and $\lambda = (\lambda_1 \ge \dots \ge \lambda_n)$ interlace if $\lambda_1 \ge \mu_1 \ge \lambda_2 \ge \dots \ge \mu_{n-1} \ge \lambda_n$. Denote interlacing by $\mu \prec \lambda$ and write $|\lambda| = \sum_i \lambda_i$. A Gelfand-Tsetlin pattern subordinate to $\lambda$ is an interlacing sequence $\mu = \{\mu^l\}_{1 \le l \le n} = \{\mu^1 \prec \mu^2 \prec \dots \prec \mu^{n-1} \prec \mu^n = \lambda\}$ ending in $\lambda$. Define the $q$-Pochhammer symbol by $(u; q) = \prod_{n \ge 0} (1 - uq^n)$. In [Mac95], Macdonald showed that $P_\lambda(x)$ satisfies the following branching rule, which yields an explicit summation expression for $P_\lambda(x)$ over Gelfand-Tsetlin patterns subordinate to $\lambda$.

**Theorem 1.1** ([Mac95, VI.7.13']). The Macdonald polynomials satisfy the branching rule

$$P_{\lambda}(x_1, \dots, x_n) = \sum_{\mu \prec \lambda} \psi_{\lambda/\mu}(q, t) P_{\mu}(x_1, \dots, x_{n-1}) x_n^{|\lambda|-|\mu|},$$

where the branching coefficient is

$$\psi_{\lambda/\mu}(q, t) = \prod_{1 \le i \le j \le \ell(\mu)} \frac{(q^{\mu_i - \mu_j} t^{j-i+1}; q)(q^{\lambda_i - \lambda_{j+1}} t^{j-i+1}; q)(q^{\lambda_i - \mu_j + 1} t^{j-i}; q)(q^{\mu_i - \lambda_{j+1} + 1} t^{j-i}; q)}{(q^{\mu_i - \mu_j + 1} t^{j-i}; q)(q^{\lambda_i - \lambda_{j+1} + 1} t^{j-i}; q)(q^{\lambda_i - \mu_j} t^{j-i+1}; q)(q^{\mu_i - \lambda_{j+1} + 1} t^{j-i+1}; q)}.$$

**Corollary 1.2.** The Macdonald polynomials admit the summation formula

$$P_{\lambda}(x, q, t) = \sum_{\mu^1 < \cdots < \mu^{n-1} < \mu^n = \lambda} \prod_{i=1}^{n} \psi_{\mu^i / \mu^{i-1}}(q, t) \prod_{i=1}^{n} x_i^{|\mu^i| - |\mu^{i-1}|}.$$
## 1.2 The quantum group $U_q(\mathfrak{gl}_n)$

Let $U_q(\mathfrak{gl}_n)$ be the finite type quantum group with generators $e_i, f_i$ for $i = 1, \dots, n-1$ and $q^{\frac{h_i}{2}}$ for $i = 1, \dots, n$ and relations

$$[h_i, h_j] = 0, [h_i, e_i] = e_i, [h_i, f_i] = -f_i, [h_i, e_{i+1}] = -e_{i+1}, [h_i, f_{i+1}] = f_{i+1},$$

$$[h_i, e_j] = [h_i, f_j] = 0 \text{ for } j \neq i, i+1, [e_i, f_j] = \delta_{ij} \frac{q^{h_i-h_{i+1}} - q^{h_{i+1}-h_i}}{q-q^{-1}}, [e_i, e_j] = [f_i, f_j] = 0 \text{ for } |i-j| > 1$$

$$e_i^2 e_j - (q+q^{-1}) e_i e_j e_i + e_j e_i^2 = 0, \quad f_i^2 f_j - (q+q^{-1}) f_i f_j f_i + f_j f_i^2 = 0 \text{ for } |i-j| = 1.$$

We take the coproduct on $U_q(\mathfrak{gl}_n)$ defined by $\Delta(e_i) = e_i \otimes q^{\frac{h_{i+1}-h_i}{2}} + q^{\frac{h_i-h_{i+1}}{2}} \otimes e_i$, $\Delta(f_i) = f_i \otimes q^{\frac{h_{i+1}-h_i}{2}} + q^{\frac{h_i-h_{i+1}}{2}} \otimes f_i$, and $\Delta(h_i) = h_i \otimes 1 + 1 \otimes h_i$. Denote the subalgebra generated by $f_i$ and $q^{h_i/2}$ by $U_q(\mathfrak{b}_i)$. For each $r < n$, embed $U_q(\mathfrak{gl}_r)$ inside $U_q(\mathfrak{gl}_n)$ as the subalgebra generated by $e_1, \dots, e_{r-1}$, $f_1, \dots, f_{r-1}$, and $q^{h_i/2}, \dots, q^{h_r/2}$. Finally, denote the finite dimensional irreducible $U_q(\mathfrak{gl}_n)$-representation corresponding to an integral signature $\lambda$ by $L_\lambda$.

## 1.3 Etingof-Kirillov Jr. approach to Macdonald polynomials

In [EK94], Etingof and Kirillov Jr. realized Macdonald polynomials via traces of $U_q(\mathfrak{gl}_n)$-intertwiners. Let $W_{k-1}$ denote the representation $L_{((k-1)(n-1),-(k-1),\dots,-(k-1))} = \mathrm{Sym}^{(k-1)n}(\mathbb{C}^n) \otimes (\mathrm{det})^{-(k-1)}$, and choose an isomorphism $W_{k-1}[0] \simeq \mathbb{C} \cdot w_{k-1}$ for some $w_{k-1} \in W_{k-1}[0]$ which spans the 1-dimensional zero weight space $W_{k-1}[0]$. Define the weight $\rho_n = (\frac{n-1}{2}, \dots, \frac{n-1}{2})$. Writing $\rho$ for $\rho_n$, for a signature $\lambda$ there exists a unique intertwiner $\Phi_\lambda^n : L_{\lambda+(k-1)\rho} \to L_{\lambda+(k-1)\rho} \otimes W_{k-1}$ normalized to send the highest weight vector $v_{\lambda+(k-1)\rho}$ in $L_{\lambda+(k-1)\rho}$ to

$$v_{\lambda+(k-1)\rho} \otimes w_{k-1} + \text{(lower order terms)},$$

where (lower order terms) denotes terms of weight lower than $\lambda + (k-1)\rho$ in the first tensor coordinate. Traces of these intertwiners lie in $W_{k-1}[0] = \mathbb{C} \cdot w_{k-1}$ and yield Macdonald polynomials when interpreted as scalar functions via the identification $w_{k-1} \mapsto 1$.

**Theorem 1.3** ([EK94, Theorem 1]). The Macdonald polynomial is given by $P_\lambda(x; q^2, q^{2k}) = \frac{\mathrm{Tr}(\Phi_\lambda^n x^h)}{\mathrm{Tr}(\Phi_0^n x^h)}$.

**Proposition 1.4** ([EK94, Main Lemma]). On $L_{(k-1)\rho}$, the trace may be expressed explicitly as

$$\mathrm{Tr}(\Phi_0^n x^h) = (x_1 \cdots x_n)^{-\frac{(k-1)(n-1)}{2}} \prod_{s=1}^{k-1} \prod_{i<j} (x_i - q^{2s} x_j).$$

**Remark.** Our notation is related to that of [EK94] via $P_\lambda^{EK}(x; q, t) = P_\lambda(x; q^2, t^2)$.

## 1.4 Gelfand-Tsetlin basis

The representation $L_\lambda$ of $U_q(\mathfrak{gl}_n)$ admits a basis $\{v_\mu\}$ indexed by Gelfand-Tsetlin patterns $\mu$ subordinate to $\lambda$. The weight of a basis vector $v_\mu$ is $\mathrm{wt}(v_\mu) = (|\mu^n| - |\mu^{n-1}|, \dots, |\mu^2| - |\mu^1|, |\mu^1|)$. It was shown in [UTS90] that these basis vectors may be expressed in terms of lowering operators $d_{r,i}$ in $U_q(\mathfrak{gl}_r) \cap U_q(\mathfrak{b}_i) \subset U_q(\mathfrak{gl}_n)$ applied to the highest weight vector $v_\lambda$.
**Proposition 1.5** ([UTS90, Theorem 2.9]). The Gelfand-Tsetlin basis vectors may be realized as $v_\mu = d_1^{\mu_1} d_2^{\mu_2-\mu_1} \cdots d_n^{\mu_n-\mu^{n-1}} v_\lambda$ for $d_{r,i} \in U_q(\mathfrak{gl}_r) \cap U_q(\mathfrak{b}_-) $ and $d_r^\tau = d_{r,1}^{\tau_1} \cdots d_{r,r}^{\tau_r}$ for a partition $\tau$.

**1.5 Statement of the main results**

Our main result shows that diagonal matrix elements of the $U_q(\mathfrak{gl}_n)$-intertwiners of Theorem 1.3 are given by application of Macdonald’s operators to a simple kernel.

**Theorem 1.6.** In the Gelfand-Tsetlin basis, the diagonal matrix element of $\Phi_\lambda^n$ on the basis vector corresponding to the Gelfand-Tsetlin pattern

$$ \{\sigma^1 < \cdots < \sigma^{n-1} < \lambda + (k-1)\rho\} $$

with $\sigma_i^l = \mu_i + (k-1) \frac{n-i}{2}$ is given by

$$ c(\mu, \lambda) = \frac{\prod_{a=1}^{k-1} D_{n-1, q^{2\mu}} (q^{2a}; q^{-2}, q^{2(k-1)}) \prod_{i \le j} [\lambda_i - \mu_j + k(j-i)]_{k-1} \prod_{i < j} [\mu_i - \lambda_j + k(j-i) + k-2]_{k-1}}{\prod_{i \le j} [\mu_i - \mu_j + k(j-i) + k-1]_{k-1} \prod_{i < j} [\lambda_i - \lambda_j + k(j-i) - 1]_{k-1}}, $$

where $\bar{\mu}_i = \mu_i - k(i-1)$, $[m] = \frac{q^m q^{-m}}{q-q^{-1}}$, $[m]_k = [m] \cdots [m-k+1]$, and

$$ D_{n-1,q^{2\mu}}(u;q^2,t^2) = \sum_{r=0}^{n-1} (-1)^{n-1-r} u^{n-1-r} D_{n-1,q^{2\mu}}^r (q^2,t^2). $$

Using Theorem 4.4, we give a new representation-theoretic proof of Macdonald’s branching rule.

**Theorem 1.7.** At $t = q^k$ for positive integer $k$, we have

$$ P_{\lambda}(x_1, \dots, x_n; q^2, q^{2k}) = \sum_{\mu \le \lambda} x_n^{\lambda|\lambda|-|\mu|} P_{\mu}(x_1, \dots, x_{n-1}; q^2, q^{2k}) \psi_{\lambda/\mu}(q^2, q^{2k}) $$

with

$$ \psi_{\lambda/\mu}(q^2, q^{2k}) = \frac{\prod_{i \le j} [\lambda_i - \mu_j + k(j-i) + k-1]_{k-1} \prod_{i < j} [\mu_i - \lambda_j + k(j-i) - 1]_{k-1}}{\prod_{i \le j} [\mu_i - \mu_j + k(j-i) + k-1]_{k-1} \prod_{i < j} [\lambda_i - \lambda_j + k(j-i) - 1]_{k-1}}. $$

**Remark.** This formulation is equivalent to that of Theorem 1.1. To see this, note that for each $\lambda$ and $\mu$ the branching coefficients $\psi_{\lambda/\mu}(q, t)$ are rational functions in $q$ and $t$ and are therefore uniquely determined by their values at $(q^2, q^{2k})$ for all positive integers $k$.

**Remark.** Theorem 1.3 gives $P_\lambda(x; q^2, q^{2k})$ as a summation over Gelfand-Tsetlin patterns subordinate to $\lambda + (k-1)\rho$ and Macdonald’s branching rule gives it as a summation over Gelfand-Tsetlin patterns subordinate to $\lambda$. Our result explains how these summations over different index sets are related.

**1.6 Degenerations of our results and connections to recent work**

We discuss now the Heckman-Opdam, Hall-Littlewood, and Jack limits of our results.
* In the quasi-classical limit $q = e^\epsilon$, $t = q^k$, $\lambda = \lfloor \epsilon^{-1}\Lambda \rfloor$, $x = e^{\epsilon X}$, and $\epsilon \to 0$, the Macdonald polynomials become the Heckman-Opdam hypergeometric functions introduced in [HO87, Opd88]. These functions were realized as integrals over Gelfand-Tsetlin polytopes in [BG13] by a scaling limit of Corollary 1.2. In [Sun14a], the expression of [BG13] was lifted to an integral over dressing orbits of a Poisson-Lie group by integration over Liouville tori and adjunction for Calogero-Sutherland Hamiltonians. The techniques of this paper degenerate to the techniques of [Sun14a].

* In the specialization $q=0$, the Macdonald polynomials become the Hall-Littlewood polynomials. In [Ven14], a summation expression was given for matrix elements of the $U_q(\mathfrak{gl}_n)$-intertwiners $\Phi_\lambda^n$ in the Gelfand-Tsetlin basis; this expression factors in the Hall-Littlewood limit. It would be interesting to understand if it may be realized as a degeneration of Proposition 4.3 or Theorem 4.4.

* The Jack polynomials are a scaling limit of Macdonald polynomials under the specialization $t = q^k$ and the limit $q \to 1$ and have a similar branching rule. They were given in [Eti95] as traces of intertwiners of $U(\mathfrak{gl}_n)$-modules using a degeneration of the Etingof-Kirillov Jr. construction, and we expect our methods to degenerate to a representation-theoretic proof of the Jack branching rule.

# 2 Quantum groups and Macdonald polynomials

## 2.1 Notations

Set $\rho_{n,i} = \frac{n+1}{2} - i$ and $\mathbf{1} = (1, \dots, 1)$. For any set of indices $I$, let $\mathbf{1}_I$ denote the vector with 1's in those indices and 0's elsewhere. Define $\tilde{\rho}_n = \rho_n - \frac{n-1}{2}\mathbf{1}$ so that $\tilde{\rho}_{n,i} = -(i-1)$ and $\tilde{\rho}_{n-1,i} = \tilde{\rho}_{n,i}$. For any signature $\lambda$, define the shifts $\tilde{\lambda} = \lambda + (k-1)\tilde{\rho}$ and $\bar{\lambda} = \lambda + k\tilde{\rho}$. Denote by $[a] = \frac{q^a-q^{-a}}{q-q^{-1}}$ the q-number, $[a]! = [a] \cdot [a-1] \cdots [1]$ the q-factorial, and $[a]_m = [a] \cdot [a-1] \cdots [a-m+1]$ the falling q-factorial.

## 2.2 Macdonald symmetry identity

We now produce Macdonald operators acting on indices. We abuse notation to write $D_{n-1,q^{2\bar{\mu}}}^r$ for difference operators acting on additive indices $\bar{\mu}$ as well as multiplicative variables $q^{2\bar{\mu}}$.

**Proposition 2.1 (Macdonald symmetry identity).** We have

$$ P_{\lambda}(q^{2\mu+2k\rho}; q^2, q^{2k}) = \prod_{i<j} \frac{[\lambda_i - \lambda_j + k(j-i) + k - 1]_k}{[\mu_i - \mu_j + k(j-i) + k - 1]_k} P_{\mu}(q^{2\lambda+2k\rho}; q^2, q^{2k}). $$

**Proposition 2.2.** The operator

$$ \tilde{D}_{n-1,q^{2\bar{\mu}}}^r (q^2, q^{2k}) = \prod_{i<j} [\bar{\mu}_i - \bar{\mu}_j + k - 1]_k \circ D_{n-1,q^{2\bar{\mu}}}^r (q^2, q^{2k}) \circ \prod_{i<j} [\bar{\mu}_i - \bar{\mu}_j + k - 1]_k^{-1} $$

satisfies

$$ \tilde{D}_{n-1,q^{2\bar{\mu}}}^r (q^2, q^{2k}) = \sum_{|I|=r} \prod_{i \in I, j \notin I, i>j} \frac{[\bar{\mu}_i - \bar{\mu}_j + k][\bar{\mu}_i - \bar{\mu}_j - k + 1]}{[\bar{\mu}_i - \bar{\mu}_j][\bar{\mu}_i - \bar{\mu}_j + 1]} T_{q^2, I} $$

$$ \text{and } \tilde{D}_{n-1,q^{2\bar{\mu}}}^r (q^2, q^{2k}) P_{\mu}(x; q^2, q^{2k}) = e_r(x) P_{\mu}(x; q^2, q^{2k}). $$

*Proof.* The expression for $\tilde{D}_{n-1,q^{2\bar{\mu}}}^r (q^2, q^{2k})$ follows by direct computation, and the eigenvalue identity from the Macdonald symmetry identity. $\square$
## 2.3 Adjoints of Macdonald difference operators

We now consider adjoints of Macdonald operators with respect to a Jackson-type inner product. Fix lower and upper limits $\zeta = (\zeta^-, \zeta^+)$ with $\zeta^- = (\zeta_1^-, \dots, \zeta_{n-1}^-)$, $\zeta^+ = (\zeta_1^+, \dots, \zeta_{n-1}^+)$, and $\zeta_i^+ - \zeta_i^- \in \mathbb{Z}_{\ge 0}$. Define the inner product $\langle f, g \rangle_\zeta := \sum_{\mu=\zeta^-}^{\zeta^+} f(q^{2\mu})g(q^{2\mu})$, with iterated summation defined by

$$ \sum_{\mu=\zeta^-}^{\zeta^+} := \sum_{\mu_1=\zeta_1^-}^{\zeta_1^+} \dots \sum_{\mu_{n-1}=\zeta_{n-1}^-}^{\zeta_{n-1}^+} . \qquad (1) $$

We consider situations where $g$ vanishes along a border of the region of summation. Say that the function $g(q^{2\mu})$ is $(\zeta, l)$-adapted if $g(q^{2\mu}) = 0$ on the set $\{\mu \mid \zeta_i^+ < \mu_i \le \zeta_i^+ + l \text{ or } \zeta_i^- - l \le \mu_i < \zeta_i^- \text{ for any } i\}$. Proposition 2.3. If $f(q^{2\mu})$ is $(\zeta, l)$-adapted, we have for any $g$ that

$$ \left\langle \prod_{i=1}^{l} \tilde{D}_{n-1,q^{2\mu}}^{r_{l+1-i}}(q^2, q^{2k})^{\dagger} f, g \right\rangle_{\zeta^{-}, \zeta^{+}+l1} = \left\langle f, \prod_{i=1}^{l} \tilde{D}_{n-1,q^{2\mu}}^{r_i}(q^2, q^{2k})g \right\rangle_{\zeta}, $$

where

$$ \tilde{D}_{n-1,q^{2\mu}}^{r}(q^2, q^{2k})^{\dagger} = \prod_{i<j} [\bar{\mu}_i - \bar{\mu}_j + k - 1]_{k-1}^{-1} \circ D_{n-1,q^{2\mu}}^{r}(q^{-2}, q^{2(k-1)}) \circ \prod_{i<j} [\bar{\mu}_i - \bar{\mu}_j + k - 1]_{k-1}. $$

*Proof.* By a direct computation for $l=1$ and induction on $l$. $\square$

<figure><img src="image_3.png" /></figure>

## 2.4 Reformulating the Etingof-Kirillov Jr. construction

We shift the weights used in the Etingof-Kirillov Jr. construction to make restriction from $U_q(\mathfrak{gl}_n)$ to $U_q(\mathfrak{gl}_{n-1})$ more notationally convenient. For a partition $\lambda$, define the intertwiner $\tilde{\Phi}_{\lambda}^n : L_{\lambda+(k-1)\bar{\rho}} \to L_{\lambda+(k-1)\bar{\rho}} \otimes W_{k-1}$ to be $\tilde{\Phi}_{\lambda}^n = \Phi_{\lambda}^n \otimes \text{id}_{(\det)^{-\frac{(k-1)(n-1)}{2}}}$. We rephrase Theorem 1.3 in terms of $\tilde{\Phi}_{\lambda}^n$.

**Corollary 2.4.** The Macdonald polynomial $P_\lambda(x; q^2, q^{2k})$ is given by $P_\lambda(x; q^2, q^{2k}) = \frac{\text{Tr}(\tilde{\Phi}_\lambda^n x^h)}{\text{Tr}(\tilde{\Phi}_\lambda^n x^h)}$.

**Corollary 2.5.** The denominator in Corollary 2.4 is given by

$$ \text{Tr}(\tilde{\Phi}_0^n x^h) = (x_1 \cdots x_n)^{-(k-1)(n-1)} \prod_{s=1}^{k-1} \prod_{i<j} (x_i - q^{2s} x_j). $$

# 3 Spherical subalgebras of DAHAs of different ranks

## 3.1 Double affine Hecke algebras

Let $\mathcal{H}_n(q, t)$ denote the double affine Hecke algebra (DAHA) of $GL_n$ defined by [Che05]. It is defined as the algebra generated by $X_1^{\pm}, \dots, X_n^{\pm}, Y_1^{\pm}, \dots, Y_n^{\pm}$, and $T_1^{\pm}, \dots, T_{n-1}^{\pm}$ subject to the relations

$$ \bullet \quad (T_i - t)(T_i + t^{-1}) = 0, \quad T_i T_{i+1} T_i = T_{i+1} T_i T_{i+1}, \quad [T_i, T_j] = 0 \text{ for } |i-j| \neq 1; $$
* $T_i X_i T_i = X_{i+1}, T_i^{-1} Y_i T_i^{-1} = Y_{i+1}$, and $[T_i, X_j] = [T_i, Y_j] = 0$ for $|i-j| > 1$;

* $[X_i, X_j] = 0$, $[Y_i, Y_j] = 0$, $Y_1 X_1 \cdots X_n = q X_1 \cdots X_n Y_1$, and $X_1^{-1} Y_2 = Y_2 X_1^{-1} T_1^{-2}$.

For a reduced decomposition $\sigma = s_{i_1} \cdots s_{i_l}$, let $T_\sigma = T_{i_1} \cdots T_{i_l}$. For $e = \frac{(1-t^2)^n}{(t^2;t^2)_n} \sum_{\sigma \in S_n} t^{\ell(\sigma)} T_\sigma$, the spherical DAHA is the subalgebra $e\mathcal{H}_n(q,t)e$.

## 3.2 Polynomial representation of DAHA and Macdonald operators

The DAHA $\mathcal{H}_n(q,t)$ admits a faithful polynomial representation $\rho$ on $\mathbb{C}[X_1^{\pm}, \dots, X_n^{\pm}]$ given by

$$\rho(X_i) = X_i$$

$$\rho(T_i) = ts_i + \frac{t - t^{-1}}{X_i/X_{i+1} - 1}(s_i - 1)$$

$$\rho(Y_i) = \rho(T_i) \cdots \rho(T_{n-1}) s_{n-1} \cdots s_1 T_{q,X_1} \rho(T_1^{-1}) \cdots \rho(T_{i-1}^{-1}),$$

where $s_i$ exchanges $X_i$ and $X_{i+1}$ and $T_{q,X_1}$ is the $q$-shift operator in $X_1$. The action of elements of $e\mathcal{H}_n(q,t)e$ on the symmetric part of the polynomial representation yields the Macdonald operators.

**Proposition 3.1.** When restricted to $\mathbb{C}[X_1^{\pm}, \dots, X_n^{\pm}]^{S_n}$, the action of $e \cdot e_r(Y_1, \dots, Y_n) \cdot e$ is via

$$\rho(e \cdot e_r(Y_1, \dots, Y_n) \cdot e) = D_{n,X}^r(q^2, t^2).$$

**Remark.** By faithfulness, we will refer interchangeably to elements of the DAHA and spherical DAHA and their images under the polynomial representation in what follows.

## 3.3 $SL_2(\mathbb{Z})$-action on DAHA

Define the isomorphisms $\varepsilon(q,t) : \mathcal{H}_n(q,t) \to \mathcal{H}_n(q^{-1}, t^{-1})$ given by

$$\varepsilon(q,t) : X_i \mapsto Y_i, Y_i \mapsto X_i, T_i \mapsto T_i^{-1}, q \mapsto q^{-1}, t \mapsto t^{-1}$$

and $\tau_+(q,t) : \mathcal{H}_n(q,t) \to \mathcal{H}_n(q,t)$ given by

$$\tau_+ : X_i \mapsto X_i, T_i \mapsto T_i, Y_1 \cdots Y_r \mapsto q^{-r/2} X_1 \cdots X_r Y_1 \cdots Y_r.$$

Define also the composition $\tau_- = \varepsilon\tau_+\varepsilon$.

**Proposition 3.2 ([Che05]).** The map $\begin{pmatrix} 1 & 1 \\ 0 & 1 \end{pmatrix} \mapsto \tau_-$ and $\begin{pmatrix} 1 & 0 \\ 1 & 1 \end{pmatrix} \mapsto \tau_+$ defines an action of $SL_2(\mathbb{Z})$ on $\mathcal{H}_n(q,t)$ which preserves $e\mathcal{H}_n(q,t)e$.

## 3.4 Multiwheel condition and the restriction map

Following [Kas05], we say that $(X_1^0, \dots, X_n^{l-1}) \in \mathbb{C}^{nl}$ satisfies the *multiwheel condition* if the indices may be permuted so that

$$X_i^a = X_i^0 t^{a-1} \text{ for } 1 \le i \le n \text{ and } 0 \le a \le l-1.$$

Define the ideal $I_{nl}(t) \subset \mathbb{C}[(X_i^a)^\pm]$ by $I_{nl}(t) = \{f \mid f(X) = 0 \text{ if } X \text{ satisfy the multiwheel condition}\}$.
**Proposition 3.3** ([Kas05, Theorem 6.3] and [ES09, Theorem 5.10]). The subspace $I_{nl}(t) \subset \mathbb{C}[(X_i^a)^\pm]$ is a $\mathcal{H}_{nl}(q, t)$-submodule and $\mathbb{C}[(X_i^a)^\pm]/I_{nl}(t)$ is irreducible.

Define the map $\mathrm{Res}_l(t^2) : \mathbb{C}[(X_i^a)^\pm]^{S_{nl}} \to \mathbb{C}[X_i^\pm]^{S_n}$ by $\mathrm{Res}_l(t^2)(X_i^a) = t^{1-l+2a}X_i$. The kernel of $\mathrm{Res}_l(t^2)$ is $I_{nl}^{S_{nl}}(t^2)$, so $\mathrm{Res}_l(q^2)$ induces by Proposition 3.3 an action of $e\mathcal{H}_{nl}(q^{-2l}, q^2)e$ on $\mathbb{C}[X_i^\pm]^{S_n}$, giving a map $\mathrm{Res}_l(q^2) : e\mathcal{H}_{nl}(q^{-2l}, q^2)e \to \mathrm{End}(\mathbb{C}[X_i^\pm]^{S_n})$. We claim that this map factors through the polynomial representation via a map of algebras $\mathrm{Res}_l(q^2) : e\mathcal{H}_{nl}(q^{-2l}, q^2)e \to e\mathcal{H}_n(q^{-2}, q^{2l})e$.

**Theorem 3.4.** The map $\mathrm{Res}_l(q^2) : e\mathcal{H}_{nl}(q^{-2l}, q^2)e \to e\mathcal{H}_n(q^{-2}, q^{2l})e$ defined by

$$ \mathrm{Res}_l(q^2)(\mathrm{ep}(X_i^a)e) = \mathrm{ep}(q^{1-l}X_1, \dots, q^{l-1}X_1, \dots, q^{1-l}X_n, \dots, q^{l-1}X_n)e \text{ for } p \in \mathbb{C}[(X_i^a)^\pm]^{S_{nl}}, \text{ and} $$

$$ \mathrm{Res}_l(q^2)(\mathrm{ep}(Y_i^a)e) = \mathrm{ep}(q^{1-l}Y_1, \dots, q^{l-1}Y_1, \dots, q^{1-l}Y_n, \dots, q^{l-1}Y_n)e $$

is well defined and satisfies

(a) for any $h \in e\mathcal{H}_{nl}(q^{-2l}, q^2)e$, as operators on $\mathbb{C}[(X_i^a)^\pm]^{S_{nl}}$ we have

$$ \mathrm{Res}_l(q^2) \circ h = \mathrm{Res}_l(h) \circ \mathrm{Res}_l(q^2); $$

(b) as operators on $e\mathcal{H}_{nl}(q^{-2l}, q^2)e$, we have

$$ \mathrm{Res}_l(q^{-2}) \circ \varepsilon_{nl}(q^{-2l}, q^2) = \varepsilon_n(q^{-2}, q^{2l}) \circ \mathrm{Res}_l(q^2); $$

(c) as operators on $e\mathcal{H}_{nl}(q^{-2l}, q^2)e$, we have

$$ \mathrm{Res}_l(q^2) \circ \tau_+ = \tau_+ \circ \mathrm{Res}_l(q^2). $$

**Corollary 3.5.** The map $\mathrm{Res}_l(q^2)$ commutes with the action of $SL_2(\mathbb{Z})$ on the spherical DAHA.

The assignment $\mathrm{Res}_l(q^2)((X_i^a)^{1/2}) = q^{a-(l-1)/2}X_i^{1/2}$ extends $\mathrm{Res}_l(q^2)$ to an operator $\mathbb{C}[(X_i^a)^{\pm 1/2}]^{S_{nl}} \to \mathbb{C}[X_i^{1/2}]^{S_n}$. Identify elements of the spherical DAHA with difference operators on $\prod_{i,a}(X_i^a)^{1/2} \cdot \mathbb{C}[(X_i^a)^\pm]^{S_{nl}} \subset \mathbb{C}[(X_i^a)^{\pm 1/2}]^{S_{nl}}$. They may not satisfy the spherical DAHA relations, but we still have the following.

**Corollary 3.6.** For any $h \in e\mathcal{H}_{nl}(q^{-2l}, q^2)e$, we have $\mathrm{Res}_l(q^2) \circ h = \mathrm{Res}_l(h) \circ \mathrm{Res}_l(q^2)$ as operators on $\prod_{i,a}(X_i^a)^{1/2} \cdot \mathbb{C}[(X_i^a)^\pm]^{S_{nl}}$.

**3.5 Computing Res_l(q^2) on a specific operator**

Define the operator

$$ D_{n,X}(u; q, t) = \sum_r (-1)^{n-r} u^{n-r} D_{n,X}^r(q, t). \qquad (2) $$

Identify $e\mathcal{H}_{nl}(q^{-2l}, q^2)e$ with its image under the polynomial representation; in this identification, we now compute the image of a specific operator under $\mathrm{Res}_l$.

**Lemma 3.7.** We have the relation $\mathrm{Res}_l(q^2)(D_{nl,X}(q^{l+1}; q^{-2l}, q^2)) = \prod_{a=1}^l D_{n,X}(q^{2a}; q^{-2}, q^{2l})$.

*Proof.* Compute using the definition of $\mathrm{Res}_l(q^2)$ and the fact that $D_{nl,X}(q^{l+1}; q^{-2l}, q^2)$ is the image of $\prod_{i=1}^n \prod_{a=0}^{l-1}(Y_i^a - q^{l+1})$ in the polynomial representation. $\square$
# 4 Diagonal matrix elements in the Gelfand-Tsetlin basis

## 4.1 Factorization of matrix elements

For $\mu^1, \dots, \mu^n = \lambda$ so that $\tilde{\mu}^i \prec \dots \prec \tilde{\mu}^n = \tilde{\lambda}$ forms a Gelfand-Tsetlin pattern subordinate to $\tilde{\lambda}$, denote the pattern by $\tilde{\mu}$. Let $c(\tilde{\mu}, \lambda)$ be the diagonal matrix coefficient of $v_{\tilde{\mu}}$ in $\tilde{\Phi}_{\lambda}$. For $\mu \prec \lambda$, define $\text{gt}(\mu)$ by $\text{gt}(\mu)_i^l = \mu_i$ for $l < n$. Define $c(\mu, \lambda)$ to be the diagonal matrix coefficient $c(\text{gt}(\mu), \lambda)$ of $v_{\text{gt}(\mu)}$ in $\tilde{\Phi}_{\lambda}$.

We show that $\tilde{\Phi}_{\lambda}$ has non-zero diagonal matrix elements only on basis vectors indexed by patterns of the form $\tilde{\mu}$ and that these elements admit a level-by-level factorization.

**Lemma 4.1.** If $v_{\tilde{\mu}}$ is not of the form $v_{\tilde{\mu}}$, $v_{\tilde{\mu}}$ has zero diagonal matrix element in $\tilde{\Phi}_{\lambda}$.

*Proof.* For some $r < n$, we cannot write $\mu^r = \tilde{\tau}$ for any $\tau$. Let $U \subset W_{k-1}$ be the $U_q(\mathfrak{gl}_r)$-submodule of vectors of weight 0 for $q^{h_{r+1}}, \dots, q^{h_n}$ so that $U \simeq L_{(k-1)(r-1,-1,\dots,-1)}$ as a $U_q(\mathfrak{gl}_r)$-module. Let $\mu^r$ denote the truncation of $\mu^r$ so that $\mu_i^r = \mu_i^r$. Consider the pattern $\xi$ given by $\xi = \{\text{gt}(\mu^r) \prec \mu^{r+1} \prec \dots \prec \mu^{n-1} \prec \lambda\}$. Let $L_{\mu^r} \subset L_{\tilde{\lambda}}$ be the $U_q(\mathfrak{gl}_r)$-submodule with highest weight $\mu^r$ generated by $v_{\xi}$. By Proposition 1.5, the diagonal matrix element of $v_{\mu}$ lies in $L_{\mu^r} \otimes U$, hence is a multiple of the matrix element of $v_{\mu}$ in the induced $U_q(\mathfrak{gl}_r)$-intertwiner $L_{\mu^r} \to L_{\tilde{\lambda}} \to L_{\tilde{\lambda}} \otimes W_{k-1} \to L_{\mu^r} \otimes U$ given by projection onto $L_{\mu^r} \otimes U$. This intertwiner is zero because $\mu^r$ is not of the form $\mu^r = \tilde{\tau}$ for some $\tau$. $\square$

**Proposition 4.2.** If $\tilde{\mu} = \{\tilde{\mu}^1 \prec \dots \prec \tilde{\mu}^n = \tilde{\lambda}\}$ is subordinate to $\tilde{\lambda}$, then $c(\tilde{\mu}, \lambda) = \prod_{i=1}^{n-1} c(\mu^i, \mu^{i+1})$.

*Proof.* By induction on $n$, it suffices to check that $c(\tilde{\mu}, \lambda) = c(\mu, \lambda)c(\{\tilde{\mu}^1 \prec \dots \prec \tilde{\mu}^{n-1}\}, \mu^{n-1})$. Let $\mu = \mu^{n-1}$. By Proposition 1.5, the basis vector $v_{\tilde{\mu}}$ lies in the $U_q(\mathfrak{gl}_{n-1})$ submodule $L_{\tilde{\mu}} \subset L_{\tilde{\lambda}}$ with highest weight vector $v_{\text{gt}(\tilde{\mu})}$. Let $U \subset W_{k-1}$ be the $U_q(\mathfrak{gl}_{n-1})$-submodule consisting of elements of weight 0 under $q^{h_n}$. Consider the $U_q(\mathfrak{gl}_{n-1})$-intertwiner $\phi: L_{\tilde{\mu}} \to L_{\tilde{\mu}} \otimes U$ given by composing $\tilde{\Phi}_{\lambda}$ with the projection onto $L_{\tilde{\mu}} \otimes U$. The matrix element $c(\tilde{\mu}, \lambda)$ lies in $U$, hence is the matrix element of $v_{\tilde{\mu}}$ in $\phi$. Notice that $\phi$ maps the $U_q(\mathfrak{gl}_{n-1})$-highest weight vector $v_{\text{gt}(\tilde{\mu})}$ to $c(\mu, \lambda)v_{\text{gt}(\tilde{\mu})} \otimes w_{k-1} + (\text{l.o.t.})$ so that $\phi = c(\mu, \lambda)\tilde{\Phi}_{\mu}$ and the matrix element of $v_{\tilde{\mu}}$ is the desired $c(\mu, \lambda)c(\{\tilde{\mu}^1 \prec \dots \prec \tilde{\mu}^{n-1}\}, \mu^{n-1})$. $\square$

## 4.2 Matrix elements as applications of Macdonald difference operators

Our main technical result expresses matrix elements of $U_q(\mathfrak{gl}_n)$-intertwiners as the application of Macdonald difference operators on an explicit kernel. Define the elements $\Delta_1^{k-1}(\mu)$ and $\Delta_2^{k-1}(\mu)$ by

$$ \Delta_1^{k-1}(\mu) = \prod_{i<j} [\tilde{\mu}_i - \tilde{\mu}_j + (k-1)]_{k-1} \qquad \Delta_2^{k-1}(\mu) = \prod_{i<j} [\tilde{\mu}_i - \tilde{\mu}_j - 1]_{k-1} \qquad (3) $$

and the element $\Delta^{k-1}(\mu, \lambda)$ by

$$ \Delta^{k-1}(\mu, \lambda) = \prod_{i \le j} [\lambda_i - \mu_j + k(j-i) + k - 1]_{k-1} \prod_{i<j} [\mu_i - \lambda_j + k(j-i) - 1]_{k-1}. \qquad (4) $$

We compute the diagonal matrix elements of $\tilde{\Phi}_{\lambda}$, resulting in the following expression after manipulation. We omit the proof, which relies on the summation expression of [AS94] for matrix elements.
**Proposition 4.3.** Let $\mu' = \mu + (k-1)1$, and $\nu' = \nu + (k-1)1$. Then $c(\mu, \lambda)$ is given by

$$c(\mu, \lambda) = \frac{(-1)^{(n-1)(k-1)} q^{(n-1)k(k-1)}}{\Delta_2^{k-1}(\lambda) \Delta_1^{k-1}(\mu)} \sum_{\substack{\bar{\nu}' = \bar{\mu}' - (k-1)1 \\ \bar{\nu}' = \bar{\mu}' - (k-1)1}} \frac{(-1)^{|\bar{\nu}'| - |\bar{\mu}'|} q^{k(|\bar{\nu}'| - |\bar{\mu}'|)}}{\prod_i \frac{1}{[\bar{\nu}_i' - \bar{\mu}_i' + (k-1)][\bar{\mu}_i' - \bar{\nu}_i']!}} \\ \frac{\prod_{i<j} [\bar{\mu}_i' - \bar{\mu}_j' + k-1]_{2k-1} \prod_{i<j} [\bar{\nu}_i' - \bar{\nu}_j']}{\prod_{i<j} [\bar{\nu}_i' - \bar{\mu}_j' + (k-1)]_k [\bar{\mu}_i' - \bar{\nu}_j']_k} \prod_{i \le j} [\bar{\lambda}_i - \bar{\nu}_j' + (k-1)]_{k-1} \prod_{i<j} [\bar{\nu}_i' - \bar{\lambda}_j - 1]_{k-1}.$$

**Theorem 4.4.** Let $\mu' = \mu + (k-1)1$. The matrix element $c(\mu, \lambda)$ is given by

$$c(\mu, \lambda) = \frac{\prod_{a=1}^{k-1} D_{n-1, q^{2\mu}}(q^{2a}; q^{-2}, q^{2(k-1)}) \Delta^{k-1}(\mu', \lambda)}{\Delta_1^{k-1}(\mu) \Delta_2^{k-1}(\lambda)},$$

where $D_{n-1,q^{2\mu}}(q^{2a}; q^{-2}, q^{2(k-1)})$ was defined in (2).

*Proof.* By Lemma 3.7, we have $\mathrm{Res}_l(q^2)D_{(n-1)l,q^{2\mu}}(q^{l+1}; q^{-2l}, q^2) = \prod_{a=1}^l D_{n-1,q^{2\mu}}(q^{2a}; q^{-2}, q^{2l})$. Compute its action on

$$\mathrm{Res}_l(q^2) \prod_{a=0}^{l-1} \prod_{i \le j} [\bar{\lambda}_i - \bar{\mu}_j^a + k/2] \prod_{i<j} [\bar{\mu}_i^a - \bar{\lambda}_j - k/2] = \prod_{i \le j} [\bar{\lambda}_i - \bar{\mu}_j + k - 1]_{k-1} \prod_{i<j} [\bar{\mu}_i - \bar{\lambda}_j - 1]_{k-1}$$

in two ways and compare with Proposition 4.3. $\square$

<figure><img src="image_4.png" /></figure>

# 5 Proving Macdonald's branching rule

In this section, we prove the branching rule. We use Proposition 4.2 to express the trace in the Gelfand-Tsetlin basis as a sum of products. Inducting on $n$ and using the expression of Theorem 4.4 for $c(\mu, \lambda)$, we obtain an expression related to the desired by the summation by parts procedure of Proposition 2.3.

*of Theorem 1.7.* We induct on $n$. The base case is trivial because $P_\lambda(x_1; q^2, q^{2k}) = x_1^{|\lambda|}$. For the inductive step, by Lemma 4.1, it is enough to consider matrix elements for basis vectors $v_{\tilde{\mu}}$. By Proposition 4.2 and the inductive hypothesis, we thus have

$$\begin{align*}
\mathrm{Tr}(\tilde{\Phi}_{\lambda}^{n} x^h) &= \sum_{\tilde{\mu}^1 < \dots < \tilde{\mu}^{n-1} < \tilde{\lambda}} c(\mu^0, \mu^1) \cdots c(\mu^{n-1}, \lambda) \prod_i x_i^{(|\tilde{\mu}^i| - |\tilde{\mu}^{i-1}|)} \\
&= \sum_{\tilde{\mu} < \tilde{\lambda}} c(\mu, \lambda) x_n^{|\tilde{\lambda}| - |\tilde{\mu}| - (k-1)(n-1)} \sum_{\tilde{\mu}^1 < \dots < \tilde{\mu}^{n-2} < \tilde{\mu}} c(\mu^0, \mu^1) \cdots c(\mu^{n-2}, \mu^{n-1}) \prod_{i=1}^{n-1} x_i^{|\tilde{\mu}^i| - |\tilde{\mu}^{i-1}|} \\
&= \sum_{\tilde{\mu} < \tilde{\lambda}} c(\mu, \lambda) x_n^{|\tilde{\lambda}| - |\tilde{\mu}| - (k-1)(n-1)} P_{\mu}(\underline{x}; q^2, q^{2k}) \mathrm{Tr}(\tilde{\Phi}_0^{n-1} x^h),
\end{align*}$$

where $\underline{x} = (x_1, \dots, x_{n-1})$. By Corollary 2.5, we have that

$$\frac{\mathrm{Tr}(\tilde{\Phi}_0^{n-1} x^h)}{\mathrm{Tr}(\tilde{\Phi}_0^n x^h)} = (x_1 \cdots x_{n-1})^{k-1} x_n^{(k-1)(n-1)} \prod_{s=1}^{k-1} \prod_{i=1}^{n-1} (x_i - q^{2s} x_n)^{-1}.$$
We conclude that

$$
\begin{align*}
\frac{\operatorname{Tr}(\tilde{\Phi}_{\lambda}^{n} x^h)}{\operatorname{Tr}(\tilde{\Phi}_{0}^{n} x^h)} &= (x_1 \cdots x_{n-1})^{k-1} \prod_{s=1}^{k-1} \prod_{i=1}^{n-1} (x_i - q^{2s} x_n)^{-1} \sum_{\tilde{\mu} < \tilde{\lambda}} c(\mu, \lambda) x_n^{\lambda|\lambda|} P_{\mu}(\underline{x}/x_n; q^2, q^{2k}) \\
&= (x_1 \cdots x_{n-1})^{k-1} \prod_{s=1}^{k-1} \prod_{i=1}^{n-1} (x_i - q^{2s} x_n)^{-1} \sum_{\mu = \lambda_{\downarrow} - (k-1)\mathbf{1}}^{\lambda^{\uparrow}} c(\mu, \lambda) x_n^{\lambda|\lambda|} P_{\mu}(\underline{x}/x_n; q^2, q^{2k}) \\
&= x_n^{(k-1)(n-1)} \prod_{s=1}^{k-1} \prod_{i=1}^{n-1} (x_i - q^{2s} x_n)^{-1} \sum_{\mu' = \lambda_{\downarrow}}^{\lambda^{\uparrow} + (k-1)\mathbf{1}} c(\mu', (k-1)\mathbf{1}, \lambda) x_n^{\lambda|\lambda|} P_{\mu'}(\underline{x}/x_n; q^2, q^{2k}),
\end{align*}
$$

where $\lambda_\downarrow = (\lambda_2, \dots, \lambda_n)$ and $\lambda^\uparrow = (\lambda_1, \dots, \lambda_{n-1})$ are vectors of lower and upper indices for $\mu$ so that
$\sum_{\mu<\lambda} = \sum_{\mu=\lambda_\downarrow}^{\lambda^\uparrow}$ in the notation of (1). Note that $\tilde{\mu} < \tilde{\lambda}$ if and only if $\lambda_i \ge \mu_i \ge \lambda_{i+1} - (k-1)$. By the
expression for $c(\mu' - (k-1)\mathbf{1}, \lambda)$ given in Theorem 4.4, we obtain

$$
P_{\lambda}(x; q^2, q^{2k}) = \frac{\operatorname{Tr}(\tilde{\Phi}_{\lambda}^{n} x^h)}{\operatorname{Tr}(\tilde{\Phi}_{0}^{n} x^h)} = x_{n}^{(k-1)(n-1)} \prod_{s=1}^{k-1} \prod_{i=1}^{n-1} (x_i - q^{2s} x_n)^{-1} \sum_{\mu' = \lambda_{\downarrow}}^{\lambda^{\uparrow} + (k-1)\mathbf{1}} x_{n}^{\lambda|\lambda|} P_{\mu'}(\underline{x}/x_n; q^2, q^{2k}) \\
\frac{\prod_{a=1}^{k-1} D_{n-1, q^{2\tilde{\mu}}} (q^{2a}; q^{-2}, q^{2(k-1)}) \prod_{i \le j} [\tilde{\lambda}_i - \tilde{\mu}'_j + k - 1]_{k-1} \prod_{i < j} [\tilde{\mu}'_i - \tilde{\lambda}_j - 1]_{k-1}}{\prod_{i \le j} [\tilde{\mu}'_i - \tilde{\mu}'_j + k - 1]_{k-1} \prod_{i < j} [\tilde{\lambda}_i - \tilde{\lambda}_j - 1]_{k-1}}.
$$

Define the operator $\tilde{D}_{n-1,q^2\tilde{\mu}'}(q^{2a}; q^2, q^{2k}) = \sum_r (-1)^{n-1-r} q^{2a(n-1-r)} \tilde{D}_{n-1,q^2\tilde{\mu}'}^r(q^2, q^{2k})$, and note
that it is diagonalized on $P_{\mu'}(\underline{x}; q^2, q^{2k})$ by Proposition 2.2. Notice now that the function

$$
\prod_{i \le j} [\bar{\lambda}_i - \bar{\mu}'_j + k - 1]_{k-1} \prod_{i < j} [\bar{\mu}'_i - \bar{\lambda}_j - 1]_{k-1}
$$

is 0 for $\lambda_{i+1} - (k-1) \le \mu'_i < \lambda_{i+1}$ and $\lambda_i < \mu'_i \le \lambda_i + (k-1)$, so it is $(\lambda_\downarrow, \lambda^\uparrow, k-1)$-adapted.
Applying Proposition 2.3 to this function yields the desired

$$
\begin{align*}
P_{\lambda}(x; q^2, q^{2k}) &= x_n^{(k-1)(n-1)} \prod_{s=1}^{k-1} \prod_{i=1}^{n-1} (x_i - q^{2s} x_n)^{-1} \sum_{\mu' = \lambda_{\downarrow}}^{\lambda^{\uparrow}} x_n^{\lambda|\lambda|} \prod_{a=1}^{k-1} \tilde{D}_{n-1, q^{2\mu'}}(q^{2a}; q^2, q^{2k}) P_{\mu'}(\underline{x}/x_n; q^2, q^{2k}) \\
&\quad \frac{\prod_{i \ge j} [\bar{\lambda}_j - \bar{\mu}'_i + k - 1]_{k-1} \prod_{i < j} [\bar{\mu}'_i - \bar{\lambda}_j - 1]_{k-1}}{\prod_{i \le j} [\bar{\mu}'_i - \bar{\mu}'_j + k - 1]_{k-1} \prod_{i < j} [\bar{\lambda}_i - \bar{\lambda}_j - 1]_{k-1}} \\
&= x_n^{(k-1)(n-1)} \prod_{s=1}^{k-1} \prod_{i=1}^{n-1} \frac{x_i/x_n - q^{2s}}{x_i - q^{2s} x_n} \sum_{\mu' = \lambda_{\downarrow}}^{\lambda^{\uparrow}} x_n^{\lambda|\lambda|-|\mu'|} P_{\mu'}(\underline{x}; q^2, q^{2k}) \\
&\quad \frac{\prod_{i \ge j} [\bar{\lambda}_j - \bar{\mu}'_i + k - 1]_{k-1} \prod_{i < j} [\bar{\mu}'_i - \bar{\lambda}_j - 1]_{k-1}}{\prod_{i \le j} [\bar{\mu}'_i - \bar{\mu}'_j + k - 1]_{k-1} \prod_{i < j} [\bar{\lambda}_i - \bar{\lambda}_j - 1]_{k-1}} \\
&= \sum_{\mu' < \lambda} x_n^{\lambda|\lambda|-|\mu'|} P_{\mu'}(\underline{x}; q^2, q^{2k}) \frac{\prod_{i \ge j} [\bar{\lambda}_j - \bar{\mu}'_i + k - 1]_{k-1} \prod_{i < j} [\bar{\mu}'_i - \bar{\lambda}_j - 1]_{k-1}}{\prod_{i \le j} [\bar{\mu}'_i - \bar{\mu}'_j + k - 1]_{k-1} \prod_{i < j} [\bar{\lambda}_i - \bar{\lambda}_j - 1]_{k-1}}.
\end{align*}
\hfill \Box
$$
# Acknowledgements

The author thanks P. Etingof for helpful discussions.

## References

[AS94] S. Ališauskas and Y. Smirnov. Multiplicity-free $u_q(n)$ coupling coefficients. *J. Phys. A*, 27(17):5925-5939, 1994.

[BG13] A. Borodin and V. Gorin. General $\beta$ Jacobi corners process and the Gaussian free field. *Comm. Pure Appl. Math.*, 2013. to appear, http://arxiv.org/abs/1305.3627.

[Che05] I. Cherednik. *Double affine Hecke algebras*, volume 319 of *London Mathematical Society Lecture Note Series*. Cambridge University Press, Cambridge, 2005.

[EK94] P. Etingof and A. Kirillov, Jr. Macdonald's polynomials and representations of quantum groups. *Math. Res. Lett.*, 1(3):279-296, 1994.

[EK96] P. Etingof and A. Kirillov, Jr. Representation-theoretic proof of the inner product and symmetry identities for Macdonald's polynomials. *Compositio Math.*, 102(2):179-202, 1996.

[ES09] P. Etingof and E. Stoica. Unitary representations of rational Cherednik algebras. *Represent. Theory*, 13:349-370, 2009. With an appendix by Stephen Griffeth.

[Eti95] P. Etingof. Quantum integrable systems and representations of Lie algebras. *J. Math. Phys.*, 36(6):2636-2651, 1995.

[HO87] G. Heckman and E. Opdam. Root systems and hypergeometric functions. I. *Compositio Math.*, 64(3):329-352, 1987.

[Kas05] M. Kasatani. Subrepresentations in the polynomial representation of the double affine Hecke algebra of type $GL_n$ at $t^{k+1}q^{r-1} = 1$. *Int. Math. Res. Not.*, (28):1717-1742, 2005.

[Mac95] I. Macdonald. *Symmetric functions and Hall polynomials*. Oxford Mathematical Monographs. The Clarendon Press, Oxford University Press, New York, second edition, 1995. With contributions by A. Zelevinsky, Oxford Science Publications.

[Opd88] E. Opdam. Root systems and hypergeometric functions. IV. *Compositio Math.*, 67(2):191-209, 1988.

[Sun14a] Y. Sun. A new integral formula for Heckman-Opdam hypergeometric functions, 2014. http://arxiv.org/abs/1406.3772.

[Sun14b] Y. Sun. A representation-theoretic proof of the branching rule for Macdonald polynomials, 2014. http://arxiv.org/abs/1412.0714.

[UTS90] K. Ueno, T. Takebayashi, and Y. Shibukawa. Construction of Gel'fand-Tsetlin basis for $U_q(\text{gl}(N+1))$-modules. *Publ. Res. Inst. Math. Sci.*, 26(4):667-679, 1990.

[Ven14] V. Venkateswaran. On the expansion of certain vector-valued characters of $U_q(\text{gl}_n)$ with respect to the Gelfand-Tsetlin basis, 2014. http://arxiv.org/abs/1409.4079.