# Pieri rules for Schur functions in superspace

Miles Eli Jones, Luc Lapointe

► To cite this version:

Miles Eli Jones, Luc Lapointe. Pieri rules for Schur functions in superspace. 27th International Conference on Formal Power Series and Algebraic Combinatorics (FPSAC 2015), Jul 2015, Daejeon, South Korea. pp.97-108, 10.46298/dmtcs.2497. hal-01337836

HAL Id: hal-01337836

https://hal.science/hal-01337836

Submitted on 27 Jun 2016

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Pieri rules for Schur functions in superspace

Miles Eli Jones¹†and Luc Lapointe¹‡

¹Instituto de Matemática y Física, Universidad de Talca, Casilla 747, Talca, Chile

**Abstract.** The Schur functions in superspace $s_\Lambda$ and $\bar{s}_\Lambda$ are the limits $q = t = 0$ and $q = t = \infty$ respectively of the Macdonald polynomials in superspace. We present the elementary properties of the bases $s_\Lambda$ and $\bar{s}_\Lambda$ (which happen to be essentially dual) such as Pieri rules, dualities, monomial expansions, tableaux generating functions, and Cauchy identities.

**Résumé.** Les fonctions de Schur dans le superspace $s_\Lambda$ et $\bar{s}_\Lambda$ sont les limites $q = t = 0$ et $q = t = \infty$ respectivement des polynômes de Macdonald dans le superspace. Nous présentons les propriétés élémentaires des bases $s_\Lambda$ et $\bar{s}_\Lambda$ (qui sont essentiellement duales l'une de l'autre) tels que les règles de Pieri, la dualité, le développement en fonctions monomiales, les fonctions génératrices de tableaux et les identités de Cauchy.

**Keywords:** Schur functions, Key polynomials, symmetric functions in superspace

## 1 Introduction

An extension to superspace of the theory of symmetric functions was developed in [2, 6, 7]. In this extension, the polynomials $f(x, \theta)$, where $(x, \theta) = (x_1, \dots, x_N, \theta_1, \dots, \theta_N)$, not only depend on the usual commuting variables $x_1, \dots, x_N$ but also on the anticommuting variables $\theta_1, \dots, \theta_N$ $(\theta_i\theta_j = -\theta_j\theta_i$, and $\theta_i^2 = 0$). In this extended abstract, we are concerned with two natural generalizations to superspace of the Schur functions that arise as special limits of the Macdonald polynomials in superspace and whose combinatorics appears to be extremely rich.

The extension to superspace of the Macdonald polynomials, ${P_\Lambda(x, \theta; q, t)}_\Lambda$, is a basis of the ring $\mathbb{Q}(q, t)[x_1, \dots, x_N; \theta_1, \dots, \theta_N]^{S_N}$ of symmetric polynomials in superspace, where the superscript $S_N$ indicates that the elements of the ring are invariant under the diagonal action of the symmetric group $S_N$ (that is, invariant under the simultaneous interchange of $x_i \leftrightarrow x_j$ and $\theta_i \leftrightarrow \theta_j$, for any $i, j$). They are indexed by superpartitions $\Lambda$ and defined as the unique basis such that

1. $P_\Lambda(q, t) = m_\Lambda$ + smaller terms

2. $\langle\langle P_\Lambda(q, t), P_\Omega(q, t) \rangle\rangle_{q,t} = 0$ if $\Lambda \neq \Omega$

<footnote>†Email: mjones@inst-mat.utalca.cl</footnote>

<footnote>‡Email: lapointe@inst-mat.utalca.cl</footnote>
where the scalar product $\langle\langle \cdot, \cdot \rangle\rangle_{q,t}$ is given by

$$ \langle\langle p_{\Lambda}, p_{\Omega} \rangle\rangle_{q,t} = \delta_{\Lambda\Omega} q^{|\Lambda|^a} z_{\Lambda^s} \prod_i \frac{1 - q^{\Lambda_i^s}}{1 - t^{\Lambda_i^s}} \quad (1) $$

on the power sum symmetric functions in superspace (see Section 2 for all the relevant definitions). It was shown in [2] that even though the limits $q = t = 0$ and $q = t = \infty$ of this scalar product are degenerate and not well-defined respectively, the corresponding limits $s_{\Lambda} := P_{\Lambda}(0, 0)$ and $\bar{s}_{\Lambda} := P_{\Lambda}(\infty, \infty)$ of the Macdonald superpolynomials exist and are related to Key polynomials [8, 11]. As we will see, the rich combinatorics of these functions makes them the genuine extensions to superspace of the Schur functions. In comparison, the a priori more relevant limit $q = t = 1$ of the Macdonald polynomials in superspace, which corresponds to the limit $\alpha = 1$ of the Jack polynomials in superspace, does not seem to be very interesting from the combinatorial point of view.

The basis $s_{\Lambda}$ is especially relevant since it plays the role of the Schur functions in the generalization to superspace of the original Macdonald positivity conjectures [1]. To be more specific, let $J_{\Lambda}(q, t) = c_{\Lambda}(q, t)P_{\Lambda}(q, t)$ be the integral form of the Macdonald superpolynomials ($c_{\Lambda}(q, t)$ is a constant belonging to $\mathbb{Z}[q, t]$) and let $\psi(s_{\Lambda})$ be a certain plethystically transformed version of the function $s_{\Lambda}$ (see [1] for more details). Then the coefficients $K_{\Omega\Lambda}(q, t)$ appearing in

$$ J_{\Lambda}(q, t) = \sum_{\Omega} K_{\Omega\Lambda}(q, t) \psi(s_{\Omega}) \quad (2) $$

are conjectured to be polynomials in $q$ and $t$ with nonnegative integer coefficients (the conjecture is known to hold when the degree in the anticommuting variables is either zero, which corresponds to the usual Macdonald case, or sufficiently large [3]).

In this extended abstract we will present the elementary properties of the bases $s_{\Lambda}$ and $\bar{s}_{\Lambda}$ (which as we will see are essentially dual) such as Pieri rules, dualities, monomial expansions, tableaux generating functions, and Cauchy identities. It is important to note that the combinatorics of the bases $s_{\Lambda}$ and $\bar{s}_{\Lambda}$ was first studied in [4]. Our work stems in large part from a desire to develop the right framework to prove the conjectures therein, especially those concerning Pieri rules and tableaux generating functions.

We are confident that this work is only the tip of the iceberg and that deeper properties of the bases $s_{\Lambda}$ and $\bar{s}_{\Lambda}$ will be uncovered in the future, such as for instance a group-theoretical interpretation of the generalization to superspace of the Macdonald positivity conjecture. At the tableau level, we are hopeful that this work will eventually lead to a Robinson-Schensted-Knuth insertion algorithm in superspace, and ultimately to a charge statistic on tableaux that would solve the case $q = 0$ of (2).

The most technical parts of this work, which are not included in this extended abstract, are the proofs of the Pieri rules [9]. They rely on the correspondence between the Schur functions in superspace and Key polynomials [2] that allows to use the powerful machinery of divided differences [10]. Once the Pieri rules are assumed to hold, the remaining results follow somewhat easily from duality arguments or well-known techniques of symmetric function theory [9, 12, 13].
## 2 Symmetric function theory in superspace: basic definitions

A polynomial in superspace, or equivalently, a superpolynomial, is a polynomial in the usual $N$ variables $x_1, \dots, x_N$ and the $N$ anticommuting variables $\theta_1, \dots, \theta_N$ over a certain field, which will be taken in the remainder of this article to be $\mathbb{Q}$. A superpolynomial $P(x, \theta)$, with $x = (x_1, \dots, x_N)$ and $\theta = (\theta_1, \dots, \theta_N)$, is said to be symmetric if the following is satisfied:

$$P(x_1, \dots, x_N, \theta_1, \dots, \theta_N) = P(x_{\sigma(1)}, \dots, x_{\sigma(N)}, \theta_{\sigma(1)}, \dots, \theta_{\sigma(N)}) \quad \forall \sigma \in S_N \quad (3)$$

where $S_N$ is the symmetric group on $\{1, \dots, N\}$.

### 2.1 Superpartitions

Before defining superpartitions, we recall some definitions related to partitions [12]. A partition $\lambda = (\lambda_1, \lambda_2, \dots)$ of degree $|\lambda|$ is a vector of non-negative integers such that $\lambda_i \ge \lambda_{i+1}$ for $i = 1, 2, \dots$ and such that $\sum_i \lambda_i = |\lambda|$. Each partition $\lambda$ has an associated Ferrers diagram with $\lambda_i$ lattice squares in the $i^{th}$ row, from the top to bottom. Any lattice square in the Ferrers diagram is called a cell (or simply a square), where the cell $(i, j)$ is in the $i^{th}$ row and $j^{th}$ column of the diagram. The conjugate $\lambda'$ of a partition $\lambda$ is represented by the diagram obtained by reflecting $\lambda$ about the main diagonal. We say that the diagram $\mu$ is contained in $\lambda$, denoted $\mu \subseteq \lambda$, if $\mu_i \le \lambda_i$ for all $i$. Finally, $\lambda/\mu$ is a horizontal (resp. vertical) $n$-strip if $\mu \subseteq \lambda$, $|\lambda| - |\mu| = n$, and the skew diagram $\lambda/\mu$ does not have two cells in the same column (resp. row).

Symmetric superpolynomials are naturally indexed by superpartitions$^{(i)}$. A superpartition $\Lambda$ of degree $(n|m)$ is a pair $(\Lambda^{\oplus}, \Lambda^{*})$ of partitions $\Lambda^{\oplus}$ and $\Lambda^{*}$ such that:

1. $\Lambda^{*} \subseteq \Lambda^{\oplus}$;

2. the degree of $\Lambda^{*}$ is $n$;

3. the skew diagram $\Lambda^{\oplus}/\Lambda^{*}$ is both a horizontal and a vertical $m$-strip$^{(ii)}$

We refer to $m$ and $n$ respectively as the fermionic degree and total degree of $\Lambda$. Obviously, if $\Lambda^{\oplus} = \Lambda^{*} = \lambda$, then $\Lambda = (\lambda, \lambda)$ can be interpreted as the partition $\lambda$.

We will also need another characterization of a superpartition. A superpartition $\Lambda$ is a pair of partitions $(\Lambda^a; \Lambda^s) = (\Lambda_1, \dots, \Lambda_m; \Lambda_{m+1}, \dots, \Lambda_N)$, where $\Lambda^a$ is a partition with $m$ distinct parts (one of them possibly equal to zero), and $\Lambda^s$ is an ordinary partition (with possibly a string of zeros at the end). The correspondence between $(\Lambda^{\oplus}, \Lambda^{*})$ and $(\Lambda^a; \Lambda^s)$ is given explicitly as follows: given $(\Lambda^{\oplus}, \Lambda^{*})$, the parts of $\Lambda^a$ correspond to the parts of $\Lambda^*$ such that $\Lambda_i^{\oplus} \neq \Lambda_i^*$, while the parts of $\Lambda^s$ correspond to the parts of $\Lambda^*$ such that $\Lambda_i^{\oplus} = \Lambda_i^*$.

The conjugate of a superpartition $\Lambda = (\Lambda^{\oplus}, \Lambda^{*})$ is $\Lambda' = ((\Lambda^{\oplus})', (\Lambda^{*})')$. A diagrammatic representation of $\Lambda$ is given by the Ferrers diagram of $\Lambda^*$ with circles added in the cells corresponding to $\Lambda^{\oplus}/\Lambda^{*}$. For instance, if $\Lambda = (\Lambda^a; \Lambda^s) = (3, 1, 0; 2, 1)$, we have $\Lambda^{\oplus} = (4, 2, 2, 1, 1)$ and $\Lambda^{*} = (3, 2, 1, 1)$, so that

$$ \Lambda^{\oplus} : \quad \Lambda^{*} : \quad \Lambda : \quad \Lambda' : \quad (4) $$

<footnote>(i) Superpartitions correspond, after a trivial bijection, to the overpartitions studied in [5].</footnote>

<footnote>(ii) Such diagrams are sometimes called $m$-rook strips.</footnote>
where the last diagram illustrates the conjugation operation that corresponds, as usual, to replacing rows by columns.

The extension of the dominance ordering to superpartitions is

$$ \Omega \leq \Lambda \quad \text{iff} \quad \deg(\Lambda) = \deg(\Omega), \quad \Omega^* \leq \Lambda^* \quad \text{and} \quad \Omega^{\otimes} \leq \Lambda^{\otimes} \qquad (5) $$

where the order on partitions is the usual dominance ordering

$$ \mu \leq \lambda \quad \text{iff} \quad |\mu| = |\lambda| \quad \text{and} \quad \mu_1 + \dots + \mu_i \leq \lambda_1 + \dots + \lambda_i \quad \forall i. \qquad (6) $$

## 2.2 Simple bases

Four simple bases of the space of symmetric polynomials in superspace will be particularly relevant to our work [6]:

1. the extension of the monomial symmetric functions, $m_\Lambda$, defined by

$$ m_{\Lambda} = \sum_{\sigma \in S_N} \theta_{\sigma(1)} \cdots \theta_{\sigma(m)} x_{\sigma(1)}^{\Lambda_1} \cdots x_{\sigma(N)}^{\Lambda_N}, \qquad (7) $$

where the sum is over the permutations of $\{1, \dots, N\}$ that produce distinct terms, and where the entries of $(\Lambda_1, \dots, \Lambda_N)$ are those of $\Lambda = (\Lambda^a; \Lambda^s) = (\Lambda_1, \dots, \Lambda_m; \Lambda_{m+1}, \dots, \Lambda_N)$ (the semicolon is removed);

2. the generalization of the power-sum symmetric functions $p_\Lambda = \tilde{p}_{\Lambda_1} \cdots \tilde{p}_{\Lambda_m} p_{\Lambda_{m+1}} \cdots p_{\Lambda_\ell}$,

$$ \text{where } \tilde{p}_k = \sum_{i=1}^{N} \theta_i x_i^k \quad \text{and} \quad p_r = \sum_{i=1}^{N} x_i^r, \quad \text{for } k \geq 0, r \geq 1; \qquad (8) $$

3. the generalization of the elementary symmetric functions $e_\Lambda = \tilde{e}_{\Lambda_1} \cdots \tilde{e}_{\Lambda_m} e_{\Lambda_{m+1}} \cdots e_{\Lambda_\ell}$,

$$ \text{where } \tilde{e}_k = m_{(0;1^k)} \quad \text{and} \quad e_r = m_{(\emptyset;1^r)}, \quad \text{for } k \geq 0, r \geq 1; \qquad (9) $$

4. the generalization of the homogeneous symmetric functions $h_\Lambda = \tilde{h}_{\Lambda_1} \cdots \tilde{h}_{\Lambda_m} h_{\Lambda_{m+1}} \cdots h_{\Lambda_\ell}$,

$$ \text{where } \tilde{h}_k = \sum_{\substack{\deg(\Lambda^s)=k \\ \deg(\Lambda^a)=1}} (\Lambda_1+1)m_\Lambda \quad \text{and} \quad h_r = \sum_{\substack{\deg(\Lambda^s)=r \\ \deg(\Lambda^a)=0}} m_\Lambda, \quad \text{for } k \geq 0, r \geq 1 \qquad (10) $$

Observe that when $\Lambda = (\emptyset; \lambda)$, we have that $m_\Lambda = m_\lambda$, $p_\Lambda = p_\lambda$, $e_\Lambda = e_\lambda$ and $h_\Lambda = h_\lambda$ are respectively the usual monomial, power-sum, elementary and homogeneous symmetric functions. Also note that if we define the operator $d = \theta_1 \partial/\partial x_1 + \dots + \theta_N \partial/\partial x_N$, we have

$$ (k+1) \tilde{p}_k = d(p_{k+1}), \quad \tilde{e}_k = d(e_{k+1}) \quad \text{and} \quad \tilde{h}_k = d(h_{k+1}) \qquad (11) $$

that is, the new generators in the superspace versions of the bases can be obtained from acting with $d$ on the generators of the usual symmetric function versions.
## 2.3 Scalar product and duality

The relevant scalar product in this article is the specialization $q = t = 1$ of the scalar product (1)(iii), that is,

$$ \langle\langle p_{\Lambda}, p_{\Omega} \rangle\rangle = \delta_{\Lambda\Omega} z_{\Lambda}^{*} \qquad (12) $$

where, as usual, $z_{\Lambda} = 1^{n_{\Lambda}(1)}n_{\Lambda}(1)!2^{n_{\Lambda}(2)}n_{\Lambda}(2)!\cdots$ with $n_{\Lambda}(i)$ the number of parts of $\lambda$ equal to $i$. This scalar product is such that

$$ \langle\langle h_{\Lambda}, m_{\Omega} \rangle\rangle = \delta_{\Lambda\Omega} \qquad (13) $$

The homomorphism $\omega$ defined by

$$ \omega(\tilde{p}_r) = (-1)^r \tilde{p}_r \quad \text{and} \quad \omega(p_r) = (-1)^{r-1} p_r \qquad (14) $$

which is obviously an involution and an isometry of the scalar product $\langle\langle \cdot, \cdot \rangle\rangle$, is also such that

$$ \omega(h_{\Lambda}) = e_{\Lambda} \qquad (15) $$

The Schur functions in superspace $s_{\Lambda}$ and $\bar{s}_{\Lambda}$ were defined in the introduction as the special limits $q = t = 0$ and $q = t = \infty$ respectively of the Macdonald polynomials in superspace. Remarkably, the functions $s_{\Lambda}$ and $\bar{s}_{\Lambda}$ are essentially dual with respect to our scalar product [2].

**Proposition 1** Let $s_{\Lambda}^{*}$ and $\bar{s}_{\Lambda}^{*}$ be the bases dual to the bases $s_{\Lambda}$ and $\bar{s}_{\Lambda}$ respectively, that is, let $s_{\Lambda}^{*}$ and $\bar{s}_{\Lambda}^{*}$ be such that

$$ \langle\langle s_{\Lambda}^{*}, s_{\Omega} \rangle\rangle = \langle\langle \bar{s}_{\Lambda}^{*}, \bar{s}_{\Omega} \rangle\rangle = \delta_{\Lambda\Omega} \qquad (16) $$

Then

$$ s_{\Lambda}^{*} = (-1)^{\binom{m}{2}} \omega \bar{s}_{\Lambda'} \quad \text{and} \quad \bar{s}_{\Lambda}^{*} = (-1)^{\binom{m}{2}} \omega s_{\Lambda'} \qquad (17) $$

where $m$ is the fermionic degree of $\Lambda$.

When $m = 0$, we have $s_{\Lambda} = s_{(\emptyset;\lambda)} = s_{\lambda}$ and $\bar{s}_{\Lambda} = \bar{s}_{(\emptyset;\lambda)} = s_{\lambda}$. In this case the proposition is simply stating that the dual of the Schur function basis $s_{\lambda}$ with respect to the Hall scalar product is the basis $s_{\lambda} = \omega s_{\lambda'}$, which is an elementary result [12].

# 3 Pieri rules

Pieri rules for the multiplication of a Schur functions in superspace $s_{\Lambda}$ or $\bar{s}_{\Lambda}$ by $e_{\ell}$ and $\tilde{e}_{\ell}$ were established in [9] using Key polynomials. By Proposition 1 and (15), these Pieri rules are the transposed of those corresponding to the multiplication of $s_{\Lambda}^{*}$ or $\bar{s}_{\Lambda}^{*}$ by $h_{\ell}$ and $\tilde{h}_{\ell}$ which will be presented in this section.

## 3.1 Pieri rules for $s_{\Lambda}^{*}$

**Theorem 2** Let $\Lambda$ be a superpartition of fermionic degree $m$. Then, for $k \ge 1$ and $\ell \ge 0$, we have

$$ s_{\Lambda}^{*} h_{k} = \sum_{\Omega} s_{\Omega}^{*} \quad \text{and} \quad s_{\Lambda}^{*} \tilde{h}_{\ell} = \sum_{\Omega} (-1)^{\#(\Omega, \Lambda)} s_{\Omega}^{*} \qquad (18) $$

where the sum is over all superpartitions $\Omega$ of fermionic degree $m$ ($h_k$ case) or $m + 1$ ($\tilde{h}_{\ell}$ case) such that

<footnote>(iii) The scalar product (1) differs from that of [2] by a sign depending on the fermionic degree.</footnote>
1. $\Omega^*/\Lambda^*$ is a horizontal $k$-strip ($h_k$ case) or $\Omega^{\oplus}/\Lambda^{\oplus}$ is a horizontal $l$-strip ($\tilde{h}_l$ case).

2. There exists in the $\tilde{h}_l$ case a (unique) circle of $\Omega$ (the new circle), let's say in column $c$, such that

* column $c$ does not contain any cell of $\Omega^*/\Lambda^*$.

* there is a cell of $\Omega^*/\Lambda^*$ in every column strictly to the left of column $c$.

3. If $\tilde{\Omega}$ is $\Omega$ without its new circle ($\tilde{\Omega} = \Omega$ when there is no new circle), then the $i$-th circle (starting from below) of $\tilde{\Omega}$ is either in the same row as the $i$-th circle of $\Lambda$ (if $\Omega^*/\Lambda^*$ does not contain a cell in that row) or one row below that of the $i$-th circle of $\Lambda$ (if $\Omega^*/\Lambda^*$ contains a cell in the row of the $i$-th circle of $\Lambda$). In the latter case, we say that the circle was moved.

and where $\#(\Omega, \Lambda)$ is the number of circles in $\Omega$ below the new circle.

We illustrate the rules by giving the expansions of $s^*_{(4,1,0;2)} h_3$ and $s^*_{(4,1;3)} \tilde{h}_3$.

<figure><img src="image_2.png" /></figure>

$$s^*_{(4,1,0;2)} h_3 = s^*_{(2,1,0;7)} + s^*_{(3,1,0;6)} + s^*_{(2,1,0;6,1)} + s^*_{(4,1,0;5)} + s^*_{(3,1,0;5,1)} + s^*_{(2,1,0;5,2)} + s^*_{(4,1,0;3,2)}$$

To generate all $\Omega$ described in Theorem 2, draw all possible horizontal strips on $\Lambda^*$. For each partition obtained this way, start from the bottom row and proceed row by row. If a new square occupies the place of a circle in $\Lambda^*$, move the circle to the next row and slide it to the first available column. If there already is a circle occupying the row then the resulting diagram is not a superpartition and should be discarded. Note that this happens twice in our example.

<figure><img src="image_3.png" /></figure>

$$s^*_{(4,1;3)} \tilde{h}_3 = s^*_{(3,1,0;7)} + s^*_{(4,1,0;6)} - s^*_{(3,2,1;5)} - s^*_{(4,2,1;4)} - s^*_{(4,3,1;3)}$$

This rule is very similar to the previous one but in this case but keep in mind that every column to the left of the new circle must have a new box. Also, multiply by $(-1)$ for every circle below the new circle.

## 3.2 Pieri rules for $\bar{s}_{\Lambda}^*$

**Theorem 3** Let $\Lambda$ be a superpartition of fermionic degree $m$. Then, for $k \ge 1$ and $l \ge 0$, we have

$$\bar{s}_{\Lambda}^{*} h_{k} = \sum_{\Omega} \bar{s}_{\Omega}^{*} \quad \text{and} \quad \bar{s}_{\Lambda}^{*} \tilde{h}_{l} = \sum_{\Omega} (-1)^{\#(\Omega, \Lambda)} \bar{s}_{\Omega}^{*} \qquad (19)$$

where the sum is over all superpartitions $\Omega$ of fermionic degree $m$ ($h_k$ case) or $m + 1$ ($\tilde{h}_l$ case) such that
* $\Omega^*/\Lambda^*$ is a horizontal k-strip ($h_k$ case) or $\Omega^*\diag\Lambda^*$ is a horizontal $(\ell+1)$-strip whose right-most cell (the new circle) belongs to $\Omega^*\diag\Lambda^*$ ($h_\ell$ case).

* the i-th circles (starting from the new circle and going down, or from the first row and going down if there is no new circle) of $\Omega$ and $\Lambda$ are either in the same row or the same column. In the case where the i-th circles are in the same row $r$, the circle in column $r$ of $\Omega$ cannot be located passed row $r-1$ of $\Lambda$ (if $r=1$ the condition does not apply).

and where $\#(\Omega, \Lambda)$ is again the number of circles in $\Omega$ below the new circle.

We illustrate this time the rules by giving the expansion of $\bar{s}^*_{(4,1;5,4)} h_3$ and $\bar{s}^*_{(4,1;5,4)} \tilde{h}_2$.

<figure><img src="image_4.png" /></figure>

$$ \bar{s}^*_{(4,1;5,4)} h_3 = \bar{s}^*_{(4,1;8,4)} + \bar{s}^*_{(4,1;7,5)} + \bar{s}^*_{(4,2;7,4)} + \bar{s}^*_{(4,1;7,4,1)} + \bar{s}^*_{(4,2;6,5)} + \bar{s}^*_{(4,1;6,5,1)} \\ + \bar{s}^*_{(4,3;6,4)} + \bar{s}^*_{(4,2;6,4,1)} + \bar{s}^*_{(4,1;6,4,2)} + \bar{s}^*_{(4,3;5,5)} + \bar{s}^*_{(4,3;5,4,1)} + \bar{s}^*_{(4,1;5,4,3)} $$

Notice here that the circle can be pushed down if there is room or pushed to the right. It cannot be pushed to the right farther than the original row above it.

<figure><img src="image_5.png" /></figure>

$$ \bar{s}^*_{(4,1;5,4)} \tilde{h}_3 = \bar{s}^*_{(7,4,1;4)} + \bar{s}^*_{(6,4,1;5)} + \bar{s}^*_{(6,4,2;4)} + \bar{s}^*_{(6,4,1;4,1)} + \bar{s}^*_{(5,4,2;5)} + \bar{s}^*_{(5,4,1;5,1)} \\ + \bar{s}^*_{(5,4,3;4)} + \bar{s}^*_{(5,4,2;4,1)} + \bar{s}^*_{(5,4,1;4,2)} - \bar{s}^*_{(4,2,1;5,4)} $$

To generate these superpartitions, start with the previous rule and replace the new box that is farthest to the right with the new circle. Discard any diagrams that have two circles in the same row or column.

# 4 Kostka coefficients in superspace

Define the Kostka coefficients in superspace $K_{\Omega\Lambda}$ and $\bar{K}_{\Omega\Lambda}$ to be respectively such that

$$ h_{\Lambda} = \sum_{\Omega} K_{\Omega\Lambda} \bar{s}_{\Omega}^{*} \quad \text{and} \quad h_{\Lambda} = \sum_{\Omega} \bar{K}_{\Omega\Lambda} s_{\Omega}^{*} \qquad (20) $$

As expected, the Kostka coefficients in superspace give the monomial expansion of the Schur functions in superspace.
**Proposition 4** We have

$$s_{\Lambda} = \sum_{\Omega \le \Lambda} \bar{K}_{\Lambda\Omega} m_{\Omega} \quad \text{and} \quad \bar{s}_{\Lambda} = \sum_{\Omega \le \Lambda} K_{\Lambda\Omega} m_{\Omega} \qquad (21)$$

The triangularity in the previous equation follows from the fact that the Schur functions in superspace are special cases of Macdonald polynomials in superspace, which are triangular by construction. The Kostka coefficients in superspace turn out to be nonnegative integers, which is relatively surprising given that signs show up in the Pieri rules.

**Proposition 5** The Kostka coefficients in superspace $K_{\Omega\Lambda}$ and $\bar{K}_{\Omega\Lambda}$ are nonnegative integers.

A combinatorial interpretation in terms of tableaux for the coefficients $\bar{K}_{\Lambda\Omega}$ and $K_{\Lambda\Omega}$ will be given in Proposition 8 and Proposition 9 respectively. We should note that a somewhat different combinatorial interpretation for $\bar{K}_{\Lambda\Omega}$ and $K_{\Lambda\Omega}$ was conjectured in [4].

# 5 Duality

We will now see that there is a natural duality that relates $s_{\Lambda}$ and $s_{\Lambda'}$. Unexpectedly, for reasons we will see later on, no such simple duality exists in the case of $\bar{s}_{\Lambda}$.

Applying $\omega$ on the first formula of (20), we obtain from (15) and Proposition 1 that

$$e_{\Lambda} = (-1)^{\binom{m}{2}} \sum_{\Omega} K_{\Omega\Lambda} s_{\Omega'} \qquad (22)$$

Now, let $H_{\Lambda} = \tilde{p}_{\Lambda^a} h_{\Lambda^s}$, that is, $H_{\Lambda}$ is $h_{\Lambda}$ with the superspace generators $\tilde{h}_r$ replaced by $\tilde{p}_r$. It is shown in [9] that

$$H_{\Lambda} = \sum_{\Omega} K_{\Omega\Lambda} s_{\Omega} \qquad (23)$$

where we stress that the coefficients $K_{\Omega\Lambda}$ are exactly those that appear in the previous equation. This suggests a natural duality between the $H_{\Lambda}$ and $e_{\Lambda}$ bases. In effect, let $\varphi$ be the homomorphism defined by

$$\varphi(\tilde{p}_r) = \tilde{e}_r \quad \text{and} \quad \varphi(h_r) = e_r \qquad (24)$$

that is, such that $\varphi(H_{\Lambda}) = e_{\Lambda}$. By (22) and (23), the following result is essentially immediate:

**Corollary 6** We have

$$\varphi(s_{\Lambda}) = (-1)^{\binom{m}{2}} s_{\Lambda'} \qquad (25)$$

Moreover, the homomorphism $\varphi$ is an involution, that is, $\varphi \circ \varphi$ is the identity.

Using Corollary 6, the Pieri rules for the multiplication of $s_{\Lambda}$ by $\tilde{p}_{\ell}$ and $h_{\ell}$ are identical to those for the multiplication of $\bar{s}_{\Lambda}^*$ by $\tilde{h}_{\ell}$ and $h_{\ell}$ respectively. We should note that these Pieri rules were conjectured to hold in [4].

**Corollary 7** We have, for $k \ge 1$ and $\ell \ge 0$,

$$s_{\Lambda} h_k = \sum_{\Omega} s_{\Omega} \quad \text{and} \quad s_{\Lambda} \tilde{p}_{\ell} = \sum_{\Omega} (-1)^{\#(\Omega, \Lambda)} s_{\Omega} \qquad (26)$$

where $\#(\Omega, \Lambda)$ is as usual the number of circles of $\Omega$ below the new circle and where the sums run over superpartitions $\Omega$ obeying the conditions of Theorem 3.
It is immediate from Proposition 1 that the linear map $\omega \circ \varphi^+ \circ \omega$ sends $\bar{s}_\Lambda$ to $\bar{s}_{\Lambda'}$ (up to a sign), where $\varphi^+$ is the adjoint of $\varphi$ with respect to the scalar product (12). But $\varphi^+$ turns out not to be a homomorphism, and as such the duality between $\bar{s}_\Lambda$ and $\bar{s}_{\Lambda'}$ is less natural (for instance, it does not lead to any analog of Corollary 7).

# 6 Tableaux

In this section, we show that the Schur functions in superspace are generating series of certain types of tableaux.

We will refer to $\{\bar{0}, \bar{1}, \bar{2}, \bar{3}, \dots\}$ as the set of fermionic nonnegative integers. In this spirit, we will also refer to the set of nonnegative integers $\{0, 1, 2, 3, \dots\}$ as the set of bosonic nonnegative integers. For $\alpha \in \{0, \bar{0}, 1, \bar{1}, 2, \bar{2}, \dots\}$, we will say that type($\alpha$) is bosonic or fermionic depending on whether the corresponding integer is fermionic or bosonic. Finally, define

$$|\alpha| = \begin{cases} a & \text{if } \alpha = \bar{a} \text{ is fermionic} \\ a & \text{if } \alpha = a \text{ is bosonic} \end{cases} \qquad (27)$$

## 6.1 s-tableaux

By (20), the tableaux needed to represent the Schur function in superspace $s_\Lambda$ are those stemming from the Pieri rules associated to the multiplication of $s_\Lambda^*$ by $h_r$ or $\bar{h}_r$ given in Theorem 2. We say that the sequence $\Omega = \Lambda_{(0)}, \Lambda_{(1)}, \dots, \Lambda_{(n)} = \Lambda$ is an s-tableau of shape $\Lambda/\Omega$ and weight $(\alpha_1, \dots, \alpha_n)$, where $\alpha_i \in \{0, \bar{0}, 1, \bar{1}, 2, \bar{2}, \dots\}$, if $\Omega = \Lambda_{(i)}$ and $\Lambda = \Lambda_{(i-1)}$ obey the conditions of Theorem 2 with $k = \alpha_i$ whenever $\alpha_i$ is bosonic or with $\ell = |\alpha_i|$ whenever $\alpha_i$ is fermionic. An s-tableau can be represented by a diagram constructed recursively in the following way:

1. the cells of $\Lambda_{(i)}^*/\Lambda_{(i-1)}^*$, which form a horizontal strip, are filled with the letter $i$. In the fermionic case, the new circle is also filled with a letter $i$.

2. the circles of $\Lambda_{(i-1)}$ that are moved a row below keep their fillings.

The sign of an s-tableau $T$, which corresponds to the product of the signs appearing in the fermionic horizontal strips, can be extracted quite efficiently from an s-tableau. Read the fillings of the circles from top to bottom to obtain a word (without repetition): the sign of the tableau $T$ is then equal to $(-1)^{\text{inv}(T)}$, where $\text{inv}(T)$ is the number of inversions of the word.

Given a diagram of an s-tableau, we define the path of a given circle (filled let's say with letter $i$) in the following way. Let $c$ be the leftmost column that does not contain a square (a cell of $\Omega^*$) filled with an $i$. The path starts in the position of the smallest entry larger than $i$ (let's say $j$) in column $c$. The path then moves to the smallest entry (let's say $k$) larger than $j$ in the row below (if there are many such $k$'s the path goes through the leftmost such $k$). We continue this way until we reach the row below that of the circle filled with an $i$.

It is important to realize that a tableau can be identified with its diagram given that the sequence $\Omega = \Lambda_{(0)}, \Lambda_{(1)}, \dots, \Lambda_{(n)} = \Lambda$ can be recovered from the diagram. We obtain the diagram corresponding to $\Omega = \Lambda_{(0)}, \Lambda_{(1)}, \dots, \Lambda_{(n-1)}$ by removing the letters $n$ from the diagram (including, possibly, the circled one), and by moving the remaining circle one row above according to the following rule. A circle (filled let's say with letter $i$) in a given row $r$ is moved to row $r-1$ if there is an $n$ in row $r-1$ that belongs to its path. Otherwise the circle in row $r$ stays in its position.
Consider the tableau $\begin{array}{c} \begin{array}{|c|c|c|} \hline 1 & 2 & 2 & 3 \\ 2 & 4 & 6 \\ 3 & 5 & 4 & 4 \\ 4 & 6 & & \\ \end{array} \\ \begin{array}{c} 2 \\ \end{array} \}$ . The path for the $\begin{array}{c} 2 \\ \end{array}$ is $\begin{array}{c} 1 & 2 & 2 & 3 \\ 2 & 4 & 6 \\ 3 & 5 & 4 & 4 \\ 4 & 6 & & \\ \end{array}$ while the path for $\begin{array}{c} 4 \\ \end{array}$ is $\begin{array}{c} 1 & 2 & 2 & 3 \\ 2 & 4 & 6 \\ 3 & 5 & 4 & 4 \\ 4 & 6 & & \\ \end{array}$ .

The sequence of superpartitions associated to that s-tableau can then be recovered by stripping successively the tableau of its largest letter:

<figure><img src="image_6.png" /></figure>

Now, define

$$ (x\theta)^T = \prod_i x_i^{|\alpha_i|} \prod_{j:\text{type}(\alpha_j)=\text{fermionic}} \theta_j \qquad (28) $$

if $T$ is of weight $(\alpha_1, \dots, \alpha_n)$. We stress that the product over anticommuting variables is ordered from left to right over increasing indices. For instance, for the $s$-tableau above, we have $(-1)^{\text{inv}(T)}(x\theta)^T = -x_1 x_2^3 x_3^2 x_4^2 x_5 x_6^2 \theta_2 \theta_4$.

**Proposition 8** *We have that*

$$ s_{\Lambda} = \sum_{T} (-1)^{\text{inv}(T)} (x\theta)^T \qquad (29) $$

where the sum is over all s-tableaux $T$ of shape $\Lambda$. Hence, from (21), $\bar{K}_{\Lambda\Omega} = \sum_{T} (-1)^{\text{inv}(T)}$, where the sum is over all s-tableaux $T$ of shape $\Lambda$ and weight $(\bar{\Omega}_1, \dots, \bar{\Omega}_m, \Omega_{m+1}, \dots, \Omega_N)$ (recall that $\Omega = (\Omega_1, \dots, \Omega_m; \Omega_{m+1}, \dots, \Omega_N)$).

We thus obtain the monomial expansion of $s_{(3,1;2,1,1)}$ by listing every filling of the shape $(3, 1; 2, 1, 1)$ whose weight corresponds to a superpartition:

<figure><img src="image_7.png" /></figure>

Therefore, $s_{(3,1;2,1,1)} = 3m_{(3,1;1,1,1,1)} + m_{(3,1;2,1,1)}$.

## 6.2 $\bar{s}$-tableaux

By (20), the tableaux needed to represent the Schur function in superspace $\bar{s}_{\Lambda}$ are this time those stemming from the Pieri rules associated to the multiplication of $\bar{s}_{\Lambda}^*$ by $h_r$ or $\tilde{h}_r$ given in Theorem 3. We say that the sequence $\Omega = \Lambda_{(0)}, \Lambda_{(1)}, \dots, \Lambda_{(n)} = \Lambda$ is an $\bar{s}$-tableau of shape $\Lambda/\Omega$ and weight $(\alpha_1, \dots, \alpha_n)$, where $\alpha_i \in \{0, \bar{0}, 1, \bar{1}, 2, \bar{2}, \dots\}$, if $\Omega = \Lambda_{(i)}$ and $\Lambda = \Lambda_{(i-1)}$ obey the conditions of Theorem 3 with $k = \alpha_i$ whenever $\alpha_i$ is bosonic or with $\ell = |\alpha_i|$ whenever $\alpha_i$ is fermionic. An $\bar{s}$-tableau can be represented by a diagram constructed recursively in the following way:

1. the cells of $\Lambda_{(i)}^* / \Lambda_{(i-1)}^*$ are filled with the letter $i$. In the fermionic case, the new circle is also filled with a letter $i$

2. the circles of $\Lambda_{(i-1)}$ that moved along a column or a row keep their fillings.
Acknowledgements

The authors would like to thank O. Blondeau-Fournier, P. Desrosiers and P. Mathieu for very fruitful discussions. This work was supported by FONDECYT (Fondo Nacional de Desarrollo Científico y Tecnológico de Chile) postdoctoral grant #3130631 and regular grant #1130696.

References

[1] O. Blondeau-Fournier, P. Desrosiers, L. Lapointe and P. Mathieu, *Macdonald polynomials in superspace: conjectural definition and positivity conjectures*, Lett. Math. Phys. **10**, 27–47 (2012).

[2] O. Blondeau-Fournier, P. Desrosiers, L. Lapointe and P. Mathieu, *Macdonald polynomials in superspace as eigenfunctions of commuting operators*, Journal of Combinatorics **3**, no. 3, 495–562 (2012).

[3] O. Blondeau-Fournier, L. Lapointe and P. Mathieu, *Double Macdonald polynomials as the stable limit of Macdonald superpolynomials*, Journal of Algebraic Combinatorics, DOI 10.1007/s10801-014-0541-6 (2014).

[4] O. Blondeau-Fournier and P. Mathieu, *Schur superpolynomials: combinatorial definition and Pieri rule*, arXiv:1408.2807.

[5] S. Corteel and J. Lovejoy, *Overpartitions*, Trans. Amer. Math. Soc. 356 (2004), 1623–1635.

[6] P. Desrosiers, L. Lapointe and P. Mathieu, *Classical symmetric functions in superspace*, J. Alg. Combin. **24**, 209–238 (2006).

[7] P. Desrosiers, L. Lapointe and P. Mathieu, *Orthogonality of Jack polynomials in superspace*, Adv. Math. **212**, 361–388 (2007).

[8] B. Ion, *Demazure characters and non-symmetric Macdonald polynomials*, Duke Math. J. **116** (2003), 299–318.

[9] M. Jones and L. Lapointe, in preparation.

[10] A. Lascoux, *Symmetric functions and combinatorial operators on polynomials*, CBMS **99**, AMS 2003.

[11] A. Lascoux and M.P. Schützenberger, *Keys and standard bases*, Invariant Theory and Tableaux, IMA Volumes in Math and its Applications **19**, Springer, 1988, p. 125–144.

[12] I. G. Macdonald, *Symmetric functions and Hall polynomials*, 2nd ed., Clarendon Press, 1995.

[13] R. P. Stanley, Enumerative Combinatorics, Vol. 2, Cambridge Studies in Advanced Mathematics Vol. 62, Cambridge University Press, 1999.