<figure><img src="image_1.png" /></figure>

HAL
open science

# Hua operators, Poisson transform and relative discrete series on line bundles over bounded symmetric domains

Khalid Koufany, Genkai Zhang

► To cite this version:

Khalid Koufany, Genkai Zhang. Hua operators, Poisson transform and relative discrete series on line bundles over bounded symmetric domains. Journal of Functional Analysis, 2012, 262 (9), pp.4140-4159. 10.1016/j.jfa.2012.02.012. hal-01284127

HAL Id: hal-01284127

https://hal.science/hal-01284127

Submitted on 7 Mar 2016

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
HUA OPERATORS, POISSON TRANSFORM AND
RELATIVE DISCRETE SERIES ON LINE BUNDLE
OVER BOUNDED SYMMETRIC DOMAINS

KHALID KOUFANY AND GENKAI ZHANG

ABSTRACT. Let $\Omega = G/K$ be a bounded symmetric domain and $S = K/L$ its Shilov boundary. We consider the action of $G$ on sections of a homogeneous line bundle over $\Omega$ and the corresponding eigenspaces of $G$-invariant differential operators. The Poisson transform maps hyperfunctions on $S$ to the eigenspaces. We characterize the image in terms of twisted Hua operators. For some special parameters the Poisson transform is of Szegö type whose image is in a relative discrete series; we compute the corresponding elements in the discrete series.

April 2011

# 1. INTRODUCTION

Let $X = G/K$ be a Riemannian symmetric space of non compact type. It is known that the Poisson transform maps certain parabolically induced representation spaces into null spaces of some systems of differential equations. For a minimal parabolic subgroup $P_{\min} \subset G$, Kashiwara et al. proved [11], that the Poisson transform gives a $G$-isomorphism from the set of hyperfunctions on the maximal boundary $G/P_{\min}$ onto the joint eigenspace of invariant differential operators on $X$, thus proving the Helgason conjecture [6]. We shall be interested in the case of a Hermitian symmetric space $G/K$ and the Poisson transform corresponding a maximal (instead of minimal) parabolic subgroup $P_{\max} \subset G$ with $G/P$ being the Shilov boundary of $G/K$. For a certain special parameter of the induced representation the image of the transform is a subspace of harmonic functions on symmetric space $G/K$. The precise description of the image for tube domains is given in [9, 10, 15] in terms of Hua-harmonic functions introduced earlier by Hua [8]. Its generalization to non-tube cases is done by Berline and Vergne [1]. For tube domains with general parameters Shimeno [25] proved an

<footnote>Key words and phrases. Bounded symmetric domains, Shilov boundary, invariant differential operators, eigenfunctions, Poisson transform, Hua systems.</footnote>

<footnote>Research by G. Zhang supported by Swedish Science Council (VR). .</footnote>
analogue of Kashiwara *et al.* theorem for $P_{\max} \subset G$. More precisely, he proved that the Poisson transform is a $G$-isomorphism from the space of hyperfunctions on the Shilov boundary onto the space of eigenfunctions of the Hua operator of the second order.

The generalization to the non-tube bounded symmetric domains has been given in our earlier paper [14].

A more interesting problem is to consider homogeneous line bundles over $\Omega$ with the corresponding weighted action of $G$. In this setting Shimeno [23] generalized the Kashiwara *et al.* theorem to homogeneous line bundles over Hermitian symmetric spaces of tube type $G/K$ for minimal parabolic $P_{\min} \subset G$. That is for a given a line bundle $E_{\nu}$ (see below) and for a generic parameter depending on $\nu$ of the induced representation from a minimal parabolic $P_{\min} \subset G$. subgroup, the Poisson transform maps as isomorphisms from hyperfunction-valued sections of a line bundle over $G/P_{\min}$ onto a space of eigenfunctions. We shall find characterizations of the Poisson integrals of hyperfunction-valued sections of a line bundle over the Shilov boundary $G/P_{\min}$ of a bounded symmetric domain for generic parameters. The Poisson transform becomes more interesting for larger parameters of $\nu$ as there appear relative discrete series, in particular, the weighted Bergman spaces, in the Plancherel formula [24], the Poisson transform on Shilov boundary for the corresponding parameter is obviously not injective. We shall compute explicitly the image for some of the relative discrete series. We proceed with some more precise description of our result.

Let $\Omega = G/K$ be a bounded symmetric domain of tube type of rank $r$ and genus $p$. For $\nu \in p\mathbb{Z}$ we consider the (unique) character $\tau_\nu$ of $K$ and the corresponding homogeneous line bundle $E_\nu$ over $\Omega$. We identify $C^\infty$-sections of $E_\nu$ with the space $C^\infty(G/K, \tau_\nu)$ of $C^\infty$-functions on $G$ such that $f(gk) = \tau_\nu(k)^{-1}f(g)$. We consider the generalized Poisson transform $(P_{s,\nu}f)(z) = \int_S P_{s,\nu}(z, u)f(u)du$ where $P_{s,\nu}$ is the generalized Shilov kernel and $S = G/P_1$ the Shilov boundary of $\Omega$. The subgroup $P_1$ is a maximal parabolic subgroup of $G$. For $s \in \mathbb{C}$, let $\mathcal{B}(S, s, \nu)$ be the space of hyperfunction-valued sections on $S = G/P_1$ associated with the character of $P_1$ given by $\text{man} \mapsto e^{(s\rho_0-\rho_1)(\log a)}\tau_\nu(m)$. Then for $\lambda_s = \rho + 2n(s-1)\xi_e^* - \nu r \xi_e^*$, the space $\mathcal{B}(S, s, \nu)$ can be considered as a subspace of $\mathcal{B}(G/P; L_{\lambda_s, \nu})$ of hyperfunction-valued sections of a line bundle over $G/P$. We construct certain Hua operators on $G/K$ and we prove (Theorem 5.2) that for generic values of $s$ the Poisson transform $P_{s,\nu}$ is a $G$-isomorphism between $\mathcal{B}(S, s, \nu)$ and the space of eigenfunctions of the Hua operator of the second order.
If $\Omega$ is a non-tube type domain it is known that Hua-type operators of second order will not be sufficient to characterize the image of Poisson transform on the Shilov boundary. However for type one non-tube domains of $r \times (r+b)$-matrices the Lie algebra $\mathfrak{k}_c$ is a direct sum of $gl_r$ and $sl_{r+b}$ and one can construct [1] a second order Hua operator by taking certain projection on the summand $gl_r$. We prove a corresponding result for line bundles in this case; see 8.1.

For singular value of $s$ we prove (Theorem 9.3) that the Poisson transform is a Szegö type map of principal series representation onto the relative discrete series representation. We compute explicitly the Poisson transform on certain spherical polynomials on the Shilov boundary.

The paper is organized as follows. In §2 we recall very briefly the Jordan algebraic characterization of bounded symmetric domains. In §3 we introduce the line bundle over the bounded symmetric domain $\Omega$. The generalized Poisson transform of hyperfunction-valued sections on the maximal and the Shilov boundaries are studied in §4. The characterization of Poisson integrals of hyperfunction-valued sections on the Shilov boundary is given in §5. In this section we also recall our geometric construction of the Hua operator. The necessary condition is proved in §6. In §7 we compute the radial part of the Hua system and prove sufficiency condition. Finally in §8 we show a relationship between the Poisson transform, Hua operator and the relative discrete series representation.

After a preliminary version of this paper was finished we were informed by Professor T. Oshima that he and N. Shimeno have obtained in [20] some similar results about Poisson transforms and Hua operators. Professor A. Koranyi communicated also his recent preprint [13] to us where he proved the necessity of Theorem 5.2 using different methods. In particular some of the questions posed in that paper are answered here.

## 2. BOUNDED SYMMETRIC DOMAINS AND JORDAN TRIPLES

We begin with a brief review of necessary facts on bounded symmetric domains and Jordan triple systems. Let $V$, $\dim V = n$ be a complex vector space, $\Omega \subset V$ an irreducible bounded symmetric domain. Let $Aut(\Omega)$ be the group of all biholomorphic automorphisms of $\Omega$, let $G$ be the connected component of the identity of $Aut(\Omega)$, and let $K$ be the isotropy subgroup of $G$ at the point $0 \in \Omega$. As a symmetric
space, $\Omega = G/K$. The group $K$ acts as linear transformations on $V$
and we can thus identify $K$ also as a subgroup of $GL(V)$.

Let $\mathfrak{g}$ be the Lie algebra of $G$ and $\mathfrak{g}_\mathbb{C}$ its complexification. The algebra
$\mathfrak{g}$ has the Cartan decomposition $\mathfrak{g} = \mathfrak{k} \oplus \mathfrak{p}$ and $\mathfrak{g}_\mathbb{C} = \mathfrak{p}^+ + \mathfrak{k}_\mathbb{C} + \mathfrak{p}^-$ the
Harish-Chandra decomposition. Denote $Z_0$ the element in the center
of $\mathfrak{k}$ which defines the complex structure on $\mathfrak{p}^+$, i.e., $\mathrm{ad}(Z_0)v = iv$ for
$v \in \mathfrak{p}^+$. We can thus identify $\mathfrak{p}^+$ with $V$, $\mathfrak{p}^+ = V$.

There exists a quadratic map $Q : V \to \mathrm{End}(\bar{V}, V)$ (where $\bar{V}$ is the complex conjugate of $V$), such that $\mathfrak{p} = \{\xi_v ; v \in V\}$ as holomorphic vector fields, where $\xi_v(z) := v - Q(z)\bar{v}$. Define $D(z, \bar{v})w := (Q_{z+w} - Q_z - Q_w)\bar{v}$. It satisfies

$$
D(z, \bar{v})w = D(w, \bar{v})z, \quad [D(u, \bar{v}), D(z, \bar{w})] = D(\{u \bar{v} z\}, \bar{w}) - D(z, \overline{\{w \bar{u} v\}})
$$

so $V$ is a Jordan triple system. Furthermore we have $[X, \xi_z] = \xi_{Xz}$ for $X \in \mathfrak{k}, z \in V$ and $[\xi_z, \xi_v] = D(z, \bar{v}) - D(v, \bar{z})$ for all $z, v \in V$. In this realization elements in $\mathfrak{p}^-$ are of the form $\{-Q(z)\bar{v}\}$ which we write as $\bar{v}$. Thus

$$
(1) \qquad [v, \bar{w}] = D(z, \bar{w}).
$$

We define

$$
(2) \qquad \langle z, w \rangle = \frac{1}{p} \mathbf{tr} D(z, \bar{w}),
$$

where **tr** is the trace functional on End($V$) and $p$ is the genus defined
below. It is a $K$-invariant Hermitian product on $V$.

The group $K$ acts on $V$ by unitary transformations. The domain $\Omega$
is realized as the open unit ball of $V$ with respect to the spectral norm,

$$
(3) \qquad \Omega = \{z \in V : \|D(z, \bar{z})\|^2 < 2\},
$$

where $\|D(z, \bar{z})\|$ is the operator norm of $D(z, \bar{z})$ on the Hilbert space
$(V, \langle \cdot, \cdot \rangle)$.

An element $e \in V$ is a tripotent if $\{e \bar{e} e\} = e$. The subspaces
$V_\lambda(e) = \ker(D(e, \bar{e}) - \lambda \mathrm{id})$ are called Pierce $\lambda$-spaces. Then we have
$V = V_0(e) \oplus V_1(e) \oplus V_2(e)$. Two tripotents $e$ and $c$ are orthogonal if
$D(e, \bar{c}) = 0$. A tripotent $e$ is minimal if it cannot be written as the sum
of two non-zero orthogonal tripotents. With the above normalization
of inner product we have $\langle e, e \rangle = 1$ for minimal tripotents $e$. The
tripotent $e$ is maximal if $V_0(e) = 0$.

A frame is a maximal family of pairwise orthogonal, minimal tripo-
tents. It is known that the group K acts transitively on frames. In
particular, the cardinality of all frames is the same, and it is equal to
the rank r of Ω.
Let us choose and fix a frame {$e_j$}$_{j=1}^r$ in $V$. Then, by transitivity of $K$ on the frames, each element $z \in V$ admits a polar decomposition $z = k \sum_{j=1}^r s_j e_j$, where $k \in K$ and $s_j = s_j(z)$ are the singular numbers of $z$. Denote $e$ the maximal tripotent $e = e_1 + \dots + e_r$. The Shilov boundary of $\Omega$ is $S = K/K_1$ where $K_1 = \{k \in K : k e = e\}$. It is known that $S$ coincides with the set of maximal tripotents of $V$.

The joint Peirce spaces are

$$
(4) \quad V_{ij} = \{z \in Z : D(e_k, \bar{e}_k)z = (\delta_{ik} + \delta_{jk})z, \forall k\}, \quad 0 \le i \le j \le r \\
\text{then } V = \bigoplus_{0 \le i \le j \le r} V_{ij}, V_{00} = 0, \text{ and } V_{ii} = \mathbb{C}e_i \quad (i > 0).
$$

The triple of integers $(r, a, b)$ with

$$
(5) \quad a := \dim V_{jk} (1 \le j < k \le r); \quad b := \dim V_{0j} (1 \le j \le r)
$$

is independent of the choice of the frame and uniquely determines the
Jordan triple. Notice that $b = 0$ exactly if $V$ is a Jordan algebra which
is equivalent to say that $\Omega$ is of tube type.

The Peirce decomposition associated with $e$ is then $V = V_2 \oplus V_1$
where

$$
(6) \qquad V_2 = \sum_{1 \le j \le k \le r} V_{jk} \qquad V_1 = \sum_{j=1}^{r} V_{0j}.
$$

Let $n_1 = \dim V_1$ and $n_2 = \dim V_2$, then

$$
n_1 = rn, \quad n_2 = r + \frac{r(r-1)}{2}a \quad \text{and} \quad n = n_1 + n_2.
$$

The genus of $\Omega$ is

$$
p = p(\Omega) = \frac{1}{r} \mathrm{tr} D(e, \bar{e}) = (r-1)a + b + 2.
$$

Let $\mathfrak{a} = \mathbb{R}\xi_{e_1} + \dots + \mathbb{R}\xi_{e_r}$. Then $\mathfrak{a}$ is a maximal abelian subspace of
$\mathfrak{p}$ with basis vectors $\{\xi_{e_1}, \dots, \xi_{e_r}\}$. Its dual basis in $\mathfrak{a}^*$ will be denoted
by $\{\frac{\beta_j}{2}\}_{j=1}^r \subset \mathfrak{a}^*$, i.e.,

$$
(7) \qquad \beta_j(\xi_{e_j}) = 2\delta_{jk}, \quad 1 \le j, k \le r.
$$

We define an ordering on $\mathfrak{a}^*$ via

$$
(8) \qquad \beta_r > \beta_{r-1} > \dots > \beta_1 > 0.
$$

It is known that the restricted roots system Σ(g, a) of g relative to a
is of type C_r or BC_r, it consists of the roots ±β_j (1 ≤ j ≤ r) with
multiplicity 1, the roots ±1/2β_j ± 1/2β_k (1 ≤ j ≠ k ≤ r) with multiplicity
a, and possibly the roots ±1/2β_j (1 ≤ j ≤ k) with multiplicity 2b. The
set of positive roots Σ⁺(g, a) consists of 1/2(β_j ± β_k) (1 ≤ j < k ≤ r), β_j
and 1/2β_j (1 ≤ j ≤ r).
The half sum of positive roots is given by

$$ (9) \qquad \underline{\rho} = \sum_{j=1}^{r} \rho_j \beta_j = \sum_{j=1}^{r} \frac{b+1+a(j-1)}{2} \beta_j. $$

Let $A$ be the analytic subgroup of $G$ corresponding to $\mathfrak{a}$. Let $\mathfrak{n}^+ = \sum_{\alpha \in \Sigma^+} \mathfrak{g}^\alpha$ and $\mathfrak{n}^- = \theta(\mathfrak{n}^+)$. Let $N^+$ and $N^-$ be the corresponding analytic subgroups of $G$. Let $M$ be the centralizer of $\mathfrak{a}$ in $K$. The subgroup $P = MAN^+$ is a minimal parabolic subgroup of $G$.

The set $\Lambda = \{\alpha_1, \dots, \alpha_{r-1}, \alpha_r\}$ of simple roots in $\Sigma^+$ is such that

$$ \alpha_j = \frac{1}{2}(\beta_{r-j+1} - \beta_{r-j}), \quad 1 \le j \le r-1 $$

and

$$ \alpha_r = \begin{cases} \beta_1 & \text{for the tube case} \\ \frac{1}{2}\beta_1 & \text{for the non-tube case.} \end{cases} $$

Let $\Lambda_1 = \{\alpha_1, \dots, \alpha_{r-1}\}$ and $P_1$ the corresponding standard parabolic subgroup of $G$ with the Langlands decomposition $P_1 = M_1 A_1 N_1^+$ such that $A_1 \subset A$. Then the Lie algebra of $A_1$ is $\mathfrak{a}_1 = \mathbb{R}\xi_e$ where $\xi_e = \xi_{e_1} + \dots + \xi_{e_r}$.

The spaces $G/P = K/M$ is the Furstenberg or maximal boundary, and $G/P_1 = K/K_1$, with $K_1 = M_1 \cap K$, is the Shilov boundary of $G/K$.

### 3. LINE BUNDLE OVER $\Omega$

Denote $Z = \mathbb{P}_n Z_0$ where $Z_0$ is the center element defined in §2. The group $K$ is factorized as $K = \exp(\mathbb{R}Z)K_s$, where $K_s$ is the analytic subgroup of $K$ with Lie algebra $\mathfrak{k}_s = [\mathfrak{k}, \mathfrak{k}]$.

For a fixed $\nu \in p\mathbb{Z}$ consider the character $\tau_\nu$ of $K$ defined by

$$ \tau_\nu(k) = \begin{cases} e^{it\nu} & \text{if } k = \exp(tZ) \in \exp(\mathbb{R}Z) \\ 1 & \text{if } k \in K_s. \end{cases} $$

In particular we have $J_k(z)^{\frac{\nu}{p}} = e^{it\nu}$ for $k = \exp(tZ)$, $z \in \Omega$. Thus $J_k(z)^{\frac{\nu}{p}} = \tau_\nu(k)$, $k \in K$. Here we denote $J_g, g \in G$, the Jacobian of the holomorphic mappings $g$ on $\Omega$. See [22] and [2].

Let $E_\nu$ be the homogeneous line bundle $G \times_K \mathbb{C}$ over $G/K = \Omega$, where $K$ acts on $\mathbb{C}$ via the one dimensional representation $\tau_\nu$. The space $C^\infty(\Omega, E_\nu)$ of smooth sections of $E_\nu$ is by definition the space $C^\infty(\Omega, E_\nu) = C^\infty(G/K; \tau_\nu)$ of $C^\infty$-functions $F$ on $G$ such that

$$ F(gk) = \tau_\nu(k)^{-1}F(g). $$
We will trivialize the bundle via the map $[(g, c)] \in E_\nu \mapsto (g \cdot 0, J_g(0)^{\frac{\nu}{p}} c) \in \Omega \times \mathbb{C}$ and thus identifies $C^\infty(G/K; \tau_\nu)$ also as the space of $C^\infty$-functions on $\Omega$ with $G$ acting as

$$ (10) \qquad g \in G : \quad f(z) \mapsto J_{g^{-1}}(z)^{\frac{\nu}{p}} f(g^{-1}z). $$

Let $D_\nu(G/K)$ denote the space of $G$-invariant differential operators on $G/K$ acting on $C^\infty(G/K; \tau_\nu)$. We have the Harish-Chandra isomorphism [23]

$$ \gamma_\nu : D_\nu(G/K) \simeq U(\mathfrak{a})^W $$

where $U(\mathfrak{a})^W$ denote the set of $W$-invariant elements in the enveloping algebra $U(\mathfrak{a})$.

The characters of $D_\nu(G/K)$ are given by

$$ \chi_{\lambda, \nu}(D) = \gamma_{\nu}(D)(\lambda), \quad D \in D_{\nu}(G/K), \quad \lambda \in \mathfrak{a}_{\mathbb{C}}^{*}. $$

For $\lambda \in \mathfrak{a}_{\mathbb{C}}^*$ we define $\mathcal{A}(G/K, \mathcal{M}_{\lambda, \nu})$ to be the space of functions $\varphi \in C^\infty(G/K; \tau_\nu)$ satisfying the system of differential equations

$$ (11) \qquad \mathcal{M}_{\lambda, \nu} : D\varphi = \chi_{\lambda, \nu}(D)\varphi \quad D \in D_{\nu}(G/K). $$

Finally, let $\mathcal{B}(G/P; L_{\lambda, \nu})$ be the space of hyperfunctions $f$ on $G$ satisfying

$$ f(gman) = e^{(\lambda-\rho)(\log a)} \tau_{\nu}(m)^{-1} f(g) $$

for all $g \in G$, $m \in M$, $a \in A$, $n \in N^+$. The space $\mathcal{B}(G/P; L_{\lambda, \nu})$ is a $G$-submodule of $\mathcal{B}(G)$ and can be identified with the space of hyperfunction valued sections of the line bundle $L_{\lambda, \nu}$ on $G/P$ associated with the character of $P$ given by $man \mapsto e^{(\rho-\lambda)(\log a)} \tau_\nu(m)$, $m \in M$, $a \in A$, $n \in N^+$.

## 4. POISSON TRANSFORM

### 4.1. Poisson integrals on $G/P$ for a minimal $P$. 
Let $\lambda \in \mathfrak{a}_{\mathbb{C}}^*$ and $\nu \in \mathbb{C}$. We define the Poisson transform $\mathcal{P}_{\lambda, \nu}$ by

$$ (12) \qquad (\mathcal{P}_{\lambda, \nu} f)(g) = \int_K f(gk) \tau_\nu(k) dk \quad g \in G $$

for any $f \in \mathcal{B}(G/P, L_{\lambda, \nu})$.

As elements $f \in \mathcal{B}(G/P, L_{\lambda, \nu})$ are uniquely determined by its restriction to $K$ it is natural to express the integral above as on $K$. Indeed, for $g \in G$ denote $\kappa(g) \in K$ and $H(g) \in \mathfrak{a}$ to be elements uniquely determined by

$$ g \in \kappa(g) \exp(H(g)) N^+ \subset KAN^+ = G. $$
Then

$$ (13) \qquad (\mathcal{P}_{\lambda,\nu} f)(g) = \int_K f(k) \tau_\nu(\kappa(g^{-1}k)) e^{-(\lambda+\rho)(H(g^{-1}k))} dk $$

and maps $\mathcal{B}(G/P, L_{\lambda,\nu})$ into $\mathcal{A}(G/K, \mathcal{M}_{\lambda,\nu})$, see [23, Theorem 5.2]

The Harish-Chandra c function

$$ c_{\nu}(\lambda) = \int_{N^{-}} e^{(\lambda+\rho)(H(\bar{n}))} \tau_{\nu}(\kappa(\bar{n})) d\bar{n} $$

can be written as

$$ c_{\nu}(\lambda) = \frac{f_{\nu}(\lambda)}{e_{\nu}(\lambda)}, $$

where the denominator $e_\nu(\lambda)$ is given, in our normalization, by

$$ e_{\nu}(\lambda) = \prod_{j>k} \Gamma(\frac{1}{2}a + \lambda_j \pm \lambda_k) \prod_{j} \Gamma(\frac{1}{2}(b+1+2\lambda_j+\nu)) \Gamma(\frac{1}{2}(b+1+2\lambda_j-\nu)); $$

see [22] and [23, page 227].

The following theorem characterizes the range of the Poisson transform.

**Theorem 4.1** ([23], Theorem 8.1). If $\lambda \in \mathfrak{a}_c^*$ and $\nu \in \mathbb{C}$ satisfy the conditions

$$ (14) \qquad -2 \frac{\langle \lambda, \alpha \rangle}{\langle \alpha, \alpha \rangle} \notin \{1, 2, 3, \dots\} \quad \text{for any } \alpha \in \Sigma^{+} $$

$$ (15) \qquad e_{\nu}(\lambda) \neq 0, $$

then the Poisson transform $\mathcal{P}_{\lambda,\nu}$ is a $G$-isomorphism of $\mathcal{B}(G/P, L_{\lambda,\nu})$ onto $\mathcal{A}(G/K, \mathcal{M}_{\lambda,\nu})$.

4.2. **Poisson integrals on the Shilov boundary.** Let $h$ be the unique $K$-invariant polynomial on $\mathfrak{p}^+ = V$ whose restriction on $\mathbb{R}e_1 \oplus \dots \oplus \mathbb{R}e_r$ is given by

$$ h(\sum_{j=1}^{r} t_j e_j) = \prod_{j=1}^{r} (1 - t_j^2). $$

Let $h(z, w)$ be its polarization, i.e. $h(z, w)$ is holomorphic on $z$ and antiholomorphic in $w$ such that $h(z, z) = h(z)$. For any complex number $s$ and for any $\nu$ we define the generalized Poisson kernel

$$ (16) \qquad P_{s,\nu}(z, u) = \left( \frac{h(z, z)}{|h(z, u)|^2} \right)^{s \frac{n}{r}} h(z, u)^{-\nu}, \quad z \in \Omega, u \in S. $$
We also define the generalized Poisson transform

$$ (17) \qquad (\mathcal{P}_{s,\nu} f)(z) = \int_S P_{s,\nu}(z, u)f(u)du, \quad \text{for } f \in \mathcal{B}(S) $$

Let

$$ \mathbf{a}_1^\top = \sum_{j=1}^r \mathbb{R}(\xi_j - \xi_{j+1}) $$

be the orthogonal complement of $\mathbf{a}_1 = \mathbb{R}\xi_e$ in $\mathbf{a}$ with respect to the Killing form. We denote $\xi_e^*$ the dual vector such that $\xi_e^*(\xi_e) = 1$ and we extend $\xi_e^*$ to $\mathbf{a}$ by the orthogonal projection defined above. Then $\rho_1 = \rho|_{\mathbf{a}_1}$ the restriction of $\rho$ to $\mathbf{a}_1$ is given by $\rho_1 = n\xi_e^*$. Define $\rho_0$ to be the linear form on $\mathbf{a}_1$ such that $\rho_0 = r\xi_e^*$.

Consider the following representation of $P_1 = M_1A_1N_1$ given by $\sigma_{s,\nu} = \tau_{\nu|_{M_1}} \otimes e^{s\rho_0-\rho} \otimes 1$ and let $\mathcal{B}(G/P_1, s, \nu)$ be the space of hyperfunction valued sections of the line bundle on $S = G/P_1$ corresponding to $\sigma_{s,\nu}$, i.e., the space of hyperfunctions $f$ on $G$ satisfying

$$ f(gman) = e^{(s\rho_0-\rho_1)(\log a)}\tau_\nu(m)^{-1}f(g) $$

for all $g \in G$, $m \in M_1$, $a \in A_1$, $n \in N_1^+$. We may fix $e \in S$ as a base point and identify also $\mathcal{B}(G/P_1, s, \nu)$ with $\mathcal{B}(S)$.

For $s \in \mathbb{C}$ we define $\lambda_s \in \mathbf{a}^*$ by

$$ (18) \qquad \lambda_s = \rho + 2n(s-1)\xi_e^* - \nu r \xi_e^* $$

Under the identification of $C^\infty(G/K; \tau_\nu)$ (and thus its subspace $\mathcal{A}(G/K, \mathcal{M}_{\lambda,\nu})$) as smooth functions on $\Omega$ we have $\mathcal{P}_{\lambda_s,\nu}$ coincides with $\mathcal{P}_{s,\nu}$. We omit the routine computations.

Let $\nu \in p\mathbb{Z}$, then we have

$$ (19) \qquad \mathcal{B}(G/P_1; s, \nu) \subset \mathcal{B}(G/P; L_{\lambda_s, \nu}) $$

therefore

$$ (20) \qquad \mathcal{P}_{s,\nu}(\mathcal{B}(G/P_1; s, \nu)) \subset \mathcal{A}(G/K; \mathcal{M}_{\lambda_s,\nu}). $$

## 5. HUA OPERATORS

The Bergman reproducing kernel of $\Omega$ is $h(z, \bar{z})^{-p}$ up to a constant. It is also

$$ (21) \qquad h(z, \bar{z})^{-p} = \det B(z, \bar{z})^{-1} $$

where

$$ B(z, \bar{w}) = I - D(z, \bar{w}) + Q(z)Q(\bar{w}) $$
is the Bergman operator. Thus $\Omega = G/K$ is a Kähler manifold with the (normalized) Bergman metric

$$ (22) \qquad (u, v)_z = \frac{1}{p} \partial_u \bar{\partial}_v \log h(z, \bar{z})^{-p} = \langle B(z, \bar{z})^{-1} u, v \rangle. $$

Let $\tau$ be a finite-dimensional holomorphic representation of $K_{\mathbb{C}}$. Let $E$ be the Hermitian vector bundle over $\Omega$ associated with $\tau$. Then there exists a unique connection operator $\nabla : C^{\infty}(\Omega, E) \to C^{\infty}(\Omega, E \otimes T')$ compatible with the Hermitian structure and the anti-holomorphic differentiation, where $T' = T'\Omega$ is the cotangent bundle over $\Omega$. That is, under the splitting in holomorphic and antiholomorphic parts, $T' = T'^{(1,0)} \oplus T'^{(0,1)}$, we have $\nabla = \mathcal{D} + \bar{\partial}$ where $\mathcal{D}$ is the differentiation operator on $E$,

$$ (23) \qquad \mathcal{D} : C^{\infty}(\Omega, E) \to C^{\infty}(\Omega, E \otimes T'^{(1,0)}). $$

Let $\bar{D}$ be the invariant Cauchy-Riemann operator on $E$ defined in [3, 30], by

$$ (24) \qquad \bar{D}f = B(z, \bar{z})\bar{\partial}f. $$

Then

$$ (25) \qquad \bar{D} : C^{\infty}(\Omega, E) \to C^{\infty}(\Omega, E \otimes T'^{(1,0)}). $$

We will use the following identifications, $T_z'^{(1,0)} = \mathfrak{p}^- = V' = \bar{V}$ and $T_z'^{(0,1)} = \mathfrak{p}^+ = V$.

We now specialize the above to the line bundle $E_\nu$ associated with the one-dimensional representation $\tau_\nu$. The Hua operator $\mathcal{H}_\nu$ is then defined as the resulting operator of the following diagram

<figure><img src="image_3.png" /></figure>

We may call $\mathcal{H} = \mathcal{H}_\nu$ the twisted Hua operator to differ it from the trivial case $\nu = 0$.

**Remarks 5.1.** (1) We can change the order of $\bar{D}$ and $\mathcal{D}$ and define another Hua operator,

$$ \mathcal{H}'f = Ad_{\mathfrak{p}+\otimes\mathfrak{p}-} \bar{D}\mathcal{D}f, $$
and one can prove, by direct computations, the following relation

$$ (26) \qquad (\mathcal{H} - \mathcal{H}')F = -\frac{2n}{r}\nu FId. $$

(2) On other hand, following Johnson-Korànyi [10], the Hua operator can also be described using the enveloping algebra: Let {$E_\alpha$} be a basis of $\mathfrak{p}^+$ and {$E_\alpha^*$} be the dual basis of $\mathfrak{p}^+$ with respect to the Killing form. Then the Hua operator is defined as element of $U(\mathfrak{g})_\mathbb{C} \otimes \mathfrak{k}_\mathbb{C}$ by

$$ (27) \qquad \mathcal{H} = \sum_{\alpha, \beta} E_{\alpha} E_{\beta}^{*} \otimes [E_{\beta}, E_{\alpha}^{*}], $$

as operator acting from $C^\infty(E_\nu)$ to $C^\infty(E_\nu \otimes \mathfrak{k}_\mathbb{C})$. However the previous definition using $\bar{D}$ and $D$ allows direct computation using coordinates on $\Omega$ and hence has some computational (and also conceptual) advantage.

Note that $\mathcal{H}$ is by definition $G$-invariant with respect to the actions of $G$ on two holomorphic bundles, i.e.,

$$ (28) \qquad \mathcal{H}(J_g(z)^{\frac{\nu}{p}} f(gz)) = J_g(z)^{\frac{\nu}{p}} \mathrm{Ad}(dg(z)^{-1})(\mathcal{H}f)(gz), $$

where $\mathrm{Ad}(dg(z)^{-1})$ stands for the adjoint action on $\mathfrak{k}_\mathbb{C}$ of $dg(z)^{-1} : \mathfrak{p}^+ = T_{gz}^{(1,0)} \mapsto \mathfrak{p}^+ = T_z^{(1,0)}$ on $\mathfrak{p}^+$, via the defining action of $\mathfrak{k}_\mathbb{C}$ on $\mathfrak{p}^+$. (Indeed the element $dg(z)^{-1}$ is in the group $K_\mathbb{C} \subset GL(V) = GL(\mathfrak{p}^+)$ via the defining realization $K \subset GL(V)$, thus $\mathrm{Ad}(dg(z)^{-1})$ acts on $\mathfrak{k}_\mathbb{C}$; see [21, Chapt II, Lemma 5.3].)

We state now our main result for case of tube domains. The proof is given in the next two sections.

**Theorem 5.2.** Let $\Omega$ be a bounded symmetric domain of tube type. Suppose $s \in \mathbb{C}$ satisfies the following condition

$$ (29) \qquad \frac{4n(1-s)}{r} \notin \Lambda_1 \cup \Lambda_2 $$

where $\Lambda_1 = \mathbb{Z}_+ - 2\nu + 2$, $\Lambda_2 = 2\mathbb{Z}_+ - 4\nu + 4$. Then the Poisson transform $\mathcal{P}_{\lambda_s, \nu}$ is a $G$-isomorphism of $\mathcal{B}(S; s, \nu)$ onto the space of analytic functions $F$ on $\Omega$ such that

$$ (30) \qquad \mathcal{H}F = 2\frac{n}{r}s(\frac{n}{r}(s-1)+\nu)FId. $$

Here $Id$ stands for the element $-iZ_0$, which acts on $\mathfrak{p}^+$ as identity.
## 6. THE NECESSARY CONDITION OF THE HUA EQUATIONS

The necessity in the above theorem is a consequence of the following

**Theorem 6.1.** For any $u \in S$ the function $z \mapsto P_{s,\nu}(z, \bar{u})$ satisfies the following system of equations

$$ (31) \qquad \mathcal{H}P_{s,\nu} = 2\frac{n}{r}s\left[\frac{n}{r}(s-1) + \nu\right]P_{s,\nu}Id $$

*Proof.* We prove first the claim for $z = 0$, in which case we have

$$ \mathcal{H}f(0) = \sum_{\alpha, \beta} \partial_{\alpha} \bar{\partial}_{\beta} f(0) [e_{\alpha}, \bar{e}_{\beta}]. $$

Recall the following formulas in [14, Lemma 5.2]: for any fixed $\bar{w} \in \bar{V}$ and any complex number $s$,

$$ (32) \qquad \bar{\partial}h(w, \bar{z})^s = -sh(w, \bar{z})^s w^{\bar{z}}, \quad \partial h(z, \bar{w})^s = -sh(z, \bar{w})^s \bar{w}^z $$

where

$$ w^{\bar{z}} = B(w, \bar{z})^{-1}(w - Q(w)\bar{z}), \quad \bar{w}^z = \overline{w^{\bar{z}}} $$

are called quasi-inverses of $w$ with respect to $\bar{z}$ and $\bar{w}$ with respect to $z$ respectively, viewed as (1,0)-form and (0,1)-form in terms of the Hermitian inner product (2).

Then we have

$$ \bar{\partial}P_{s,\nu}(z, \bar{u}) = \bar{\partial} \left( h(z, u)^{-\nu} \left( \frac{h(z, z)}{|h(z, u)|^2} \right)^{\frac{n}{r}s} \right) = -(\frac{n}{r}s)P_{s,\nu}(z, \bar{u})[z^{\bar{z}} - u^{\bar{z}}], $$

and

$$ -(\frac{n}{r}s)\partial(P_{s,\nu}(z, \bar{u})[z^{\bar{z}} - u^{\bar{z}}]) = -(\frac{n}{r}s)(\nu + \frac{n}{r}s)P_{s,\nu}(z, \bar{u})[z^{\bar{z}} - u^{\bar{z}}] \otimes \bar{u}^z \\ + (\frac{n}{r}s)^2 P_{s,\nu}(z, \bar{u})[z^{\bar{z}} - u^{\bar{z}}] \otimes \bar{z}^z \\ -(\frac{n}{r}s)P_{s,\nu}(z, \bar{u})\partial[z^{\bar{z}} - u^{\bar{z}}]. $$

Using (22) the last derivative can be written as

$$ \partial[z^{\bar{z}} - u^{\bar{z}}] = \partial(z^{\bar{z}}) = \partial\bar{\partial}\log h(z, \bar{z})^{-1} = B(z, \bar{z})^{-1}Id $$

where $Id$ is the identity form in $\mathfrak{p}^+ \otimes \mathfrak{p}^- = \mathfrak{p}^+ \otimes (\mathfrak{p}^+)'$. Evaluating at $z = 0$ we get then, by the commutation relation (1)

$$ \mathcal{H}P_{s,\nu}(0, \bar{u}) = \left(\frac{n}{r}s\right)(\nu + \frac{n}{r}s)D(u, \bar{u}) \\ +0 \\ -(\frac{n}{r}s)\sum_{\alpha=1}^{n} D(e_{\alpha}, \bar{e}_{\alpha}). $$

Since $D(u, \bar{u}) = 2I$ and $\sum_{\alpha=1}^n D(e_\alpha, \bar{e}_\alpha) = 2\frac{n}{r}I$ by [14, Lemma 5.1], the claim is proved for $z = 0$. Note furthermore that the Poisson kernel satisfies

$$ P_{s,\nu}(gz, gu) = J_g(z)^{\frac{\nu}{p}} P_{s,\nu}(gz, gu) \overline{J_g(u)^{\frac{\nu}{p}}}. $$
Thus the claim is true for general $z$ by the invariant property (28) of $\mathcal{H}$.

## 7. THE SUFFICIENCY CONDITION OF THE HUA EQUATIONS

The aim of this section is to prove that, using the radial part of the Hua operator, each solution $F$ of the system (30) satisfies the system of equations (11). Then under the condition (29) it can be proved that the boundary value of $F$ is contained in $\mathcal{B}(S; s, \nu)$.

We start to show that eigenfunctions of the Hua operator (30) are all $\tau_\nu$-spherical functions. We need only to prove the claim for $K$-invariant functions $F$ on $\Omega$, i.e., $F(kz) = F(z)$, $k \in K$.

For that purpose we compute the radial part of the Hua operator. The functions $F(z)$ will be identified as permutation invariant and even function $F(t)$ on the diagonal $z = \sum_{j=1}^r t_j e_j$. The operator $\mathcal{H}$ has the form

$$ \mathcal{H}F(z) = \sum_{j=1}^{r} \mathcal{H}_j F(z) D(e_j, e_j), $$

for some operators $\mathcal{H}_j$ in $t = (t_1, \dots, t_r)$. It's convenient to find the radial part of $4\mathcal{H}$ (due to the usual convention that $\bar{\partial}\partial = \frac{1}{4}(\partial_x^2 + \partial_y^2) = \frac{1}{4}(\partial_r^2 + \frac{1}{r}\partial_\nu)$ for radial functions in $z \in \mathbb{C}$).

**Theorem 7.1.** Let $\Omega$ be the tube type domain. Let $F$ be $C^2$ and $K$-invariant function, then for $a = \sum_{j=1}^r t_j e_j$,

$$ (33) \qquad 4\mathcal{H}F(a) = \sum_{j=1}^{r} \mathcal{H}_j F(t_1, \dots, t_r) D(e_j, \bar{e}_j), $$

where the scalar-valued operators $\mathcal{H}_j$ are given by

$$ \begin{aligned} \mathcal{H}_j = & (1 - t_j^2)^2 \left( \frac{\partial^2}{\partial t_j^2} + \frac{1}{t_j} (1 + 2\nu) \frac{\partial}{\partial t_j} \right) + \\ & + \frac{a}{2} \sum_{k \neq j} (1 - t_j^2)(1 - t_k^2) \left[ \frac{1}{t_j - t_k} \left( \frac{\partial}{\partial t_j} - \frac{\partial}{\partial t_k} \right) + \frac{1}{t_j + t_k} \left( \frac{\partial}{\partial t_j} + \frac{\partial}{\partial t_k} \right) \right] + \\ & + (-2\nu)(1 - t_j^2) \frac{1}{t_j} \frac{\partial}{\partial t_j}. \end{aligned} $$

*Proof.* Recall [30] that the operator $\mathcal{D}$ acting on $T^{(1,0)}$-valued function takes the form

$$ \mathcal{D}F(z) = h^{-\nu}(z, \bar{z}) B(z, \bar{z}) \sum_k \partial_k (h^{\nu}(z, \bar{z}) B(z, \bar{z})^{-1} F(z)) \otimes \bar{v}_k $$
which is a $\mathfrak{p}^+ \otimes \mathfrak{p}^-$-valued function. Thus

$$ \mathcal{H}F(z) = \mathrm{Ad}_{\mathfrak{p}^+ \otimes \mathfrak{p}^-} \sum_{k,l} h^{-\nu}(z, \bar{z}) B(z, \bar{z}) \sum_k \partial_k (h^{\nu}(z, \bar{z}) (\bar{\partial}_l F(z)) v_l) \otimes \bar{v}_k, $$

where we have performed the cancellation of the $B^{-1}$-term in $\mathcal{D}$ with $B$ in $\bar{D}$. Performing the Leibniz rule for $\partial_k$ we get the above as sum of two terms, say I and II where

$$ \mathrm{II} = \mathrm{Ad}_{\mathfrak{p}^+ \otimes \mathfrak{p}^-} \sum_{k,l} B(z, \bar{z}) \sum_k \partial_k ((\bar{\partial}_l F(z)) v_l) \otimes \bar{v}_k $$

and

$$ I = \mathrm{Ad}_{\mathfrak{p} \otimes \bar{\mathfrak{p}}} \sum_{k,l} h^{-\nu}(z, z) B(z, \bar{z}) \sum_k (\partial_k h^{\nu}(z, \bar{z})) ((\bar{\partial}_l F(z)) v_l) \otimes \bar{v}_k $$

The second order term II is computed as in [14] or [5] and we need only to treat I. Recall that

$$ \partial_v h^{\nu}(z, \bar{z}) = \partial_v e^{\nu \log h(z, \bar{z})} = -\nu e^{\nu \log h(z, z)} \partial_v (-\log h(z, \bar{z})) = -\nu h^{\nu}(z, \bar{z}) (v, z^{\bar{z}}) $$

where $z^{\bar{z}} = \bar{\partial}(-\log h(z, \bar{z}))$ is quasi-inverse of $z$ with respect to $\bar{z}$; see [16], [32]. We have then

$$ I = -\nu \sum_{k,l} (v_k, z^{\bar{z}}) (\bar{\partial}_l F(z)) [B(z, \bar{z}) v_l, \bar{v}_k]. $$

For $z = \sum_{j=1}^r t_j e_j$ we have $z^{\bar{z}} = \sum_{j=1}^r \frac{t_j}{1-t_j^2} e_j$, and

$$ B(z, \bar{z}) = \sum_{j=1}^{r} (1 - t_j^2)^2 D(e_j, \bar{e}_j) $$

which is diagonalized under the Peirce decomposition of $V$. In particular the above sum reduces to

$$ I = -\nu \sum_{j=1}^{r} \frac{t_j}{1-t_j^2} (\bar{\partial}_{e_j} F(z)) (1-t_j^2)^2 D(e_j, \bar{e}_j). $$

Being a $K$-invariant function $F(z)$ is rotation invariant on the plan $\mathbb{C}e_j$ and we have then $\bar{\partial}_{e_j} F(z) = \frac{1}{2} \partial_{t_j} F(t)$. Finally we have

$$ I = -\frac{\nu}{2} \sum_{j=1}^{r} t_j (1 - t_j^2) \partial_{t_j} F(t) D(e_j, \bar{e}_j). $$

To put $I+II$ in a better form we write $t_j(1-t_j^2) = (1-t_j^2)(\frac{1-t_j^2}{t_j} - \frac{(1-t_j^2)^2}{t_j})$ and we get then the form $\mathcal{H}_j$ as claimed. $\square$
Note that this formula is consistent with the formula for Laplace-Beltrami operator on line bundle with parameter $\nu$ where the root multiplicity 1 for the root $\gamma_j$ is replaced by $1 + 2\nu$ and the multiplicity $2b$ of $\frac{\gamma_j}{2}$ by $2b - 2\nu$; see e.g. [24].

We prove now the sufficiency of the Hua equation (30) in Theorem 5.2. Let $\varphi_{\lambda_s, \nu}$ the unique elementary spherical function of type $\tau_\nu$ in $E_\nu$,

$$ \varphi_{\lambda_s, \nu}(g) = \int_K e^{(\lambda_s + \rho)H} \tau_\nu(k^{-1}\kappa(g^{-1}k))dk, \quad g \in G $$

With some slightly abuse of notation we denote $\varphi_{\lambda_s, \nu}(z)$ the corresponding $K$-invariant function on $\Omega$ via our trivialization.

Suppose $F$ be a $K$-invariant analytic eigenfunction of the Hua equation (30). Let $g \in G$. Then the function

$$ \Phi_g(z) = \int_K F(gkz)dk $$

is $K$-invariant solution of the differential system

$$ \mathcal{H}_j \Phi = 2 \frac{n}{r} s \left( \frac{n}{r} (s-1) + \nu \right) \Phi, \quad j = 1, \dots, r. $$

Therefore by a result of Yan [29], $\Phi_g$ is proportional to $\varphi_{\lambda_s, \nu}$ and thus,

$$ \int_K F(gkz)dk = \varphi_{\lambda_s, \nu}(z)F(g \cdot 0). $$

By Shimeno [24, Theorem 3.2] we see that this integral formula characterizes the joint eigenfunctions in $E_\lambda$, that is $F \in \mathcal{A}(G/K, \mathcal{M}_{\lambda_s, \nu})$. Now applying Theorem 4.1 we can find $f \in \mathcal{B}(G/P, L_{\lambda_s, \nu})$ such that $F = \mathcal{P}_{\lambda_s, \nu}(f)$. The rest of the proof is the same as in [1, 14], and we get, under the condition (29), that $f$ is actually a function on $S$, i.e., $f \in \mathcal{B}(S, s, \nu)$. We shall not repeat the computations.

## 8. TYPE ONE DOMAINS

Let $V = M_{r,r+b}(\mathbb{C})$ be the vector space of complex $r \times (r+b)$-matrices. $V$ is a Jordan triple system for the following triple product $\{x\bar{y}z\} = xy^*z + zy^*x$, and the endomorphisms $D(z, \bar{v})$ are given by

$$ D(z, \bar{v})w = \{z\bar{v}w\} = zv^*w + wv^*z. $$

Let

$$ \mathbf{I}_{r,r+b} = \{z \in M_{r,r+b}(\mathbb{C}) : I_r - z^*z \gg 0\} $$

where $I_r$ denote the unit matrix of rank $r$. Then $\mathbf{I}_{r,r+b}$ is a bounded symmetric domain of dimension $r(r+b)$, rank $r$ and genus $2r+b$. The
multiplicities are 2b and $a = 2$ if $2 \le r$, $a = 0$ if $r = 1$. The domain $\mathbf{I}_{r,r+b}$ is of tube type if and only if $b = 0$. Its Shilov boundary is

$$S_{r,r+b} = \{z \in M_{r,r+b}(\mathbb{C}) : z^*z = I_r\}.$$

As a homogeneous space, the bounded domain $\mathbf{I}_{r,r+b}$ can be identified with $SU(r, r+b)/S(U(r) \times U(r+b))$.

The complex Lie algebra $\mathfrak{k}_C$ is given by the set of all matrices

$$\begin{pmatrix} \mathbf{a} & 0 \\ 0 & \mathbf{d} \end{pmatrix}, \mathbf{a} \in M_{r,r}(\mathbb{C}), \mathbf{d} \in M_{r+b,r+b}(\mathbb{C}), \mathbf{tr}(\mathbf{a}) + \mathbf{tr}(\mathbf{d}) = 0.$$

Hence, $\mathfrak{k}_C$ can be written as the sum

$$\mathfrak{k}_C = \mathfrak{k}_C^{(1)} \oplus \mathfrak{k}_C^{(2)},$$

where $\mathfrak{k}_C^{(1)}$ and $\mathfrak{k}_C^{(2)}$ are the following ideals

$$\mathfrak{k}_C^{(1)} = \left\{ \begin{pmatrix} \mathbf{a} & 0 \\ 0 & -\frac{\mathbf{tr}(\mathbf{a})}{r+b} I_{r+b} \end{pmatrix}, \mathbf{a} \in M_{r,r}(\mathbb{C}) \right\},$$

$$\mathfrak{k}_C^{(2)} = \left\{ \begin{pmatrix} 0 & 0 \\ 0 & \mathbf{d} \end{pmatrix}, \mathbf{d} \in M_{r+b,r+b}(\mathbb{C}), \mathbf{tr}(\mathbf{d}) = 0 \right\}.$$

Let $\mathcal{H}^{(1)}$ be the projection of the Hua operator $\mathcal{H}$ on $\mathfrak{k}_C^{(1)}$. It maps $C^{\infty}(G/K, \tau_{\nu})$ into $C^{\infty}(G/K, \tau_{\nu} \otimes \mathfrak{k}_C^{(1)})$.

The main result of this section is the following theorem which is the analogue of Theorem 5.2 for non tube domains of type one.

**Theorem 8.1.** Let $\Omega = \mathbf{I}_{r,r+b}$ be a bounded symmetric domain of type one. Suppose $s \in \mathbb{C}$ satisfies the condition (29). Then the Poisson transform $\mathcal{P}_{\lambda_s, \nu}$ is a $SU(r, r+b)$-isomorphism of $\mathcal{B}(S_{r,r+b}; s, \nu)$ onto the space of analytic functions $F$ on $\mathbf{I}_{r,r+b}$ such that

$$(34) \qquad \mathcal{H}^{(1)} F = (r+b)s((r+b)(s-1) + \nu)F \otimes I_r.$$

The bounded domain $\mathbf{I}_{r,r+b}$ is of non tube type, however the characterization of Poisson integrals involves a Hua operator of the second order. As a consequence, the proof of Theorem 8.1 is similar to Theorem 5.2 and [14, Theorem 6.1].

The necessarily condition is guaranteed by the following

**Proposition 8.2.** For any $u \in S_{r,r+b}$ the function $z \mapsto P_{s,\nu}(z, \bar{u})$ satisfies

$$\mathcal{H} P_{s,\nu}(z, \bar{u}) = (r+b)s[(r+b)(s-1) + \nu]P_{s,\nu}(z, \bar{u})I_r.$$
Indeed, using the computations of the proof of Theorem 6.1 and some arguments in the proof of [14, Theorem 5.3] we can extend the formula (31) to any (not necessarily tube) bounded symmetric domain:

$$ \mathcal{H}^1 P_{s,\nu}(z, \bar{u}) = \left[ \begin{aligned} & (\frac{n}{r}s)^2 D(B(z, \bar{z})(\bar{z}^z - \bar{u}^z), z^{\bar{z}} - \bar{u}^{\bar{z}}) - (\frac{n}{r}s)pZ_0 \\ & - (\frac{n}{r}s)\nu D(B(z, \bar{z})(\bar{u}^z - \bar{u}^{\bar{z}}), z^{\bar{z}} - \bar{u}^{\bar{z}}) \end{aligned} \right] P_{s,\nu}(z, \bar{u})I_r. $$

Specifying this formula to the domain $\mathbf{I}_{r,r+b}$ we get proposition.

On the other hand, to prove the sufficiency condition, the key point is to compute the radial part of the Hua operator $\mathcal{H}^{(1)}$. This follows immediately from the proof of Theorem 7.1.

**Theorem 8.3.** Let $f$ be $\mathcal{C}^2$ and $K$-invariant function, then for $a = \sum_{j=1}^r t_j e_j$,

$$ (35) \qquad 4\mathcal{H}^{(1)}f(a) = \sum_{j=1}^{r} \mathcal{H}_j F(t_1, \dots, t_r) D(e_j, \bar{e}_j)^{(1)}, $$

where the scalar-valued operators $\mathcal{H}_j$ are given by

$$ \begin{aligned} \mathcal{H}_j = & (1 - t_j^2)^2 \left( \frac{\partial^2}{\partial t_j^2} + \frac{1}{t_j} (1 + 2\nu) \frac{\partial}{\partial t_j} \right) + \\ & + \sum_{k \neq j} (1 - t_j^2)(1 - t_k^2) \left[ \frac{1}{t_j - t_k} \left( \frac{\partial}{\partial t_j} - \frac{\partial}{\partial t_k} \right) + \frac{1}{t_j + t_k} \left( \frac{\partial}{\partial t_j} + \frac{\partial}{\partial t_k} \right) \right] + \\ & + (2b - 2\nu)(1 - t_j^2) \frac{1}{t_j} \frac{\partial}{\partial t_j} \end{aligned} $$

and $D(e_j, \bar{e}_j)^{(1)}$ is the $\mathfrak{t}_C^{(1)}$ component of $D(e_j, \bar{e}_j) \in \mathfrak{t}_C$.

## 9. RELATIVE DISCRETE SERIES

The Poisson transform is not injective for singular $s$ being in the set $\Lambda_1 \cup \Lambda_2$ in (29). It arises thus a question of understanding the image. A finite subset of such $s$ corresponds to the relative discrete series, i.e. the images constitute discrete components in the decomposition of the $L^2$ space for the bundle. In this final section we find the precise parameters and compute explicitly the corresponding Poisson transforms on some distinguished functions, thus producing elements in the relative discrete series.

Fix $\nu > p - 1$ and $\nu \in p\mathbb{Z}$. Let $\alpha = \nu - p > -1$ and consider the weighted probability measure on $\Omega$

$$ (36) \qquad d\mu_{\alpha}(z) = c_{\alpha}h(z, \bar{z})^{\alpha} dm(z) $$

The group $G$ acts unitarily on $L^2(\Omega, \mu_{\alpha})$ via (10).
The irreducible decomposition of $L^2(\Omega, \mu_\alpha)$ under the $G$-action has been given by Shimeno in [24, Theorem 5.10] where he proved abstractly that all discrete parts called relative discrete series appearing in the decomposition are holomorphic discrete series. In this section we need their explicit realization given by the second author in [32].

Let us introduce the conical functions, see [5]. Let $c_1, \dots, c_r$ be the fixed Jordan frame. Put $e_j = c_1 + \dots + c_j$, for $j = 1, \dots, r$. Let $U_j = \{z \in V : D(c_j, c_j)z = z\}$. Then $U_j$ is a Jordan subalgebra of $V_1 = V_1(e)$ with a determinant polynomial $\Delta_j$. We extend the principal minors $\Delta_j$ to all $V$ via $\Delta_j(z) := \Delta_j(P_{U_j}(z))$, where $P_{U_j}$ is the orthogonal projection onto $U_j$. Notice that $\Delta_r(z) = \Delta(z) = \det(z)$.

For any $\underline{m} = (m_1, \dots, m_r) \in \mathbb{C}^r$, consider the associated conical function

$$ \Delta_{\underline{m}}(z) := \Delta_1^{m_1-m_2}(z) \Delta_2^{m_2-m_3}(z) \cdots \Delta_r^{m_r}(z). $$

If $z = \sum_{j=1}^r z_j c_j$ then $\Delta_{\underline{m}}(z) = \prod_{j=1}^r z_j^{m_j}$.

Denote

$$ \ell = \begin{cases} \frac{\alpha+1}{2} - 1 = \frac{\nu-p-1}{2} & \text{if } \alpha \text{ is an odd integer} \\ \lfloor \frac{\alpha+1}{2} \rfloor = \lfloor \frac{\nu-p+1}{2} \rfloor & \text{otherwise} \end{cases} $$

here $[t]$ stands for the integer part of $t \in \mathbb{R}$. Define

$$ (37) \qquad D_{\nu} = \{\underline{m} = \sum_{j=1}^{r} m_j \gamma_j, 0 \le m_1 \le \dots \le m_r \le \ell, m_j \in \mathbb{Z}\}. $$

We let $A_{\underline{m}}^{2,\alpha}$ to be the subspace of $L^2(\Omega, \mu_\alpha)$ generated by the function $\bar{\Delta}_{\underline{m}}(q(z))$, for $\underline{m} \in D_\nu$, where

$$ q(z) = \bar{z}^z $$

is quasi-inverse of $\bar{z}$ with respect to $z$.

We reformulate [24, Theorem 5.10] and [32, Theorem 4.7, remark 4.8] in the following.

**Theorem 9.1.** The relative discrete series representations appearing in $L^2(\Omega, \mu_\alpha)$ are all holomorphic discrete series of the form $A_{\underline{m}}^{2,\alpha}$ with $\underline{m} \in D_\nu$. The highest weight vector of $A_{\underline{m}}^{2,\alpha}$ is given by $\bar{\Delta}_{\underline{m}}(q(z))$.

We can now state the main theorem of this section.
**Theorem 9.2.** Let $\underline{\delta} = (\delta, \delta, \dots, \delta)$ such that $s_{\underline{r}}^n = \frac{n}{r} + \delta - \nu$ and $0 \le \delta < \left[\frac{\nu-p}{2}\right]$. Then the Poisson transform $\mathcal{P}_{\lambda_s, \nu}$ is a $G$-equivariant Sz\"ge\" type map from the space $\mathcal{B}(S, s, \nu)_K$ of $K$-finite elements onto the $K$-finite elements in the relative discrete series $A_{\underline{\delta}}^{2, \alpha}$.

This theorem is consequence of the following proposition.

**Proposition 9.3.** Under the same conditions as in Theorem 9.3 we have

$$ (38) \qquad (\mathcal{P}_{s,\nu} \bar{\Delta}_{\underline{\delta}})(z) = \int_S P_{s,\nu}(z, u) \bar{\Delta}_{\underline{\delta}}(u) du = \frac{(s_{\underline{r}}^n)_{\underline{\delta}}}{(\frac{n}{r})_{\underline{\delta}}} \bar{\Delta}_{\underline{\delta}}(q(z)). $$

Let us first prove the formula below

**Lemma 9.4.** For any highest weight $\underline{m}$ the following formula holds

$$ (39) \qquad \Delta(z)^{\delta} \bar{\Delta}(w)^{\delta} K_{\underline{m}}(z, w) = c(\underline{m}, \delta) K_{\underline{m}+\delta}(z, w) $$

$$ \text{where } c(\underline{m}, \delta) = \left(\frac{n}{r} + \underline{m}\right)_{\underline{\delta}} := \prod_{j=1}^{r} \left(\frac{a}{2}(r-j) + 1 + m_j\right)_{\underline{\delta}} $$

*Proof.* Since $f(z) \mapsto \Delta(z)^{\delta} f(z)$ is a $K$-intertwining map from $\mathcal{P}_{\underline{m}}$ onto $\mathcal{P}_{\underline{m}+\delta}$ we have that $\Delta(z)^{\delta} \bar{\Delta}(w)^{\delta} K_{\underline{m}}(z, w)$ is equal to $K_{\underline{m}+\delta}(z, w)$ up to a constant $c(\underline{m}, \delta)$. Now taking $z = w = e$ and using [5, Lemma 3.1, Theorem 3.4] we find that the constant is

$$ c(\underline{m}, \delta) = \frac{K_{\underline{m}}(e, e)}{K_{\underline{m}+\delta}(e, e)} = \frac{d_{\underline{m}}}{(n/r)_{\underline{m}}} \frac{(n/r)_{\underline{m}+\delta}}{d_{\underline{m}+\delta}} = \left(\frac{n}{r} + \underline{m}\right)_{\underline{\delta}}. $$

□

*Proof of Proposition 9.3.* Put $\sigma = s_{\underline{r}}^n$. We compute the image $\mathcal{P}_{s,\nu} \bar{\Delta}_{\underline{\delta}}$,

$$ \begin{aligned} (\mathcal{P}_{s,\nu} \bar{\Delta}_{\underline{\delta}})(z) &= \int_S h(z, u)^{-\nu} h(z, z)^{\sigma} h(z, u)^{-\sigma} h(u, z)^{-\sigma} \bar{\Delta}_{\underline{\delta}}(u) du \\ &= h(z, z)^{\sigma} \int_S h(z, u)^{-\nu-\sigma} h(u, z)^{-\sigma} \bar{\Delta}_{\underline{\delta}}(u) du \end{aligned} $$

We use the Faraut-Koranyi expansion [4, Theorem 3.8] of the reproducing kernels $h(z, u)^{-\nu-\sigma}$ and $h(u, z)^{-\sigma}$ so that,

$$ (\mathcal{P}_{s,\nu} \bar{\Delta}_{\underline{\delta}})(z) = h(z, z)^{\sigma} \int_S \left[ \sum_{\underline{m} \ge 0} (\nu + \sigma)_{\underline{m}} K_{\underline{m}}(z, u) \times \sum_{\underline{m}' \ge 0} (\sigma)_{\underline{m}'} K_{\underline{m}'}(u, z) \right] \bar{\Delta}_{\underline{\delta}}(u) du $$
which by Lemma 9.4 is

$$ (\mathcal{P}_{s,\nu}\bar{\Delta}_{\underline{\delta}})(z) = h(z, z)^{\sigma} \int \left[ \sum_{\mathbf{m} \ge 0} (\nu + \sigma)_{\mathbf{m}} K_{\mathbf{m}+\underline{\delta}}(z, u) \Delta_{\underline{\delta}}(z)^{-1} c(\mathbf{m}, \delta) \times \right. \\ \left. \qquad \times \sum_{\mathbf{m}' \ge 0} (\sigma)_{\mathbf{m}'} K_{\mathbf{m}'}(u, z) \right] du \\ = h(z, z)^{\sigma} \sum_{\mathbf{m} \ge 0} \left[ (\nu + \sigma)_{\mathbf{m}} (\sigma)_{\mathbf{m}+\underline{\delta}} c(\mathbf{m}, \delta) \Delta_{\underline{\delta}}(z)^{-1} \times \right. \\ \left. \qquad \times \int K_{\mathbf{m}+\underline{\delta}}(z, u) K_{\mathbf{m}+\underline{\delta}}(u, z) du \right] $$

The last equality follows from the Schur orthogonality relation. Furthermore, since the ratio of the Fischer inner product and the standard K-invariant inner product of $L^2(S)$ is constant, see [5, Corollary 3.5] or [28], we obtain

$$ (\mathcal{P}_{s,\nu}\bar{\Delta}_{\underline{\delta}})(z) = h(z, z)^{\sigma} \sum_{\mathbf{m} \ge 0} \left[ (\nu + \sigma)_{\mathbf{m}} (\sigma)_{\mathbf{m}+\underline{\delta}} c(\mathbf{m}, \delta) \Delta_{\underline{\delta}}(z)^{-1} \times \right. \\ \left. \qquad \times \left(\frac{n}{r}\right)^{-1}_{\mathbf{m}+\underline{\delta}} K_{\mathbf{m}+\underline{\delta}}(z, z) \right] $$

which again by Lemma 9.4 is

$$ \begin{align*} (\mathcal{P}_{s,\nu}\bar{\Delta}_{\underline{\delta}})(z) &= h(z, z)^{\sigma}\bar{\Delta}_{\underline{\delta}}(z) \sum_{\mathbf{m} \ge 0} (\nu + \sigma)_{\mathbf{m}} (\sigma)_{\mathbf{m}+\underline{\delta}} \left(\frac{n}{r}\right)^{-1}_{\mathbf{m}+\underline{\delta}} \times \\ & \qquad \times \bar{\Delta}_{\underline{\delta}}(z) K_{\mathbf{m}}(z, z) \\ &= h(z, z)^{\sigma} \frac{(\sigma)_{\underline{\delta}}}{(\frac{n}{r})_{\underline{\delta}}} \bar{\Delta}_{\underline{\delta}}(z) \sum_{\mathbf{m} \ge 0} (\sigma + \delta)_{\mathbf{m}} K_{\mathbf{m}}(z, z) \\ &= h(z, z)^{\sigma} \frac{(\sigma)_{\underline{\delta}}}{(\frac{n}{r})_{\underline{\delta}}} \bar{\Delta}_{\underline{\delta}}(z) h(z, z)^{-(\sigma+\delta)} \\ &= \frac{(\sigma)_{\underline{\delta}}}{(\frac{n}{r})_{\underline{\delta}}} \frac{\bar{\Delta}_{\underline{\delta}}(z)}{h(z, z)^{\delta}} \end{align*} $$

Finally since $\frac{\bar{\Delta}(z)}{h(z,z)} = \bar{\Delta}(q(z))$, see [32, Corollary 4.4], the proof is completed. □

REFERENCES

1. Berline, N.; Vergne, M. Équations de Hua et noyau de Poisson. *Noncommutative harmonic analysis and Lie groups* (Marseille, 1980), 1–51, Lecture Notes in Math., **880**, Springer, Berlin-New York, 1981.
2. Dooley, A. H.; Ørsted, B; Zhang, G. Relative discrete series of line bundles over bounded symmetric domains *Ann. Inst. Fourier* **46** (1996), 1011-1026.

3. Engliš, M.; Peetre, J. Covariant Cauchy-Riemann operators and higher Laplacians on Kähler manifolds. *J. Reine Angew. Math.* **478** (1996), 17–56.

4. Faraut, J.; Korányi, A. Function spaces and reproducing kernels on bounded symmetric domains. *J. Funct. Anal.* **88** 1990, 64–89.

5. Faraut, J.; Korányi, A. Analysis on Symmetric Cones, *Oxford Mathematical Monographs*, Clarendon Press, Oxford, 1994.

6. Helgason, S. A duality for symmetric spaces with applications to group representations. *Advances in Math.* **5** (1970), 1–154.

7. Helgason, S. Groups and geometric analysis. Integral geometry, invariant differential operators, and spherical functions. Pure and Applied Mathematics, 113. Academic Press, Inc., Orlando, FL, 1984.

8. Hua, L. K. Harmonic analysis of functions of several complex variables in the classical domains. *American Mathematical Society, Providence, R.I.* 1963 iv+164 pp

9. Johnson, K. D. Generalized Hua-operators and parabolic subgroups. The cases of SL(n,C) and SL(n,R). *Trans. Amer. Math. Soc.* **281** (1984), 417429.

10. Johnson, K. D.; Korányi, A. The Hua operators on bounded symmetric domains of tube type. *Ann. of Math. (2)* **111** (1980), no. 3, 589–608.

11. Kashiwara, M.; Kowata, A.; Minemura, K.; Okamoto, K.; Oshima, T.; Tanaka, M. Eigenfunctions of invariant differential operators on a symmetric space. *Ann. of Math. (2)* **107** (1978), no. 1, 1–39.

12. Knapp, A. W.; Wallach, N. R. Szegő kernels associated with discrete series. *Invent. Math.* **34** (1976), 163–200.

13. Korányi, A. Poisson transform for line bundles from the Shilov boundary to bounded symmetric domains. Preprint, March 2011, to appear.

14. Koufany, K.; Zhang, G. Hua operators on bounded symmetric domains]Hua operators and Poisson transform for bounded symmetric domains. *J. Funct. Anal.* **236** (2006), 546–580.

15. Lassalle, M. Les équations de Hua d'un domaine borné symétrique du type tube. *Invent. Math.* **77** (1984), no. 1, 129–161.

16. Loos, O. Bounded symmetric domains and Jordan pairs. *University of California, Irvine*, 1977.

17. Oshima, T. A definition of boundary values of solutions of partial differential equations with regular singularities. *Publ. Res. Inst. Math. Sci.* **19** (1983), no. 3, 1203–1230.

18. Oshima, T. Boundary value problems for systems of linear partial differential equations with regular singularities. *Adv. Stud. Pure Math.*, **4** (1984) 391–432.

19. Oshima, T. A realization of semisimple symmetric spaces and construction of boundary value maps. *Adv. Stud. Pure Math.*, **14** (1988) 603–650.

20. Oshima, T.; Shimeno, N. Boundary value problems on Riemannian Symmetric Spaces of the noncompact Type. arXiv:1011.1314

21. Satake, I, Algebraic structures of symmetric domains, Iwanami Shoten and Princeton Univ. Press, 1980, Tokyo and Princeton, NJ.

22. Schlichtkrull, H. One-dimensional K-types in finite dimensional representations of semisimple Lie groups : A generalization of Helgason’s theorem, *Math. Scand.* **54** (1984), 279–294
23. Shimeno, N. Eigenspaces of invariant differential operators on a homogeneous line bundle on a Riemannian symmetric space. *J. Fac. Sci. Univ. Tokyo Sect. IA Math. 37* (1990), 201–234.

24. Shimeno, N. The Plancherel formula for the spherical functions with one-dimensional $K$-type on a simply connected simple Lie group of hermitian type. *J. Funct. Anal. 121* (1994), 331–388.

25. Shimeno, N. Boundary value problems for the Shilov boundary of a bounded symmetric domain of tube type. *J. Funct. Anal. 140* (1996), no. 1, 124–141.

26. Shimeno, N. Boundary value problems for various boundaries of Hermitian symmetric spaces, *J. Funct. Anal. 170* (2000), no.2, 265–285.

27. Shimura, G. Invariant differential operators on Hermitian symmetric spaces, *Ann. Math. 132* (1990), 232–272.

28. Upmeier, H. Toeplitz operators on bounded symmetric domains. *Trans. Amer. Math. Soc. 280* (1983), 221–237.

29. Yan, Z. A class of generalized hypergeometric functions in several variables. *Canad. J. Math. 44* (1992), no. 6, 1317–1338.

30. Zhang, G. Shimura invariant differential operators and their eigenvalues, *Math. Ann. 319* (2001), 235–265.

31. Zhang, G. Invariant differential operators on hermitian symmetric spaces and their eigenvalues, *Israel J. Math. 119* (2000), 157–185.

32. Zhang, G. Nearly holomorphic functions and relative discrete series of weighted $L^2$-spaces on bounded symmetric domains. *J. Math. Kyoto Univ.* **42** (2002), 207-221.

KHALID KOUFANY – INSTITUT ÉLIE CARTAN, UMR 7502, UNIVERSITÉ HENRI POINCARÉ (NANCY 1) B.P. 239, F-54506 VANDŒUVRE-LÈS-NANCY CEDEX, FRANCE

E-mail address: khalid.koufany@iecn.u-nancy.fr

GENKAI ZHANG – DEPARTMENT OF MATHEMATICS, CHALMERS UNIVERSITY OF TECHNOLOGY AND GÖTEBORG UNIVERSITY, S-41296 GÖTEBORG, SWEDEN

E-mail address: genkai@math.chalmers.se