ORBITS OF TORI EXTENDED BY FINITE GROUPS AND
THEIR POLYNOMIAL HULLS: THE CASE OF CONNECTED
COMPLEX ORBITS
V.M. GICHEV
Abstract. Let V be a complex linear space, G ⊂ GL(V ) be a compact group.
We consider the problem of description of polynomial hulls cGv for orbits Gv,
v ∈ V , assuming that the identity component of G is a torus T . The paper
contains a universal construction for orbits which satisfy the inclusion Gv ⊂
TCv and a characterization of pairs (G, V ) such that it is true for a generic
v ∈ V . The hull of a finite union of T -orbits in TCv can be distinguished in
closTCv by a finite collection of inequalities of the type |z1|
s1 . . . |zn|
sn ≤ c.
In particular, this is true for Gv. If powers in the monomials are independent
of v, Gv ⊂ TCv for a generic v, and either the center of G is finite or TC has an
open orbit, then the space V and the group G are products of standard ones;
the latter means that G = SnT , where Sn is the group of all permutations
of coordinates and T is either Tn or SU(n) ∩ Tn, where Tn is the torus of all
diagonal matrices in U(n). The paper also contains a description of polynomial
hulls for orbits of isotropy groups of bounded symmetric domains. This result
is already known, but we formulate it in a different form and supply with a
shorter proof.
Introduction
Let V be a finite-dimensional complex linear space and G ⊂ GL(V ) be a compact
subgroup of GL(V ). We consider the problem of description of polynomially convex
hulls for orbits Ov = Gv, v ∈ V . The polynomially convex hull (or polynomial hull)
Q̂ of a compact set Q ⊂ V is defined as
Q̂ = {z ∈ V : |p(z)| ≤ sup
|p(ζ)| for all p ∈ P(V )},(0.1)
where P(V ) is the algebra of all holomorphic polynomials on V . It is usually
difficult to find Q̂. For Q = Gv, the answer is known if G is an isotropy group of a
bounded symmetric domain in Cn. Paper [9] contains a description of G-invariant
polynomially convex compact sets, including hulls of orbits (Q ⊂ V is polynomially
convex if Q̂ = Q); it continues paper [10] and uses results of [8]. On the other
hand, it is known that an orbit of a compact linear group is polynomially convex
if and only if the complex orbit GCv is closed and Gv is its real form ([7]). The
cases G = U(2), SU(2) were considered in [1], [4]. The problem of determination
of polynomial hulls of orbits admits the following natural generalization: given a
homogeneous space M of a compact group G, describe maximal ideal spaces MA of
G-invariant closed subalgebras A of C(M), where C(M) is the Banach algebra of all
1991 Mathematics Subject Classification. Primary 32E20; Secondary 32M15, 32M05.
Key words and phrases. Polynomial hulls, bounded symmetric domains.
The author was partially supported by RFBR Grants 06-08-01403 and 06-07-89051.
http://arxiv.org/abs/0704.1095v1
2 V.M. GICHEV
continuous complex-valued functions on M with the sup-norm. If A is generated by
a finite-dimensional invariant subspace, then MA can be realized as the polynomial
hull of an orbit. Paper [6] contains a description of MA for bi-invariant algebras
on compact groups and partial results on spherical homogeneous spaces. Maximal
ideal spaces for U(n)-invariant algebras on spheres in C
n are described in [11].
In this paper we consider orbits Gv of groups G = FT , where F ⊆ G is a finite
subgroup and T is a torus, such that GCv = TCv. Let t ⊆ gl(V ) be the Lie algebra
of T and set tR = it, TR = exp(tR). Suppose that v ∈ V has a trivial stable
subgroup in T and let X ⊂ TRv be finite. The hull of Y = TX admits a simple
description. If X = {v}, then Ŷ = T̂ v is the closure of T exp(CT )v, where CT is
a cone in tR. If TC is closed, then Ŷ = T exp(QX)v, where QX ⊆ tR is a convex
polytope (the convex hull of the inverse image of X for the mapping ξ → exp(ξ)v,
ξ ∈ tR). Any segment in QX corresponds to an analytic strip or an annulus in
Ŷ . In general, Ŷ is the union of T̂ u, where u runs over exp(QX)v. Also, Ŷ is
distinguished in closTCv by a finite family of monomial inequalities of the type
|z1|s1 . . . |zn|sn ≤ c,(0.2)
where c ≥ 0 and s = (s1, . . . , sn) ∈ Rn depend on v and X . Vectors s correspond
to normals of faces of CT +QX .
Thus, the problem of determination of Ĝv is not difficult if Gv ⊂ TCv. The
latter is equivalent to the assumption that the complex orbit GCv is connected. In
Example 3.4, we give a construction for orbits which satisfy this condition; here is
a sketch. The group G = FT acts on the space V = C(K), where K is a finite
F -invariant subset of t∗: F acts naturally on C(K), t = t∗∗ is naturally embedded
into C(K), and T = exp(t) acts on C(K) by multiplication. If v ∈ C(K) is an
F -invariant function, then Gv ⊂ TCv. According to Theorem 3.5, each connected
complex orbit can be realized in this way. Further, we describe pairs (V,G) such
Gv ⊂ TCv for a generic v ∈ V.(0.3)
By Theorem 4.3, under the additional assumption that the complex linear span of
TCv coincides with V , this happens if and only if the group GCZ, where Z is the
centralizer of G in GL(V ), has an open orbit in V . There are two extreme cases:
(A) Z ⊆ GC; (B) G has a finite center. An example for (A) is the group G = SnTn
acting in Cn, where Tn is the torus of all diagonal matrices in U(n) and Sn is the
group of all permutations of coordinates. Replacing Tn with SU(n)∩Tn, we get an
example for (B). Example 4.4 contains a construction for pairs (V,G) that satisfy
(0.3). Theorem 4.5 states that the construction is a universal one. In Theorem 4.10,
we determine pairs which satisfy (0.3) and the following condition:
vectors s in (0.2) are independent of v.(0.4)
The paper also contains a description of hulls Ĝv for G = Aut0(D), where D is a
bounded symmetric domain in the canonical realization and Aut0(D) is the stable
subgroup of zero, which coincides with the group of all linear automorphisms of
D. These hulls have already been described: the final step was done in paper [9],
which essentially used [10], partial results appear in [14] and [8]. Most of them
use the technique of Jordan triples and Jordan algebras. We use Lie theory, in
particular, an explicit construction of paper [15] for a maximal abelian subspace a.
A compact group acting in a Euclidean space is called polar if there exists a subspace
ORBITS OF TORI EXTENDED BY FINITE GROUPS AND THEIR POLYNOMIAL HULLS: THE CASE OF CONNECTED COMPLEX ORBITS3
(a Cartan subspace) such that each orbit meets it orthogonally. The group G is
polar in the ambient linear space d, and a is the Cartan subspace for G. Real polar
representations are classified in paper [3]; they are orbit equivalent (i.e., have the
same orbits) to isotropy representations of Riemannian symmetric spaces. If D is
a polydisc Dn ⊂ Cn, where D is the unit disc in C, then G = SnTn; the polynomial
hulls Ĝv are determined by the inequalities
µk(z) ≤ µk(v),(0.5)
where k = 1, . . . , n and µk are defined by
µk(z) = max{|zσ(1) . . . zσ(k)| : σ ∈ Sn}.(0.6)
The general case can be reduced to this one in the following way. Any bounded
symmetric domain D ⊂ d of rank n admits an equivariant embedding of Cn to
d, which induces an embedding of Dn to D, such that Rn ⊂ Cn is the maximal
abelian subspace a, and, for any v ∈ a , the hull of Aut0(D)v is the orbit of the hull
of Aut0(D
n)v. Each µk(z) has a unique continuation to a K-invariant function on
d. The extended functions determine hulls by the same inequalities. Moreover, they
are plurisubharmonic and can be treated as products of singular values of z ∈ d or
as norms of exterior powers of adjoint operators in suitable spaces. The subsystem
of long roots of the restricted root system (i.e., the root system for a) has type nA1;
this defines the above embedding Cn → d. Furthermore, this makes it possible to
determine hulls in terms of the adjoint representation (Theorem 5.7). Thus, there
is no need to consider different types of domains separately.
The reduction to the case of a torus extended by a finite group, which is de-
scribed above, is contained in Section 5 (in papers [9], [14], the problem is also
reduced to this case by another method). It does not use essentially the results
of the previous sections (only Proposition 3.2, in proof of Theorem 5.7). These
extensions satisfy conditions (0.3) and (0.4); in addition, they possess the property
that the complexified groups have open orbits. According to Theorem 4.10, any
group with these properties is the product of groups SnT
n acting in Cn; it admits
a natural realization as a group of automorphisms of a bounded symmetric domain
(Corollary 5.3).
The following simple examples illustrate the case Gv 6⊆ TCv and show that
condition (0.3) is essential. Let G = SnT
n, and let ǫ1, . . . , ǫn be the standard base
in Cn. Then Ĝǫ1 is the closure the union of discs Dǫk, k = 1, . . . , n. Set H = SnT,
where T acts by z → eitz, t ∈ R, z ∈ Cn. Then Ĥǫ1 = Ĝǫ1. For v = ǫ1 + ǫ2, Ĝv is
the closure of the union of
bidiscs but Tn contains no proper torus T such that
Ĝv = Ĥv for H = SnT . However, for any subgroup F ⊆ Sn which acts transitively
on 2-sets and H = FTn we have Ĝv = Ĥv.
1. Preliminaries
We keep the notation of Introduction, in particular, (0.1) and (0.6). Linear
spaces are supposed to be finite dimensional and complex unless the contrary is
explicitly stated. ”Generic” means ”in some open dense subset”. Throughout the
paper, we use the following notation:
D and T are the open unit disc and the unit circle in C, respectively;
V denotes a complex linear space (except for Section 5);
4 V.M. GICHEV
if V is equipped with a linear base identifying it with Cn, then Tn is the
group of all diagonal unitary transformations;
Zn2 consists of all transformations in T
n with eigenvalues ±1;
ǫ1, . . . , ǫn is the standard base in C
n and Rn;
Rn+ is the set of vectors in R
n with positive entries;
SK denotes the group of all permutations of a finite setK; ifK = {1, . . . , n},
then SK = Sn;
C(K) is the algebra of all complex-valued functions on K;
1 is the identity of C(K);
G ⊂ GL(V ) is a compact group whose identity component is a torus T
(except for Section 5);
t ⊂ gl(V ) is the Lie algebra of T , tR = it, tC = t+ tR;
TR = exp(tR), TC = exp(tC);
C∗ = TC = C \ {0};
Ť = Hom(T,T) is the dual group to T ;
Aut(D) is the group of all holomorphic automorphisms of a domain D ⊂ V ,
Aut0(D) = Aut(D) ∩GL(V );
coneX denotes the least convex cone which contains the set X ;
convX is the convex hull of X ;
closX is the closure of X ;
spanFX is the linear span of X over the field F = C,R,Q.
Clearly, exp is bijective on tR and TR ∼= TC/T . The differentiating at the identity e
defines an embedding of Ť into the dual space t∗: χ → −ideχ, where χ ∈ Ť . This
is a lattice in the vector group t∗, moreover, T ∼= t/L, where L is the dual lattice
to Ť in t. For χ ∈ Ť , let
Vχ = {v ∈ V : gv = χ(g)v for all g ∈ T }
be the corresponding isotypical component of V . Then
⊕Vχ.(1.1)
We assume that V is equipped with a G-invariant inner product 〈 , 〉. Then de-
composition (1.1) is orthogonal. Let spec(v) denote the spectrum of v ∈ V (the set
of χ ∈ Ť such that the χ-component of v is nonzero); for X ⊆ V ,
spec(X) = ∪x∈X spec(x).
We say that T has a simple spectrum if
dimVχ ≤ 1(1.2)
for all χ ∈ Ť . If (1.2) is true, then there exists a unique (up to scaling factors)
orthogonal base in V which agree with (1.1) and a unique maximal torus Tn in
GL(V ) which contains T . In what follows, we assume that (1.2) holds; we shall see
in the next section that such assumption is not restrictive. Thus, we may fix an
identification
V = Cn = C(K),(1.3)
where K = {1, . . . , n}. If F is a subgroup of SK , then C(K)F denotes the set
of all F -invariant functions on K; clearly, 1 ∈ C(K)F . Further, (C∗)n is the
multiplicative group of all invertible functions in C(K), Tn consists of functions
with values in T, and (Tn)C = (C∗)n. The Lie algebra of Tn is realized as iRn ⊂ Cn.
ORBITS OF TORI EXTENDED BY FINITE GROUPS AND THEIR POLYNOMIAL HULLS: THE CASE OF CONNECTED COMPLEX ORBITS5
The embedding T → Tn induces embeddings of the Lie algebra and the fundamental
group: t → iRn, π1(T ) → iZn ⊂ iRn, respectively. Let Γ be the image of π1(T ).
Then spanR Γ = t; moreover, t∩ iZn = Γ and t/Γ = T . The dual mapping Ťn → Ť ,
which is defined by the restriction of characters e−i〈x,y〉, where x ∈ iZn, to t, is
the orthogonal projection πt : iZ
n → t. Thus, Γ is a subgroup of finite index in
Ť = πtiZ
n. Vectors in spanQ Ť are called rational. The image of t in iR
n can be
distinguished by linear equations with integer coefficients. Hence, clos(TCv), for
a generic v ∈ V , is the set of all solutions to a finite number of equalities with
holomorphic monomials. Thus, Y ⊂ TC implies Ŷ ⊂ clos(TCv). Set
CT = t
R ∩ clos(−Rn+),(1.4)
The cone iCT is dual to cone(specV ) ⊆ t∗ ⊆ iRn. If −ξ ∈ clos(Rn+), then ι =
limt→+∞ exp(tξ) is an idempotent in C(K) such that the multiplication by the
complementary idempotent 1− ι is a projection onto spanC(spec(ξ)). Set
IT = {limt→+∞ exp(tξ) : ξ ∈ CT }.(1.5)
Clearly, IT is finite and contains 1.
Lemma 1.1. The closure of exp(CT ) is equal to IT exp(CT ).
Proof. Due to the evident inclusion clos(exp(CT )) ⊇ IT exp(CT ), it is sufficient to
prove that the set ST = IT exp(CT ) is closed. Clearly, ST is an abelian semigroup.
The cone CT is polyhedral; hence, it is finitely generated:
CT = cone{ξ1, . . . , ξm},
where R+ξk are the extreme rays of CT , k = 1, . . . ,m. Obviously, IT is a finite
semigroup, which is generated by the idempotents limt→+∞ exp(tξk). Thus, the cor-
respondence (e−t1 , . . . , e−tm) → exp(t1ξ1 + . . . , tmξm) defines a mapping of (0, 1]m
onto exp(CT ), which continuously extends to [0, 1]
m. It follows that its image is
closed and coincides with ST . �
Note that there is a natural one-to-one correspondence between IT and the set
of faces of CT .
2. Hulls of finite unions of T -orbits in a TC-orbit
Let v ∈ TC. If v =
χ∈Ť vχ, where vχ ∈ Vχ, g ∈ TC, and u = gv, then
χ∈Ť χ(g)vχ. Since χ(g) 6= 0 for all g ∈ G and χ ∈ Ť , we get
u ∈ TCv =⇒ spec(u) = spec(v);(2.1)
dim (Vχ ∩ spanC Tv) ≤ 1 for all v ∈ V and χ ∈ Ť .(2.2)
Thus, the assumption that T has a simple spectrum in V is not restrictive in
the problem of description of polynomial hulls of orbits Gv such that Gv ⊂ TCv.
Clearly, Dn = T̂n in L(V ). For each x ∈ CT and any polynomial p on L(V ), the
holomorphic function f(ζ) = p(exp(ζx)) is bounded in the halfplane Π : Re ζ ≥ 0.
Hence, exp(Π) is contained in T̂ . On the other hand, if z ∈ Dn ∩ TC, then z =
t exp(x) for some t ∈ T and x ∈ CT (the polar decomposition). By Lemma 1.1,
T̂ = clos(Dn ∩ TC) = T clos(exp(CT )) = TIT exp(CT ).
6 V.M. GICHEV
If v ∈ (C∗)n, then (C∗)nv = (C∗)n, and the mapping z → zv is a linear nondegen-
erate transformation of Cn. Therefore,
v ∈ (C∗)n =⇒ T̂ v = T̂ v = TIT exp(CT )v.(2.3)
For an arbitrary v ∈ V = Cn, set
CvT = {ξ ∈ tR : ξk ≤ 0 if vk 6= 0, k = 1, . . . , n}.
Applying (2.3) to spanC(spec(v)) = C
nv, we get
T̂ v = T clos(exp(CvT )v.(2.4)
Clearly, CvT depends only on spec(v). For s ∈ Rn and z ∈ (C∗)n, set
νs(z) =
|zk|sk .
If sk ≥ 0, then the k-th factor in (C∗)n can be replaced with C (i.e., νs continuously
extends to this product).
It is well known that for any holomorphically convex T -invariant set U ⊆ TC,
the set log(U ∩ TR) ⊆ tR is convex. In particular, this is true for sets of g ∈ TC
such that gv ∈ T̂X, where X ⊂ TCv, v ∈ V . Nevertheless, it is convenient to have
an explicit construction of an analytic strip (or an annulus, if it is periodic) in a
TC-orbit, which corresponds to a segment that joins two points in tR; it is contained
in the following lemma. Set
S = {z ∈ C : 0 ≤ Im z ≤ 1}.
Lemma 2.1. Let v ∈ Cn and u ∈ TRv. Then, there exists ξ ∈ tR such that
λ(z) = exp(zξ)v(2.5)
is a holomorphic mapping λ : S → TCv which satisfies conditions
λ(∂S) ⊆ Tv ∪ Tu,
λ(0) = v, λ(1) = u.
If the stable subgroup of v in TR is trivial, then ξ is unique.
Proof. These properties hold for ξ ∈ tR such that exp(ξ)v = u; such a ξ exists, since
exp is a bijection tR → TR. The last assertion is clear. �
If ξ ∈ CT , then (2.5) defines an analytic halfplane in T̂ v; for Γ-rational ξ, λ is
periodic and defines an analytic disc in T̂ v. Together with Lemma 2.1 this gives a
characterization of hulls for finite unions of T -orbits in TC. Suppose that X ⊂ TRv
is finite and the stable subgroup of v in T is trivial. Then, the inverse to the
mapping x → exp(x)v is well defined. Let us denote it by logv and set
QX = conv(logv X),(2.6)
PX = QX + CT .(2.7)
The set PX is a convex polyhedron, which is unbounded if CT 6= 0. Hence, there
exists a finite set NX ⊂ Rn and, for each s ∈ NX , real numbers cs such that
PX = {x ∈ tR : 〈x, s〉 ≤ cs for all s ∈ NX}.(2.8)
The set NX consists of vectors orthogonal to faces of PX , whose projections into
spanR PX look outside of it; clearly, it is not unique in general.
ORBITS OF TORI EXTENDED BY FINITE GROUPS AND THEIR POLYNOMIAL HULLS: THE CASE OF CONNECTED COMPLEX ORBITS7
Proposition 2.2. Let v ∈ (C∗)n. Suppose that Y ⊂ TCv is a finite union of
T -orbits (including Tv), and set X = TRv ∩ Y . Then X is finite and
Ŷ = clos (T exp(PX)v)(2.9)
= clos {z ∈ (C∗)n : νs(z) ≤ ecsνs(v), s ∈ NX}(2.10)
u∈exp(QX )v
T̂ u(2.11)
= T exp(PX)IT v,(2.12)
where QX , PX ,NX are as above and IT is defined in (1.5).
Proof. Due to the polar decomposition, the set Tu∩TRv, for each u ∈ TCv, is non-
void and consists of a single point. Hence, X is finite and Y = TX . The inclusion
(expQX)v ⊆ Ŷ follows from Lemma 2.1 and Phragmén–Lindelöf Principle. The in-
clusion exp(CT )u ⊆ T̂ u is true for any u ∈ Cn. Since it holds for all u ∈ T exp(QX),
the left-hand side of (2.9) includes the right-hand side. If z = exp(ξ)v, where ξ ∈ tC,
then zk = e
ξkvk, k = 1, . . . , n; due to (2.8), this implies that the right-hand side
of (2.9) coincides with (2.10). According to (2.3), the right-hand side of (2.9) and
(2.11) intersect TCv by the set
T exp(PX)v = exp(QX)T exp(CT )v;
clearly, it is dense in (2.11). Since QX is compact, the set (2.11) is closed. The
compactness of QX , the above equality, and Lemma 1.1 imply that (2.12) is closed;
hence, it is the same as the right-hand side of (2.9).
Each of the sets (2.9)–(2.12) includes Y . Thus, it remains to prove that (2.12)
is polynomially convex. If x ∈ tR \ PX , then there exists s ∈ Rn such that
sup{〈y, s〉 : y ∈ PX} < 〈x, s〉 .(2.13)
Since QX is compact, the linear functional on t in the right-hand side of (2.13) must
be nonnegative on CT . According to (1.4), we may assume that s ∈ closRn+. It
follows that (2.13) holds in a neighborhood of s in closRn+. Thus, s can be assumed
rational (hence, integer) with strictly positive entries. Then, p(z) = zs11 . . . z
a holomorphic polynomial such that |p | separates exp(x)v and T clos(exp(PX)v).
Therefore,
Ŷ ∩ TCv = T exp(PX)v.
For any ι ∈ IT , the projection z → ιz commutes with T . This makes it possible to
apply the above arguments to the vector ιv, the set ιX , and to the restriction of T
to ιCn. Consequently,
ι̂Y ∩ TCιv = T exp(PιX)ιv = ιT exp(PX)v(2.14)
(clearly, ι exp(PX)v = exp(PιX)ιv). By (1.5), ιY ⊆ Ŷ , hence, ι̂Y ⊆ Ŷ ; on the other
hand, ιŶ ⊆ ι̂Y since p◦ι is a polynomial on Cn for any polynomial p on ιCn. Thus,
ιŶ = ι̂Y = Ŷ ∩ ιCn. Together with (2.14), this implies the polynomial convexity of
(2.12). �
If T = Tn, then Proposition 2.2 follows from the well-known characterization of
polynomially convex Reinhardt domains.
Corollary 2.3. For any v ∈ (C∗)n, the orbit TCv is closed in Cn if and only if Tv
is polynomially convex, and this is equivalent to CT = 0. Then, Ŷ = T exp(QX)v
for all Y,X as above.
8 V.M. GICHEV
Proof. The orbit TCv is closed if and only if the convex hull of spec(v) = spec(Cn)
contains 0 in its relative interior (see, for example, [13, Proposition 6.15]). Since
T ⊆ GL(n,C), the set spec(Cn) is generating in t∗. Hence, TCv is closed if and
only if CT = 0; by (2.4), this is equivalent to T̂ v = Tv. Then, Ŷ = T exp(QX)v by
(2.9) and (2.1). �
There is a version of the first assertion for an arbitrary compact linear group G:
a GC-orbit is closed if and only if it contains a polynomially convex G-orbit ([7,
Theorem 1 and Theorem 5]). For a torus T , all T -orbits in TCv are simultaneously
polynomially convex or non-convex, but this is not true if G is not abelian.
3. Finite extensions of T that keep a TC-orbit
In this section, we consider the case where the set X defined in the previous
section is an orbit of a finite group F which normalizes T and keeps the TC-orbit.
We assume that T ⊆ G, T is a torus, G is a subgroup of GL(V ), F is a finite
subgroup of G, and
G = FT = TF, F ∼= G/T,(3.1)
Gv ⊆ TCv,(3.2)
v ∈ (C∗)n ⊂ Cn = V.(3.3)
By (3.1), T is normal in G. Clearly, (3.2) is equivalent to Fv ⊆ TCv and to the
connectedness of GC. Here is an illustrating example.
Example 3.1. Let G = Aut0(D
2) be the group of linear automorphisms of the
bidisc D2 ⊂ C2. Clearly, G = FT , where F = S2 is generated by the transposition
τ of the coordinates, T = T2, TC = (C∗)2, and TCv = (C∗)2 for any v that lies
outside the coordinate lines. Thus, (3.2) holds for all v ∈ (C∗)2 (however, (3.2) fails
for any v 6= 0 in C2 \ (C∗)2). The hull Ĝv can be distinguished by the inequalities
max{|z1|, |z2|} ≤ max{|v1|, |v2|},(3.4)
|z1z2| ≤ |v1v2|.(3.5)
Clearly, (3.4) and (3.5) define a polynomially convex set. Let z1, z2 > 0 (a generic
T -orbit evidently contains such a point z). Then, z and τz can be joined by an
analytic strip with the boundary in Tz ∪ Tτz:
λz(s) = (z
2 ), s ∈ S.
Set q = ln z1
and let z1 > z2. Then, the strip can be written in the form
λz(s) = (e
−sz1, e
sz2), 0 ≤ Re s ≤ q.
It is periodic with the period 2πi and defines a τ -invariant annulus in Ĝv with τ -
fixed points (
z1z2,
z1z2) and (−
z1z2,−
z1z2). As z2 → 0, the annulus tends
to a couple of discs: (e−sz1, 0) and (0, e
−sz1), where Re s > 0, 0 ≤ Im s ≤ 2π (the
circle Re s = q
, 0 ≤ Im s ≤ 2π collapses to zero). Let z ∈ Ĝv ∩ R2. Then Ĝv
contains a bidisc D2z. It intersects R2 by a rectangle, which is symmetric with
respect to the coordinate axes. If z lies on an axis, then the rectangle degenerates
into a segment. Let v1 > v2 > 0. The union of these rectangles with vertices
in the set Q of real points of the annulus, which joins v and τv, is a curvilinear
octagon. It degenerates into a pair of segments if v2 = 0 and into a square if v1 = v2
(see [10, Fig. 2] for the 3-dimensional case). In the logarithmic coordinates in the
ORBITS OF TORI EXTENDED BY FINITE GROUPS AND THEIR POLYNOMIAL HULLS: THE CASE OF CONNECTED COMPLEX ORBITS9
first quadrant, Q is a segment. Also, note that all nontrivial TC-orbits are not
closed. �
In [2], Björk found a typical situation where analytic annuli appear in the maxi-
mal ideal space MA of a commutative Banach algebra A which admits a nontrivial
action of T by automorphisms: this happens if T -invariant functions on MA do
not separate distinct T-orbits. In [7], it was noted that analytic strips and/or an-
nuli appear in Ĝv if the stable subgroup of v in GC does not coincide with the
complexification of the stable subgroup of v in G.
Proposition 3.2. The hulls Ĝv for orbits of G = Aut0(D
n) = SnT
n are distin-
guished by inequalities (0.5), where µk are defined by (0.6).
Proof. The approximation by decreasing sequences of hulls makes it possible to
reduce the proposition to the case of a generic v in (0.5). Then, applying to v =
(v1, . . . , vn) a suitable transformation in T
n, we may assume that
v1 > v2 > · · · > vn > 0.(3.6)
Moreover, we may use Proposition 2.2 with X = Snv, CT = − closRn+ (we keep
the notation of Proposition 2.2). Since X , QX , CT , PX , and µk are Sn-invariant,
Sn is transitive on X , by (0.6), (0.5), and (2.10), it is sufficient to prove that the
vectors ξk =
r=1 ǫr, k = 1, . . . , n, correspond to the faces of PX that meet at v,
are orthogonal to them, and look outside of PX .
Set η1 = ǫ2 − ǫ1, . . . , ηn−1 = ǫn − ǫn−1, ηn = −ǫn. Then {−ηk}nk=1 is a base in
Rn, which is dual to the base {ξk}nk=1. We claim that the cone of the polyhedron
PX at the vertex v is generated by {ηk}nk=1. This implies the assertion above
(note that both cones are simplicial). If τ ∈ Sn is a transposition (k, j), then
v − τv = (vk − vj)(ǫk − ǫj). If σ, κ ∈ Sn then v − σκv = (v − κv) + (κv − σκv).
Furthermore, Sn is generated by transpositions (k, k + 1), and vk − vk+1 > 0 by
(3.6), where k = 1, . . . , n− 1. Therefore, vectors η1, . . . , ηn−1 generate the cone of
QX at v. Since −ǫk =
r=0 ηn−r and CT is generated by −ǫk, k = 1, . . . , n, this
proves the proposition. �
Property (3.2) implies spanC Gv = spanC T
Cv. Hence, we may assume that (1.2)
is valid. Then, a generic ξ ∈ t has a simple spectrum. Any f ∈ F permutes
eigenvalues and eigenspaces. Thus, assuming (1.2) and identifying V with C(K)
in accordance with (1.3), we get that each element of F is a composition of a
permutation of K and a multiplication by a function on K. Further, (3.3) implies
that the stable subgroup of v in TC is trivial. Hence,
TRv ∼= TC/T ∼= tR,
where the identification of TRv and tR is realized by ξ → exp(ξ)v, ξ ∈ tR.
Lemma 3.3. Let G ⊂ GL(V ), a subgroup F ⊆ G, and v ∈ V satisfy (3.1)–
(3.3). Then TCv contains a G-invariant T -orbit. Moreover, there exists a mapping
f → tf , F → T , such that F̃ = {tff : f ∈ F} is a subgroup of G which has a fixed
point in TC and satisfies (3.1)–(3.3).
Proof. The group F naturally acts on TRv ∼= TC/T ∼= tR. Any g ∈ F is a composi-
tion of σ ∈ SK and a multiplication by a function in C(K). Since t acts on C(K)
by multiplication on linear functions and σ induces a linear transformation in tC,
the induced action of F on tR is affine. Since F is finite, it has a fixed point in tR.
10 V.M. GICHEV
Hence, TCv contains a G-invariant T -orbit. Let us fix a point u in it and define
tf by tffu = u; the choice is unique due to (3.3). Taken together with (3.1), this
implies that F̃ is a group, which obviously satisfies the lemma. �
According to Lemma 3.3, we may assume without loss of generality that
fv = v for all f ∈ F.(3.7)
In the following example we give a construction (associated with a given finite group
F ) for orbits with property (3.2).
Example 3.4. Let t be a real linear space, t∗ be the dual space to t, L be a lattice
in t, and L∗ ⊂ t∗ be the dual lattice to L. Set
λx(y) = y(x), where x ∈ t, y ∈ t∗.
Let K be a finite subset of L∗ that generates L∗ as a subgroup of the vector group
t∗. Then
t∗ = spanR K,(3.8)
L = {x ∈ t : λx(K) ⊂ Z}.(3.9)
Further, let F be a finite subgroup of GL(t) which keeps K. Set V = C(K). The
mapping
λ : x → iλx
(3.10)
is an embedding t → V , which has a natural extension to tC. Set
exp(x) = e2πiλx .(3.11)
Clearly, L = ker exp. Hence, exp defines an embedding of T = t/L and TC into the
group (C∗)n:
TC = exp(tC) ⊆ (C∗)n.
The group TC acts on C(K) by multiplication. The inclusion v ∈ C(K)F is the
same as (3.7); it implies (3.2). Furthermore, if v ∈ (C∗)n, then
spanC Tv = V.(3.12)
Indeed, the space spanC T is a subalgebra of C(K), which separates points of the
finite set K. Hence, it coincides with C(K). �
Theorem 3.5. Let a group G ⊂ GL(V ), a finite subgroup F ⊆ G, a torus T , and
a vector v ∈ V satisfy (3.1)–(3.3), (3.7), and (3.12). Then V,G, F, T, v can be
realized as in Example 3.4, where
v ∈ (C∗)n ∩ C(K)F .(3.13)
Conversely, if V,G, F, T, v are as in Example 3.4 and v satisfies (3.13), then (3.1)–
(3.3), (3.7), and (3.12) are true.
Proof. The group F acts in t and t∗ by the adjoint action. Let K ⊂ t∗ be the
collection of all weights for the representation of T in V ; clearly, K is F -invariant.
It follows from (3.12) and (3.2) that the weights are multiplicity free. This defines
an equivariant linear isomorphism between V and C(K), where the group T acts by
multiplication. Thus, λ and exp are well defined by (3.10) and (3.11). According
to (3.7) and (3.3), (3.13) is true; (3.8) holds since T ⊂ GL(V ) is compact and acts
effectively on V (note that the annihilator of spanR K in t acts trivially due to
ORBITS OF TORI EXTENDED BY FINITE GROUPS AND THEIR POLYNOMIAL HULLS: THE CASE OF CONNECTED COMPLEX ORBITS11
(3.11) and (3.13)). Let us define L by (3.9). Then L = ker exp by (3.11). Hence, L
is a lattice in t and the group L∗ generated by K is the dual lattice in t∗.
The converse was proved in Example 3.4. �
4. Finite extensions of T which keep generic TC-orbits
In what follows, we use the setting of Example 3.4. Let Z denote the centralizer
of G in GL(V ). We assume that (C∗)n acts in V = C(K) by multiplication.
Lemma 4.1. Z = C(K)F ∩ (C∗)n.
Proof. Since λ(t) separates points of K, Z ⊆ (C∗)n. The multiplication by u ∈
C(K) commutes with F if and only if u is F -invariant. �
In general, condition (3.2) does not hold for a generic vector v. Hence, there is
a natural problem: describe V and G such that generic orbits satisfy (3.2). The
following proposition contains a simple criterion.
Proposition 4.2. Let V,G be as in Example 3.4. Then G satisfies (3.2) for a
generic v ∈ V if and only if
C(K) = λ(tC) + C(K)F .(4.1)
In this case, each TC-orbit in (C∗)n intersects C(K)F .
Proof. It follows from Lemma 4.1 that the right-hand side of (4.1) is the tangent
space at 1 to the set TC Z. Clearly, G̃C = ZGC is a group, TCZ is the identity
component of G̃C, and the right-hand side of (4.1) is the tangent space to G̃C1.
Hence, (4.1) holds if and only if G̃C1 is open. Moreover, this is equivalent to the
equality TCZ = exp(λ(tC) + C(K)F ) = (C∗)n. Therefore, each TC-orbit in (C∗)n
intersects C(K)F , i.e., contains an F -fixed point. Thus, (4.1) implies (3.2) for
v ∈ (C∗)n.
Let (3.2) hold and letW be an F -invariant neighborhood of 1. IfW is sufficiently
small, then the condition log 1 = 0 defines a branch of log in W . We may assume
that logW is convex and symmetric. This makes it possible to define roots in W :
r = exp
. For v ∈ W 12 and f ∈ F , set gf =
, where r = cardF ,
and g =
f∈F gf . Then gv is F -fixed. If (3.2) holds for v, then gf ∈ TC for all
f ∈ F ; hence, gv ∈ TCv. Consequently, for all v ∈ W , TCv intersects C(K)F . Since
Z keeps this property of orbits, it follows that TCZ has a nonempty interior. This
implies (4.1). �
Theorem 4.3. Let G ⊂ GL(V ) be a semidirect product of a torus T and a finite
subgroup F , and let Z be the centralizer of G in GL(V ). Suppose that spanC Tv = V
for some v ∈ V . Then the following conditions are equivalent:
(i) Gv ⊂ TCv for a generic v ∈ V ;
(ii) GCZv is open in V for a generic v ∈ V .
Proof. By Theorem 3.5, we may use the construction of Example 3.4. According to
Lemma 4.1, (ii) is equivalent to (4.1), and the assertion follows from Proposition 4.2.
12 V.M. GICHEV
We shall give a constructive description of these spaces and groups. Set
C0(K) =
u ∈ C(K) :
u(q) = 0
Sometimes, we identify points in K with their characteristic functions.
Example 4.4. Let V = Cn = C(K), where K = {1, . . . , n}, let F be a subgroup
of Sn, and
K = K1 ∪ · · · ∪Kp(4.2)
be the partition of K into F -orbits. For k ∈ {1, . . . , p}, set Vk = C(Kk). Then
V = V1 ⊕ · · · ⊕ Vp. Set
t0k = C0(Kk) ∩ iRn,
= exp(t0
) ⊂ C(Kk),
where exp is defined by (3.11). Set t0 = t01 ⊕ · · · ⊕ t0p,
T 0 = exp(t0) = T 01 × · · · × T 0p .
Let T be an F -invariant torus such that
T 0 ⊆ T ⊆ Tn(4.3)
and set G = FT . Then generic GC-orbits satisfy (3.2). The group G is irreducible
if and only if F is transitive on K; in general, F -orbits in K define G-irreducible
components of V . There are two extreme cases in (4.3).
(A) If T = Tn, then there is one open orbit (C∗)n of the groupGC = FTC, which
evidently satisfies (3.2). If F is nontrivial, then there exist degenerate orbits
that do not satisfy (3.2); moreover, if F is transitive onK, then all non-open
GC-orbits, except for zero, are nontrivial finite unions of TC-orbits.
(B) If T = T 0, then generic orbits are closed. They have codimension p and
are distinguished by equations
zr = ck,
where ck ∈ C∗, k = 1, . . . , p.
Note that (A) and (B) are invariant under the Cartesian product (the group F
need not be the product of groups Fk of irreducible components but must have the
same orbits in K as F1 × · · · × Fp). In terms of Example 3.4: in (A), t = Rn, the
mapping λ : tC → C(K) is surjective, K = {ǫ1, . . . , ǫn}; in (B), t = iRn ∩ C0(K),
λ(tC) = C0(K), and the set K is the projection of {ǫ1, . . . , ǫn} into t∗ = t. In both
cases, K is the set of all vertices of a regular simplex. �
Theorem 4.5. Let V,G be as in Theorem 4.3 and let (i) hold. Then V,G can be
realized as in Example 4.4. Furthermore,
(1) V,G are of type (A) if and only if GC has an open orbit,
(2) (B) is equivalent to the assumption that the center of G is finite,
(3) if G is irreducible, then either (A) or (B) holds.
Let C(K)F+ be the cone of all nonnegative functions in C(K)
Lemma 4.6. Let G and V be as in Example 3.4. Then, the orbit GCv is closed
for a generic v ∈ V if and only if
R ∩ C(K)F+ = 0.(4.4)
ORBITS OF TORI EXTENDED BY FINITE GROUPS AND THEIR POLYNOMIAL HULLS: THE CASE OF CONNECTED COMPLEX ORBITS13
Proof. Clearly, GCv is closed if and only if TCv is closed. Let v ∈ (C∗)n. By
Proposition 2.2 and Corollary 2.3, TCv is not closed if and only if CT 6= 0. Since
CT is F -invariant by (1.4), it contains
f∈F fu for each u ∈ CT . Thus, CT = 0 is
equivalent to (4.4). �
Proof of Theorem 4.5. Suppose that G is irreducible or, equivalently, F is transi-
tive. Then Z = C∗ 1 according to Lemma 4.1. If 1 ∈ λ(tC), then TC ⊇ Z and
TCv is open for a generic v ∈ V by Theorem 4.3. If 1 /∈ λ(tC), then (4.4) is true;
by Lemma 4.6, TCv is closed for a generic v ∈ V . By Proposition 4.2, a generic
TC-orbit intersects C∗1. Consequently, we have
codimGCv = 1.(4.5)
Let 1 ∈ TC ∩ Z. The orthogonal projection of 1 into the tangent space T1TC1 is
F -fixed. Hence, it is proportional to 1; since 1 /∈ λ(tC), this implies 1 ⊥ λ(tC)1.
Therefore, T1T
C1 coincides with the tangent space to the hypersurface z1 . . . zn = 1
at 1; since the monomial on the left is an eigenfunction of TC, this group keeps it.
Due to (4.5), TC1 coincides with this hypersurface. Then, T = Tn ∩ SU(n), and
any TC-orbit that intersects Z is a hypersurface z1 . . . zn = c, for some c ∈ C∗. This
implies tC = C0(K) and T = T
Thus, the theorem is proved for all irreducible G. The projection onto each
irreducible component keeps the property (3.2) for generic orbits since it commutes
with G. Hence, (i) holds for all irreducible components. They correspond to F -
orbits Kk in the partition (4.2). Let t
, k = 1, . . . , p , be defined as in Example 4.4.
According to the arguments above, λ (t|Kk) ⊇ λ(t0k) for all k. If x ∈ t, then the
averaging
Ax = 1
f∈F fx, r = cardF,
distinguishes the F -fixed component of x (i.e., Ax ∈ C(K)F ∩ t and x− Ax ∈ t0);
since t is F -invariant, it contains both components. By Lemma 4.1, if G has a finite
center, then λ (t|Kk) = λ(t0k) for all k. It follows that
t ⊆ t0 = t01 ⊕ · · · ⊕ t0p.
On the other hand, (ii) and Lemma 4.1 imply codim t ≤ dimC(K)F = p. Hence,
the inclusion above is in fact the equality. Thus, we get (B) assuming that G has
a finite center. The converse is true since t0 does not contain a nontrivial F -fixed
element. The same arguments show that any F -invariant torus T includes T 0 if (i)
is true. This proves that V,G admit the realization of Example 4.4; (1) and (2) are
clear. �
Corollary 4.7. Let G be as in Theorems 4.5 and 4.3. Then G contains a closed
subgroup G0 such that
(1) each connected component of G contains a connected component of G0,
(2) G0 has a finite center,
(3) generic orbits of (G0)C are closed,
(4) Gv ∩ TRv = G0v ∩ (T 0)Rv for a generic v ∈ V .
Proof. By Theorem 4.5 and (4.3), G ⊇ T 0, where T 0 is as in (B). Clearly, F
normalizes T 0. Hence, G0 = FT 0 is a group, which satisfies the corollary. �
14 V.M. GICHEV
Proposition 2.2 makes it possible to find Ĝv for G as above. If T = Tn, then
T ⊃ Zn2 and generic T -orbits intersect Rn+; hence, we may assume v ∈ Rn+. Then
T̂ v ∩ Rn is a parallelepiped Πv = conv{(±v1, . . . ,±vn)}. Clearly, Πv = Zn2Π+v ,
where Π+v = Πv ∩ closRn+. Since Rn+ = TRv, we may use Proposition 2.2 with
X = Fv, CT = − closRn+, PX = conv(Fv) − Rn+:
Ĝv = ∪u∈exp(Qv)Dnu = T ∪u∈exp(Qv) Πu = T ∪u∈exp(Qv) Π+u ,
where Qv = convFv. For the description in the form (2.10), one has to know
normal vectors to faces of convFv. Since F may be an arbitrary subgroup of
Sn, they need not be proportional to rational vectors (for example, this is true
for the cyclic subgroup of order 3 in S3). We shall describe the situation where
they are locally independent of v; since they depend on v continuously, this is
equivalent to the condition that they are rational. Note that the vector which joins
two points in tR as in Lemma 2.1 is rational if and only if the strip reduces to an
annulus. In Example 4.4, F need not be the product of groups corresponding to
the irreducible components; we shall see that F possesses this property in the case
under consideration.
Let U be a real vector space and F ⊂ GL(U) be a finite group. Set
Cu = cone(u− Fu);
this is the cone at the vertex u of the polytope conv(Fu) (which may be degenerate).
We say that Cu is locally independent of u if, for a generic u ∈ U , Cu = Cw for all
w that are sufficiently close to u.
Lemma 4.8. Let U be a real vector space and F be a finite subgroup of GL(U).
Suppose that Cu is locally independent of u. Then F is generated by reflections in
hyperplanes in U .
Proof. We may assume without loss of generality that U is equipped with an inner
product and that F ⊆ O(U). Let R+(u − fu), f ∈ F , be an extreme ray of Cu.
The equality Cu = Cw for w in a neighborhood of u implies that this ray does not
change near u. Hence, dim(1 − f)U = 1. Since f is orthogonal and nontrivial, it
is a refection in a hyperplane. The stable subgroup of a generic u ∈ U is trivial
(hence, F acts freely on a generic orbit) and each vertex of conv(Fu) can be joined
with u by a chain of edges. Applying the above arguments repeatedly to u, fu, etc.,
we get that F is generated by reflections in hyperplanes. �
For any g ∈ Zn2Sn and k = 1, . . . , n, gǫk = ± ǫσ(k) for some σ ∈ Sn. The mapping
f → σ is a natural homomorphism Zn2Sn → Sn, which we denote by φ.
Lemma 4.9. Let F be a transitive subgroup of Sn acting in R
n by permutations
of coordinates and let a group H ⊆ Zn2Sn be generated by reflections in hyperplanes
in Rn. If φ(H) = F , then F = Sn.
Proof. Let ρ be a reflection in a hyperplane in Rn. If ρ ∈ Zn2Sn = BCn, then it
is conjugate to a reflection in a wall of the Weyl chamber that is distinguished by
the inequalities x1 > · · · > xn > 0. Hence, φ(ρ) is a transposition if it is nontrivial.
Since F = φ(H), F is generated by transpositions. It remains to note that any
subgroup of Sn, which is generated by transpositions, coincides with Sn if it is
transitive on {1, . . . , n} (consider the graph with the vertices {1, . . . , n} and edges
corresponding to transpositions and note that inclusions (k, l) ∈ F , (l,m) ∈ F
imply (k,m) ∈ F ; this makes it possible to use the induction). �
ORBITS OF TORI EXTENDED BY FINITE GROUPS AND THEIR POLYNOMIAL HULLS: THE CASE OF CONNECTED COMPLEX ORBITS15
We say that a pair (V,G) is standard if it is isomorphic to (A) or (B) in Ex-
ample 4.4 with F = SK . The product of pairs (Vk, Gk), k = 1, . . . ,m, is the pair
k=1 Vk,
k=1 Gk).
Theorem 4.10. Let G = FT be a compact subgroup of GL(n,C), where T ⊆ Tn
is a torus and F is a subgroup of Sn. Suppose that Gv ⊂ TCv for a generic v ∈ V
(1) either T = Tn or the center of G is finite,
(2) for a generic v ∈ Cn, Ĝv can be distinguished in closTCv by a family of
inequalities
|z1|s1 . . . |zn|sn ≤ ρs(v),
where ρs(v) ≥ 0 and vector s = (s1, . . . , sn) runs over a certain finite subset
of Rn which is independent of v.
Then (V,G) is isomorphic to the product of standard pairs. Moreover, if G is
irreducible, then (V,G) is standard.
Proof. Let G be irreducible. Then F is transitive and (V,G) are as in (A) or as
in (B) by Theorem 4.5. Suppose that (B) is the case. It follows from (2) and
Proposition 2.2 that the polytope QX ⊂ tR, where X = Gv ∩ TRv, for a generic
v, satisfies the assumption of Lemma 4.8. Therefore, F is generated by reflections
(we may assume that F ⊂ O(tR)). They extend to reflections in hyperplanes in
tR+R1 = Rn if we assume that they fix 1. Then, Lemma 4.9 implies F = Sn. The
case (A) can be reduced to (B): it is sufficient to replace Tn with T = SU(n) ∩ Tn
since F evidently keeps T and to note that (2) remains true due to Proposition 2.2.
Thus, (V,G) is standard.
Let the center of G be finite. According to Theorem 4.5, T may be identified
with the group T 0 in Example 4.4. In particular, GCv is closed for a generic v
and CT = 0 due to Proposition 2.2. By Proposition 4.2, generic orbits contain
F -fixed points. Applying the arguments above (which did not use the assumption
that G is irreducible), we get that the cones at the vertices of the convex polytope
QX , X = Gv ∩ TR ⊂ tR, are locally independent of v. Clearly, the same is true
for its projection into each space t0k corresponding to an irreducible component
Vk of V = C
n. This implies that all irreducible components are standard. Thus,
Fk = S(Kk), where k = 1, . . . , p and K = K1 ∪ · · · ∪Kp is the partition of K into
F -orbits. Due to Theorem 4.5, it is sufficient to prove that
F = F1 × · · · × Fp.(4.6)
By Lemma 4.8, F |t0 is generated by reflections in hyperplanes in t0; the condition
that they keep real F -invariant functions on K uniquely defines their extension to
Rn. Hence, F is generated by reflections in Rn. A permutation which induces a
reflection in a hyperplane in Rn is a transposition of a pair of coordinates; this pair
is necessarily contained in only one of the sets Kk, k = 1, . . . , p. This proves (4.6).
If T = Tn, then T is a product of tori in irreducible components. Thus, the case
T = Tn follows from the above case, since the assumptions of the theorem hold
true for the group T 0 if they hold for T in (4.3) in Example 4.4. �
5. Hulls of isotropy orbits of bounded symmetric domains
We start with a preliminary material on hermitian symmetric spaces following
[15] but adapting the exposition to our purpose in order to be as self contained as
16 V.M. GICHEV
possible. For a subset X of a Lie algebra g, z(X) = {z ∈ g : [z,X ] = 0} is the
centralizer of X . Let G be a simple real noncompact Lie group with a finite center,
K be its maximal compact subgroup, and g, k be their Lie algebras, respectively.
If the center z = z(k) of k is nontrivial, then g is called hermitian. Then k = z(z)
and dim z = 1 (note that K is irreducible in g/k). Let c be a Cartan subalgebra
of k. Then c is also a Cartan subalgebra of g and z ⊆ c. There exists k ∈ z such
that ad(k) has eigenvalues 0,±i (it is unique up to a sign; ker ad(k) = k). Then
κ = eπ ad(k) is the Cartan involution which defines the Cartan decomposition
g = k⊕ d,(5.1)
where k, d are eigenspaces for 1,−1, respectively. Furthermore, j = ad(k) is a
complex structure in d. This defines the structure of a hermitian symmetric space of
noncompact type in D = G/K. These spaces can be realized as bounded symmetric
domains in Cn with K = Aut0(D). Any irreducible bounded symmetric domain
admits such a realization. Let ∆ ⊆ ic∗ be the root system of gC. Each α ∈ ∆
corresponds to an sl2-triple hα, eα, fα such that ihα ∈ c. Thus, α(hα) = 2, [eα, fα] =
hα, and
[h, eα] = α(h)eα, [h, fα] = −α(h)fα(5.2)
for all h ∈ cC. We identify cC and (c∗)C equipping g with an Ad(K)-invariant
sesquilinear inner product and normalize it by the condition
max{|α| : α ∈ ∆} =
2.(5.3)
Then short roots must have length 1 (note that G2 has no real hermitian form).
The set ∆∨ = {hα : α ∈ ∆} is the dual root system. The above normalization
implies hα = α for long roots and hα = 2α for short ones. Since ad(h), h ∈ c, has
eigenvalues 0 and α(h), where α ∈ ∆, we get α(ik) = 0,±1, i.e., ik is a microweight
(of ∆∨). For s = 0,±1, set
∆s = {α ∈ ∆ : α(ik) = s}.(5.4)
Since k⊕ id is a compact real form of gC and spanR{ihα, eα − fα, i(eα + fα)} is the
su(2)-subalgebra corresponding to a root α ∈ ∆, we have
d = spanR{eα + fα, i(eα − fα) : α ∈ ∆1}.(5.5)
Set sα = spanR{ihα, eα + fα, i(eα − fα)}. Then sα is an sl(2,R)-subalgebra of gC
α ∈ ∆±1 ⇐⇒ sα ⊆ g.(5.6)
Let E be a maximal subset of pairwise orthogonal long roots in ∆1. Set
α∈E hα, e =
α∈E eα, f =
α∈E fα;
α∈E ⊕ sα.
Let α, β ∈ E, α 6= β. Since α, β are long and orthogonal, ±α± β /∈ ∆. Hence,
α, β ∈ E, α 6= β =⇒ [sα, sβ] = 0.(5.7)
It follows that h, e, f is an sl2-triple and s is a subalgebra of g. Set
θ = e
π ad(e−f),(5.8)
a = spanR θE.(5.9)
ORBITS OF TORI EXTENDED BY FINITE GROUPS AND THEIR POLYNOMIAL HULLS: THE CASE OF CONNECTED COMPLEX ORBITS17
Here is the standard realization of root systems Bn and Cn:
Bn = {±ǫk ± ǫl, ±ǫm : k, l,m = 1, . . . , n, k < l};
Cn = {±ǫk ± ǫl, ±2ǫm : k, l,m = 1, . . . , n, k < l}.
Then Cn = B
n , but Cn does not satisfy (5.3). These systems have microweights;
up to the action of the Weyl group, they are:
Bn : ǫ1;
(ǫ1 + · · ·+ ǫn).
There are no other irreducible root systems which have microweights and contain
roots of different lengths. Also, Bn and Cn have the sameWeyl group BCn = Z
Lemma 5.1. The space a is a maximal abelian subspace of d.
Proof. A straightforward calculation with 2-matrices shows that θh = e + f. By
(5.7),
θhα = eα + fα for all α ∈ E.(5.10)
It follows from (5.5) that a ⊆ d. Moreover, a is abelian due to (5.7). Set Ξ = ∆∩E⊥.
We claim that
Ξ ⊆ ∆0.(5.11)
Indeed, a root in ∆1 ∩ Ξ must be short. This may happen only in Bn or Cn, since
G2 and F4 have no microweights and other irreducible root systems have no roots
of different lengths. In Bn, k is a short root and all other short roots are orthogonal
to k. Hence, they do not belong to ∆1. In Cn, E = {2ǫ1, . . . 2ǫn}; then Ξ = ∅.
Since ∆−1 = −∆1, this proves (5.11).
Set b = E⊥ ∩ c and m = spanC{eα, fα : α ∈ Ξ}. It follows from (5.11) that
m ⊆ kC. Clearly, z(E) = cC ⊕ m. The space m is θ-invariant, because θ fixes roots
in Ξ. Due to (5.9), we get
z(a) = θz(E) = bC ⊕ aC ⊕m
Since bC ⊕m ⊆ kC, this implies z(a) ∩ d = a. �
The projection of θ∆ into a is the restricted root system ∆a (it is also the set of
roots for ad(a) in g). The group
W = {Ad(g) : g ∈ K, Ad(g)a = a}|a,
acting in a, is the Weyl group of a.
In what follows, we denote by v the complexification of a with respect to the
complex structure j (thus, v ⊂ d). The set θE is a base in v; enumerating it, we
identify v with Cn. Set t = spanR iE, T = exp t, H = WT . The torus T = T
n is a
maximal compact subgroup in the group exp s ⊆ G.
Proposition 5.2. The following assertions hold:
(1) ∆a is a root system of type BCn or Cn;
(2) the pair (v, H) is standard with T = Tn.
Proof. (1). Let ∆a \ θE contain a long root α. Then α = 12 (α1 + α2 + α3 +
α4) for some α1, . . . , α4 ∈ θE, since |α|2 = 2 and 〈α, β〉 = 0,±1 for all β ∈ E
due to the normalization (5.3) (note that α, β generate A2 if 〈α, β〉 6= 0). Roots
α, α1, . . . , α4 generate D4, since only A4 and D4 among irreducible systems of rank
18 V.M. GICHEV
4 consist of roots of equal length, but A4 does not contain an orthogonal base. Since
〈ik, β〉 = 1 for all β ∈ E, the projection of iθk into spanR D4 is a microweight ω
such that 〈ω, αk〉 = 1, k = 1, . . . , 4, but D4 has no microweight with this property
(in the realization above, D4 = B4 ∩ C4 and the microweights are either ±ǫk or
(±ǫ1± ǫ2± ǫ3± ǫ4)). Thus, E ∪ (−E) is the set of all long roots in ∆a. According
to the classification of irreducible root systems, only Cn and BCn = Bn ∪ Cn has
the property that linearly independent long roots are mutually orthogonal.
(2). The maximal compact subgroup of the group corresponding to s is Tn.
Hence, T = Tn ⊃ Zn2 . Systems Cn and BCn have the same Weyl group W = BCn.
Therefore, H = WT = SnT
Let D be a bounded symmetric domain in a complex linear space d (may be,
reducible) and v ⊆ d be the complex linear span of a maximal abelian subspace in
d (thus, we identify d with the corresponding space in the Cartan decomposition
(5.1), which is induced by the Cartan involutions in irreducible components). Let
Aut00(v, D) denote the subgroup of all linear transformations in Aut(D) which keep
v and each irreducible component of D.
Corollary 5.3. Let F be a subgroup of Sn, G = FT
n ⊂ GL(n,C). Then G
satisfies condition (2) of Theorem 4.10 if and only if (V,G) is isomorphic to a pair
(v,Aut00(v, D)) for a bounded symmetric domain D.
Proof. All pairs (Cn, SnT
n) appear as (v,Aut00(v, D)) for matrix balls D. It re-
mains to combine Theorem 4.10 and Proposition 5.2. �
It is possible now to describe hulls of K-orbits in d (with respect to the complex
structure j) it terms of Proposition 3.2. The key point is that K is polar in d: each
K-orbit meets a orthogonally (i.e., a is a Cartan subspace). This is true, since all
maximal abelian subspaces are conjugate in d by K, ad(a) is symmetric if a ∈ d
and, for a generic a ∈ a, ker ad(a) = a; hence,
[a, g] = a⊥.(5.12)
We may include the linear base in v into a base in d as the first n vectors of the
latter. Then z1, . . . , zn are coordinates in v and linear functions in d. The functions
µk in (0.6) admit a K-invariant extension to d:
µk(z) = sup{|(gz)1 . . . (gz)k| : g ∈ K},(5.13)
where k = 1, . . . , n. The following lemma shows that (5.13) is an extension indeed.
Lemma 5.4. For z ∈ v, (0.6) and (5.13) coincide.
Proof. It follows from (5.12) that any critical point of the linear function Re z1 on
the orbit Kz belongs to a. If the lemma is not true, then there exist z ∈ v and
k ∈ {1, . . . , n} such that |(gz)k| > |zk|. Transformations in Sn and T reduce the
problem to the case z1 > · · · > zn > 0 and k = 1, but then the assumption implies
that Re z1 attains its maximal value on Kz outside of a. �
Proposition 5.5. For any v ∈ d, K̂v = {z ∈ d : µk(z) ≤ µk(v), k = 1, . . . , n}.
Proof. Due to (5.13), each µk is a supremum of absolute values of holomorphic
polynomials. Hence, the right-hand side is polynomially convex. Thus, it includes
K̂v. The inverse inclusion holds, since each K-orbit intersects v by an H-orbit
and hulls of H-orbits are distinguished in v by the same inequalities according to
Proposition 3.2 and Lemma 5.4. �
ORBITS OF TORI EXTENDED BY FINITE GROUPS AND THEIR POLYNOMIAL HULLS: THE CASE OF CONNECTED COMPLEX ORBITS19
The functions µk can be written in more invariant terms. To do it, note that the
Weyl group of ∆a has the form Z
2Sn in the base θE by (5.9); thus, zk = αk(z),
k = 1, . . . , n, where αk ∈ θE and z ∈ a. Therefore, zk are eigenvalues of ad(z)
in the subspace generated by the corresponding root vectors. The problem is to
distinguish this subspace (in fact, we use a slightly different version). After that,
functions µk can be defined as norms of some operators according to the following
lemma (this observation was used in [12] in another context).
Lemma 5.6. Let V be a Euclidean space and A be a symmetric nonnegative oper-
ator in V with eigenvalues λ1 ≥ λ2 ≥ · · · ≥ λm ≥ 0, where m = dim V . Let A∧k be
its natural extension to the k-th exterior power V ∧k =
V . Then
‖A∧k‖V ∧k = λ1 . . . λk,
where ‖ ‖k is the operator norm with respect to the inner product in V ∧k.
Proof. The norm of a nonnegative symmetric operator is equal to its maximal
eigenvalue. �
Let v ∈ g be semisimple and π(v) denote the projection onto ker ad(v) along
other eigenspaces of ad(v) (note that π(v) is a function of ad(v), since it is the
residue at zero of the resolvent of ad(v)). Set
a(v) = ad([v, [v, k]])π(v) ad(k),
pk(v) = ‖a(v)∧k‖g∧k , k = 1, . . . , n.
The space d is a(v)-invariant and ker a(v) ⊇ k. We assume that g is equipped
with some K-invariant inner product, which extends the inner product in d. It
follows from the calculation below that a(v) is symmetric and has range ad(k)a.
Let n = dim a be the rank of the symmetric space D. It is equal to the codimension
of a generic K-orbit in d.
Theorem 5.7. For any v ∈ d,
K̂v = {z ∈ d : pk(z) ≤ pk(v), k = 1, . . . , n}.
Proof. It is sufficient to prove the assertion for a generic v ∈ d. Clearly, pk are
K-invariant. Hence, we may assume v ∈ a. Then, by (5.9) and (5.10),
vα(eα + fα),
where vα ∈ R. According to (5.2) and (5.4), [k, v] =
α∈E ivα(eα − fα). Thus,
[v, [v, k]] =
2iv2αhα
due to (5.7). Also, (5.7) implies that ad(ihα) keeps v and has eigenvalues 0,±2i
in it for each α ∈ E. Therefore, v is ad([v, [v, k]])-invariant and its eigenvalues are
±4v2αi, α ∈ E. Since ad(k)g = d and π(v)d = a for a generic v ∈ a, the space v is
a(v)-invariant; moreover, a(v)g = ad(k)a ⊆ v. Thus, a(v) has eigenvalues 0,±4v2α
in g. According to Lemma 5.6 and (0.6),
pk(v) = 4
kµ2k(v)(5.14)
for v ∈ v and k = 1, . . . , n. Since pk and µk are K-invariant, (5.14) holds for all
v ∈ d. The theorem follows from Proposition 5.5. �
20 V.M. GICHEV
Corollary 5.8. Functions pk, k = 1, . . . , n, are plurisubharmonic in d with respect
to the complex structure j = ad(k).
Proof. By (5.14) and (5.13),
pk(z) = 4
k sup{|(gz)21 . . . (gz)2k| : g ∈ K}.
The right-hand side is plurisubharmonic, since the functions z2
are j-holomorphic
and j is K-invariant. �
One can get the same functions pk by replacing g with d, endowed with the
complex structure j, and a(v) with ad([v, jv])(π(v) + π(jv)).
Acknowledgement. I thank Anton Pankratiev for helpful comments.
References
[1] J.T. Anderson, Polnomial hulls of sets invariant under an action of the special unitary group,
Can. J. Math., v. XL, (1988) no. 5, 1256-1271.
[2] J.-E. Björk, Compact groups operating on Banach algebras, Math. Ann., v. 205 (1973), no. 4,
281–297.
[3] J. Dadok, Polar coordinates induced by actions of compact Lie groups, Transactions Amer.
Math. Soc., 288 (1985), 125-137.
[4] A. Debiard, B. Gaveau, Equations de CauchyRiemann sur SU(2) et leurs enveloppes dholo-
morphie, Can. J. Math., v. 38 (1986), 1009-1024.
[5] T. Gamelin, Uniform Algebras, Prentice-Hall, Englewood Cliffs, N. J., 1969.
[6] V.M. Gichev, Maximal ideal spaces of invariant function algebras on compact groups, preprint,
46 pp., available at http://arXiv.org/abs/math/0603449
[7] V.M. Gichev, I.A. Latypov, Polynomially Convex Orbits of Compact Lie Groups, Transfor-
mation Groups, v. 6 (2001) no. 4, 321-331.
[8] J. Faraut, L. Bouattour, Enveloppes polynomiales densembles compacts invariants, Math.
Nachrichten 266 (2004), 20-26.
[9] W. Kaup, Bounded symmetric domains and polynomial convexity, Manuscr. Math. 114 (2004),
No.3, 391-398.
[10] W. Kaup, D. Zaitsev, On the CR-structure of compact group orbits associated with bounded
symmetric domains, Invent. Math., 153 (2003), 45-104.
[11] J. Kane, Maximal ideal spaces of U-algebras, Illinois J. Math., v.27 (1983), No 1, 1-13.
[12] B. Kostant, On convexity, the Weyl group and the Iwasawa decimposition, Ann. Sci. Ecole.
Norm. Super., 6 (1973), 413–455.
[13] V.L. Popov, E.B. Vinberg, Invariant theory, Itogi Nauki i Tekhniki, Sovr. Probl. Mat. Fund.
Napravl., v. 55, VINITI, Moscow 1989, pp. 137–309 (in Russian); English transl.: Algebraic
Geometry IV, Encyclopaedia of Math. Sciences, v. 55, Springer-Verlag, Berlin 1994, pp. 123-
[14] C. Sacre, Enveloppes polynomiales de compacts, Bull. Sci. math. 116 (1992), 129-144.
[15] J.A. Wolf, Fine Structure of Hermitian Symmetric Spaces, Symmetric spaces (Short Courses,
Washington Univ., St. Louis, Mo., 1969-1970), Pure Appl. Math., Vol. 8. New York: Dekker
1972, pp. 271-357.
Omsk Branch of Sobolev Institute of Mathematics, Pevtsova 13, 644099, Omsk, Rus-
E-mail address: gichev@ofim.oscsbras.ru
http://arXiv.org/abs/math/0603449
	Introduction
	1. Preliminaries
	2. Hulls of finite unions of T-orbits in a TC-orbit
	3. Finite extensions of T that keep a TC-orbit
	4. Finite extensions of T which keep generic TC-orbits
	5. Hulls of isotropy orbits of bounded symmetric domains
	Acknowledgement
	References

  Let $V$ be a complex linear space, $G\subset\GL(V)$ be a compact group. We
consider the problem of description of polynomial hulls $\wh{Gv}$ for orbits
$Gv$, $v\in V$, assuming that the identity component of $G$ is a torus $T$. The
paper contains a universal construction for orbits which satisfy the inclusion
$Gv\subset T^\bbC v$ and a characterization of pairs $(G,V)$ such that it is
true for a generic $v\in V$. The hull of a finite union of $T$-orbits in
$T^\bbC v$ can be distinguished in $\clos T^\bbC v$ by a finite collection of
inequalities of the type $\abs{z_1}^{s_1}...\abs{z_n}^{s_n}\leq c$. In
particular, this is true for $Gv$. If powers in the monomials are independent
of $v$, $Gv\subset T^\bbC v$ for a generic $v$, and either the center of $G$ is
finite or $T^\bbC$ has an open orbit, then the space $V$ and the group $G$ are
products of standard ones; the latter means that $G=S_nT$, where $S_n$ is the
group of all permutations of coordinates and $T$ is either $\bbT^n$ or
$\SU(n)\cap\bbT^n$, where $\bbT^n$ is the torus of all diagonal matrices in
$\rU(n)$. The paper also contains a description of polynomial hulls for orbits
of isotropy groups of bounded symmetric domains. This result is already known,
but we formulate it in a different form and supply with a shorter proof.

Introduction
Let V be a finite-dimensional complex linear space and G ⊂ GL(V ) be a compact
subgroup of GL(V ). We consider the problem of description of polynomially convex
hulls for orbits Ov = Gv, v ∈ V . The polynomially convex hull (or polynomial hull)
Q̂ of a compact set Q ⊂ V is defined as
Q̂ = {z ∈ V : |p(z)| ≤ sup
|p(ζ)| for all p ∈ P(V )},(0.1)
where P(V ) is the algebra of all holomorphic polynomials on V . It is usually
difficult to find Q̂. For Q = Gv, the answer is known if G is an isotropy group of a
bounded symmetric domain in Cn. Paper [9] contains a description of G-invariant
polynomially convex compact sets, including hulls of orbits (Q ⊂ V is polynomially
convex if Q̂ = Q); it continues paper [10] and uses results of [8]. On the other
hand, it is known that an orbit of a compact linear group is polynomially convex
if and only if the complex orbit GCv is closed and Gv is its real form ([7]). The
cases G = U(2), SU(2) were considered in [1], [4]. The problem of determination
of polynomial hulls of orbits admits the following natural generalization: given a
homogeneous space M of a compact group G, describe maximal ideal spaces MA of
G-invariant closed subalgebras A of C(M), where C(M) is the Banach algebra of all
1991 Mathematics Subject Classification. Primary 32E20; Secondary 32M15, 32M05.
Key words and phrases. Polynomial hulls, bounded symmetric domains.
The author was partially supported by RFBR Grants 06-08-01403 and 06-07-89051.
http://arxiv.org/abs/0704.1095v1
2 V.M. GICHEV
continuous complex-valued functions on M with the sup-norm. If A is generated by
a finite-dimensional invariant subspace, then MA can be realized as the polynomial
hull of an orbit. Paper [6] contains a description of MA for bi-invariant algebras
on compact groups and partial results on spherical homogeneous spaces. Maximal
ideal spaces for U(n)-invariant algebras on spheres in C
n are described in [11].
In this paper we consider orbits Gv of groups G = FT , where F ⊆ G is a finite
subgroup and T is a torus, such that GCv = TCv. Let t ⊆ gl(V ) be the Lie algebra
of T and set tR = it, TR = exp(tR). Suppose that v ∈ V has a trivial stable
subgroup in T and let X ⊂ TRv be finite. The hull of Y = TX admits a simple
description. If X = {v}, then Ŷ = T̂ v is the closure of T exp(CT )v, where CT is
a cone in tR. If TC is closed, then Ŷ = T exp(QX)v, where QX ⊆ tR is a convex
polytope (the convex hull of the inverse image of X for the mapping ξ → exp(ξ)v,
ξ ∈ tR). Any segment in QX corresponds to an analytic strip or an annulus in
Ŷ . In general, Ŷ is the union of T̂ u, where u runs over exp(QX)v. Also, Ŷ is
distinguished in closTCv by a finite family of monomial inequalities of the type
|z1|s1 . . . |zn|sn ≤ c,(0.2)
where c ≥ 0 and s = (s1, . . . , sn) ∈ Rn depend on v and X . Vectors s correspond
to normals of faces of CT +QX .
Thus, the problem of determination of Ĝv is not difficult if Gv ⊂ TCv. The
latter is equivalent to the assumption that the complex orbit GCv is connected. In
Example 3.4, we give a construction for orbits which satisfy this condition; here is
a sketch. The group G = FT acts on the space V = C(K), where K is a finite
F -invariant subset of t∗: F acts naturally on C(K), t = t∗∗ is naturally embedded
into C(K), and T = exp(t) acts on C(K) by multiplication. If v ∈ C(K) is an
F -invariant function, then Gv ⊂ TCv. According to Theorem 3.5, each connected
complex orbit can be realized in this way. Further, we describe pairs (V,G) such
Gv ⊂ TCv for a generic v ∈ V.(0.3)
By Theorem 4.3, under the additional assumption that the complex linear span of
TCv coincides with V , this happens if and only if the group GCZ, where Z is the
centralizer of G in GL(V ), has an open orbit in V . There are two extreme cases:
(A) Z ⊆ GC; (B) G has a finite center. An example for (A) is the group G = SnTn
acting in Cn, where Tn is the torus of all diagonal matrices in U(n) and Sn is the
group of all permutations of coordinates. Replacing Tn with SU(n)∩Tn, we get an
example for (B). Example 4.4 contains a construction for pairs (V,G) that satisfy
(0.3). Theorem 4.5 states that the construction is a universal one. In Theorem 4.10,
we determine pairs which satisfy (0.3) and the following condition:
vectors s in (0.2) are independent of v.(0.4)
The paper also contains a description of hulls Ĝv for G = Aut0(D), where D is a
bounded symmetric domain in the canonical realization and Aut0(D) is the stable
subgroup of zero, which coincides with the group of all linear automorphisms of
D. These hulls have already been described: the final step was done in paper [9],
which essentially used [10], partial results appear in [14] and [8]. Most of them
use the technique of Jordan triples and Jordan algebras. We use Lie theory, in
particular, an explicit construction of paper [15] for a maximal abelian subspace a.
A compact group acting in a Euclidean space is called polar if there exists a subspace
ORBITS OF TORI EXTENDED BY FINITE GROUPS AND THEIR POLYNOMIAL HULLS: THE CASE OF CONNECTED COMPLEX ORBITS3
(a Cartan subspace) such that each orbit meets it orthogonally. The group G is
polar in the ambient linear space d, and a is the Cartan subspace for G. Real polar
representations are classified in paper [3]; they are orbit equivalent (i.e., have the
same orbits) to isotropy representations of Riemannian symmetric spaces. If D is
a polydisc Dn ⊂ Cn, where D is the unit disc in C, then G = SnTn; the polynomial
hulls Ĝv are determined by the inequalities
µk(z) ≤ µk(v),(0.5)
where k = 1, . . . , n and µk are defined by
µk(z) = max{|zσ(1) . . . zσ(k)| : σ ∈ Sn}.(0.6)
The general case can be reduced to this one in the following way. Any bounded
symmetric domain D ⊂ d of rank n admits an equivariant embedding of Cn to
d, which induces an embedding of Dn to D, such that Rn ⊂ Cn is the maximal
abelian subspace a, and, for any v ∈ a , the hull of Aut0(D)v is the orbit of the hull
of Aut0(D
n)v. Each µk(z) has a unique continuation to a K-invariant function on
d. The extended functions determine hulls by the same inequalities. Moreover, they
are plurisubharmonic and can be treated as products of singular values of z ∈ d or
as norms of exterior powers of adjoint operators in suitable spaces. The subsystem
of long roots of the restricted root system (i.e., the root system for a) has type nA1;
this defines the above embedding Cn → d. Furthermore, this makes it possible to
determine hulls in terms of the adjoint representation (Theorem 5.7). Thus, there
is no need to consider different types of domains separately.
The reduction to the case of a torus extended by a finite group, which is de-
scribed above, is contained in Section 5 (in papers [9], [14], the problem is also
reduced to this case by another method). It does not use essentially the results
of the previous sections (only Proposition 3.2, in proof of Theorem 5.7). These
extensions satisfy conditions (0.3) and (0.4); in addition, they possess the property
that the complexified groups have open orbits. According to Theorem 4.10, any
group with these properties is the product of groups SnT
n acting in Cn; it admits
a natural realization as a group of automorphisms of a bounded symmetric domain
(Corollary 5.3).
The following simple examples illustrate the case Gv 6⊆ TCv and show that
condition (0.3) is essential. Let G = SnT
n, and let ǫ1, . . . , ǫn be the standard base
in Cn. Then Ĝǫ1 is the closure the union of discs Dǫk, k = 1, . . . , n. Set H = SnT,
where T acts by z → eitz, t ∈ R, z ∈ Cn. Then Ĥǫ1 = Ĝǫ1. For v = ǫ1 + ǫ2, Ĝv is
the closure of the union of
bidiscs but Tn contains no proper torus T such that
Ĝv = Ĥv for H = SnT . However, for any subgroup F ⊆ Sn which acts transitively
on 2-sets and H = FTn we have Ĝv = Ĥv.
1. Preliminaries
We keep the notation of Introduction, in particular, (0.1) and (0.6). Linear
spaces are supposed to be finite dimensional and complex unless the contrary is
explicitly stated. ”Generic” means ”in some open dense subset”. Throughout the
paper, we use the following notation:
D and T are the open unit disc and the unit circle in C, respectively;
V denotes a complex linear space (except for Section 5);
4 V.M. GICHEV
if V is equipped with a linear base identifying it with Cn, then Tn is the
group of all diagonal unitary transformations;
Zn2 consists of all transformations in T
n with eigenvalues ±1;
ǫ1, . . . , ǫn is the standard base in C
n and Rn;
Rn+ is the set of vectors in R
n with positive entries;
SK denotes the group of all permutations of a finite setK; ifK = {1, . . . , n},
then SK = Sn;
C(K) is the algebra of all complex-valued functions on K;
1 is the identity of C(K);
G ⊂ GL(V ) is a compact group whose identity component is a torus T
(except for Section 5);
t ⊂ gl(V ) is the Lie algebra of T , tR = it, tC = t+ tR;
TR = exp(tR), TC = exp(tC);
C∗ = TC = C \ {0};
Ť = Hom(T,T) is the dual group to T ;
Aut(D) is the group of all holomorphic automorphisms of a domain D ⊂ V ,
Aut0(D) = Aut(D) ∩GL(V );
coneX denotes the least convex cone which contains the set X ;
convX is the convex hull of X ;
closX is the closure of X ;
spanFX is the linear span of X over the field F = C,R,Q.
Clearly, exp is bijective on tR and TR ∼= TC/T . The differentiating at the identity e
defines an embedding of Ť into the dual space t∗: χ → −ideχ, where χ ∈ Ť . This
is a lattice in the vector group t∗, moreover, T ∼= t/L, where L is the dual lattice
to Ť in t. For χ ∈ Ť , let
Vχ = {v ∈ V : gv = χ(g)v for all g ∈ T }
be the corresponding isotypical component of V . Then
⊕Vχ.(1.1)
We assume that V is equipped with a G-invariant inner product 〈 , 〉. Then de-
composition (1.1) is orthogonal. Let spec(v) denote the spectrum of v ∈ V (the set
of χ ∈ Ť such that the χ-component of v is nonzero); for X ⊆ V ,
spec(X) = ∪x∈X spec(x).
We say that T has a simple spectrum if
dimVχ ≤ 1(1.2)
for all χ ∈ Ť . If (1.2) is true, then there exists a unique (up to scaling factors)
orthogonal base in V which agree with (1.1) and a unique maximal torus Tn in
GL(V ) which contains T . In what follows, we assume that (1.2) holds; we shall see
in the next section that such assumption is not restrictive. Thus, we may fix an
identification
V = Cn = C(K),(1.3)
where K = {1, . . . , n}. If F is a subgroup of SK , then C(K)F denotes the set
of all F -invariant functions on K; clearly, 1 ∈ C(K)F . Further, (C∗)n is the
multiplicative group of all invertible functions in C(K), Tn consists of functions
with values in T, and (Tn)C = (C∗)n. The Lie algebra of Tn is realized as iRn ⊂ Cn.
ORBITS OF TORI EXTENDED BY FINITE GROUPS AND THEIR POLYNOMIAL HULLS: THE CASE OF CONNECTED COMPLEX ORBITS5
The embedding T → Tn induces embeddings of the Lie algebra and the fundamental
group: t → iRn, π1(T ) → iZn ⊂ iRn, respectively. Let Γ be the image of π1(T ).
Then spanR Γ = t; moreover, t∩ iZn = Γ and t/Γ = T . The dual mapping Ťn → Ť ,
which is defined by the restriction of characters e−i〈x,y〉, where x ∈ iZn, to t, is
the orthogonal projection πt : iZ
n → t. Thus, Γ is a subgroup of finite index in
Ť = πtiZ
n. Vectors in spanQ Ť are called rational. The image of t in iR
n can be
distinguished by linear equations with integer coefficients. Hence, clos(TCv), for
a generic v ∈ V , is the set of all solutions to a finite number of equalities with
holomorphic monomials. Thus, Y ⊂ TC implies Ŷ ⊂ clos(TCv). Set
CT = t
R ∩ clos(−Rn+),(1.4)
The cone iCT is dual to cone(specV ) ⊆ t∗ ⊆ iRn. If −ξ ∈ clos(Rn+), then ι =
limt→+∞ exp(tξ) is an idempotent in C(K) such that the multiplication by the
complementary idempotent 1− ι is a projection onto spanC(spec(ξ)). Set
IT = {limt→+∞ exp(tξ) : ξ ∈ CT }.(1.5)
Clearly, IT is finite and contains 1.
Lemma 1.1. The closure of exp(CT ) is equal to IT exp(CT ).
Proof. Due to the evident inclusion clos(exp(CT )) ⊇ IT exp(CT ), it is sufficient to
prove that the set ST = IT exp(CT ) is closed. Clearly, ST is an abelian semigroup.
The cone CT is polyhedral; hence, it is finitely generated:
CT = cone{ξ1, . . . , ξm},
where R+ξk are the extreme rays of CT , k = 1, . . . ,m. Obviously, IT is a finite
semigroup, which is generated by the idempotents limt→+∞ exp(tξk). Thus, the cor-
respondence (e−t1 , . . . , e−tm) → exp(t1ξ1 + . . . , tmξm) defines a mapping of (0, 1]m
onto exp(CT ), which continuously extends to [0, 1]
m. It follows that its image is
closed and coincides with ST . �
Note that there is a natural one-to-one correspondence between IT and the set
of faces of CT .
2. Hulls of finite unions of T -orbits in a TC-orbit
Let v ∈ TC. If v =
χ∈Ť vχ, where vχ ∈ Vχ, g ∈ TC, and u = gv, then
χ∈Ť χ(g)vχ. Since χ(g) 6= 0 for all g ∈ G and χ ∈ Ť , we get
u ∈ TCv =⇒ spec(u) = spec(v);(2.1)
dim (Vχ ∩ spanC Tv) ≤ 1 for all v ∈ V and χ ∈ Ť .(2.2)
Thus, the assumption that T has a simple spectrum in V is not restrictive in
the problem of description of polynomial hulls of orbits Gv such that Gv ⊂ TCv.
Clearly, Dn = T̂n in L(V ). For each x ∈ CT and any polynomial p on L(V ), the
holomorphic function f(ζ) = p(exp(ζx)) is bounded in the halfplane Π : Re ζ ≥ 0.
Hence, exp(Π) is contained in T̂ . On the other hand, if z ∈ Dn ∩ TC, then z =
t exp(x) for some t ∈ T and x ∈ CT (the polar decomposition). By Lemma 1.1,
T̂ = clos(Dn ∩ TC) = T clos(exp(CT )) = TIT exp(CT ).
6 V.M. GICHEV
If v ∈ (C∗)n, then (C∗)nv = (C∗)n, and the mapping z → zv is a linear nondegen-
erate transformation of Cn. Therefore,
v ∈ (C∗)n =⇒ T̂ v = T̂ v = TIT exp(CT )v.(2.3)
For an arbitrary v ∈ V = Cn, set
CvT = {ξ ∈ tR : ξk ≤ 0 if vk 6= 0, k = 1, . . . , n}.
Applying (2.3) to spanC(spec(v)) = C
nv, we get
T̂ v = T clos(exp(CvT )v.(2.4)
Clearly, CvT depends only on spec(v). For s ∈ Rn and z ∈ (C∗)n, set
νs(z) =
|zk|sk .
If sk ≥ 0, then the k-th factor in (C∗)n can be replaced with C (i.e., νs continuously
extends to this product).
It is well known that for any holomorphically convex T -invariant set U ⊆ TC,
the set log(U ∩ TR) ⊆ tR is convex. In particular, this is true for sets of g ∈ TC
such that gv ∈ T̂X, where X ⊂ TCv, v ∈ V . Nevertheless, it is convenient to have
an explicit construction of an analytic strip (or an annulus, if it is periodic) in a
TC-orbit, which corresponds to a segment that joins two points in tR; it is contained
in the following lemma. Set
S = {z ∈ C : 0 ≤ Im z ≤ 1}.
Lemma 2.1. Let v ∈ Cn and u ∈ TRv. Then, there exists ξ ∈ tR such that
λ(z) = exp(zξ)v(2.5)
is a holomorphic mapping λ : S → TCv which satisfies conditions
λ(∂S) ⊆ Tv ∪ Tu,
λ(0) = v, λ(1) = u.
If the stable subgroup of v in TR is trivial, then ξ is unique.
Proof. These properties hold for ξ ∈ tR such that exp(ξ)v = u; such a ξ exists, since
exp is a bijection tR → TR. The last assertion is clear. �
If ξ ∈ CT , then (2.5) defines an analytic halfplane in T̂ v; for Γ-rational ξ, λ is
periodic and defines an analytic disc in T̂ v. Together with Lemma 2.1 this gives a
characterization of hulls for finite unions of T -orbits in TC. Suppose that X ⊂ TRv
is finite and the stable subgroup of v in T is trivial. Then, the inverse to the
mapping x → exp(x)v is well defined. Let us denote it by logv and set
QX = conv(logv X),(2.6)
PX = QX + CT .(2.7)
The set PX is a convex polyhedron, which is unbounded if CT 6= 0. Hence, there
exists a finite set NX ⊂ Rn and, for each s ∈ NX , real numbers cs such that
PX = {x ∈ tR : 〈x, s〉 ≤ cs for all s ∈ NX}.(2.8)
The set NX consists of vectors orthogonal to faces of PX , whose projections into
spanR PX look outside of it; clearly, it is not unique in general.
ORBITS OF TORI EXTENDED BY FINITE GROUPS AND THEIR POLYNOMIAL HULLS: THE CASE OF CONNECTED COMPLEX ORBITS7
Proposition 2.2. Let v ∈ (C∗)n. Suppose that Y ⊂ TCv is a finite union of
T -orbits (including Tv), and set X = TRv ∩ Y . Then X is finite and
Ŷ = clos (T exp(PX)v)(2.9)
= clos {z ∈ (C∗)n : νs(z) ≤ ecsνs(v), s ∈ NX}(2.10)
u∈exp(QX )v
T̂ u(2.11)
= T exp(PX)IT v,(2.12)
where QX , PX ,NX are as above and IT is defined in (1.5).
Proof. Due to the polar decomposition, the set Tu∩TRv, for each u ∈ TCv, is non-
void and consists of a single point. Hence, X is finite and Y = TX . The inclusion
(expQX)v ⊆ Ŷ follows from Lemma 2.1 and Phragmén–Lindelöf Principle. The in-
clusion exp(CT )u ⊆ T̂ u is true for any u ∈ Cn. Since it holds for all u ∈ T exp(QX),
the left-hand side of (2.9) includes the right-hand side. If z = exp(ξ)v, where ξ ∈ tC,
then zk = e
ξkvk, k = 1, . . . , n; due to (2.8), this implies that the right-hand side
of (2.9) coincides with (2.10). According to (2.3), the right-hand side of (2.9) and
(2.11) intersect TCv by the set
T exp(PX)v = exp(QX)T exp(CT )v;
clearly, it is dense in (2.11). Since QX is compact, the set (2.11) is closed. The
compactness of QX , the above equality, and Lemma 1.1 imply that (2.12) is closed;
hence, it is the same as the right-hand side of (2.9).
Each of the sets (2.9)–(2.12) includes Y . Thus, it remains to prove that (2.12)
is polynomially convex. If x ∈ tR \ PX , then there exists s ∈ Rn such that
sup{〈y, s〉 : y ∈ PX} < 〈x, s〉 .(2.13)
Since QX is compact, the linear functional on t in the right-hand side of (2.13) must
be nonnegative on CT . According to (1.4), we may assume that s ∈ closRn+. It
follows that (2.13) holds in a neighborhood of s in closRn+. Thus, s can be assumed
rational (hence, integer) with strictly positive entries. Then, p(z) = zs11 . . . z
a holomorphic polynomial such that |p | separates exp(x)v and T clos(exp(PX)v).
Therefore,
Ŷ ∩ TCv = T exp(PX)v.
For any ι ∈ IT , the projection z → ιz commutes with T . This makes it possible to
apply the above arguments to the vector ιv, the set ιX , and to the restriction of T
to ιCn. Consequently,
ι̂Y ∩ TCιv = T exp(PιX)ιv = ιT exp(PX)v(2.14)
(clearly, ι exp(PX)v = exp(PιX)ιv). By (1.5), ιY ⊆ Ŷ , hence, ι̂Y ⊆ Ŷ ; on the other
hand, ιŶ ⊆ ι̂Y since p◦ι is a polynomial on Cn for any polynomial p on ιCn. Thus,
ιŶ = ι̂Y = Ŷ ∩ ιCn. Together with (2.14), this implies the polynomial convexity of
(2.12). �
If T = Tn, then Proposition 2.2 follows from the well-known characterization of
polynomially convex Reinhardt domains.
Corollary 2.3. For any v ∈ (C∗)n, the orbit TCv is closed in Cn if and only if Tv
is polynomially convex, and this is equivalent to CT = 0. Then, Ŷ = T exp(QX)v
for all Y,X as above.
8 V.M. GICHEV
Proof. The orbit TCv is closed if and only if the convex hull of spec(v) = spec(Cn)
contains 0 in its relative interior (see, for example, [13, Proposition 6.15]). Since
T ⊆ GL(n,C), the set spec(Cn) is generating in t∗. Hence, TCv is closed if and
only if CT = 0; by (2.4), this is equivalent to T̂ v = Tv. Then, Ŷ = T exp(QX)v by
(2.9) and (2.1). �
There is a version of the first assertion for an arbitrary compact linear group G:
a GC-orbit is closed if and only if it contains a polynomially convex G-orbit ([7,
Theorem 1 and Theorem 5]). For a torus T , all T -orbits in TCv are simultaneously
polynomially convex or non-convex, but this is not true if G is not abelian.
3. Finite extensions of T that keep a TC-orbit
In this section, we consider the case where the set X defined in the previous
section is an orbit of a finite group F which normalizes T and keeps the TC-orbit.
We assume that T ⊆ G, T is a torus, G is a subgroup of GL(V ), F is a finite
subgroup of G, and
G = FT = TF, F ∼= G/T,(3.1)
Gv ⊆ TCv,(3.2)
v ∈ (C∗)n ⊂ Cn = V.(3.3)
By (3.1), T is normal in G. Clearly, (3.2) is equivalent to Fv ⊆ TCv and to the
connectedness of GC. Here is an illustrating example.
Example 3.1. Let G = Aut0(D
2) be the group of linear automorphisms of the
bidisc D2 ⊂ C2. Clearly, G = FT , where F = S2 is generated by the transposition
τ of the coordinates, T = T2, TC = (C∗)2, and TCv = (C∗)2 for any v that lies
outside the coordinate lines. Thus, (3.2) holds for all v ∈ (C∗)2 (however, (3.2) fails
for any v 6= 0 in C2 \ (C∗)2). The hull Ĝv can be distinguished by the inequalities
max{|z1|, |z2|} ≤ max{|v1|, |v2|},(3.4)
|z1z2| ≤ |v1v2|.(3.5)
Clearly, (3.4) and (3.5) define a polynomially convex set. Let z1, z2 > 0 (a generic
T -orbit evidently contains such a point z). Then, z and τz can be joined by an
analytic strip with the boundary in Tz ∪ Tτz:
λz(s) = (z
2 ), s ∈ S.
Set q = ln z1
and let z1 > z2. Then, the strip can be written in the form
λz(s) = (e
−sz1, e
sz2), 0 ≤ Re s ≤ q.
It is periodic with the period 2πi and defines a τ -invariant annulus in Ĝv with τ -
fixed points (
z1z2,
z1z2) and (−
z1z2,−
z1z2). As z2 → 0, the annulus tends
to a couple of discs: (e−sz1, 0) and (0, e
−sz1), where Re s > 0, 0 ≤ Im s ≤ 2π (the
circle Re s = q
, 0 ≤ Im s ≤ 2π collapses to zero). Let z ∈ Ĝv ∩ R2. Then Ĝv
contains a bidisc D2z. It intersects R2 by a rectangle, which is symmetric with
respect to the coordinate axes. If z lies on an axis, then the rectangle degenerates
into a segment. Let v1 > v2 > 0. The union of these rectangles with vertices
in the set Q of real points of the annulus, which joins v and τv, is a curvilinear
octagon. It degenerates into a pair of segments if v2 = 0 and into a square if v1 = v2
(see [10, Fig. 2] for the 3-dimensional case). In the logarithmic coordinates in the
ORBITS OF TORI EXTENDED BY FINITE GROUPS AND THEIR POLYNOMIAL HULLS: THE CASE OF CONNECTED COMPLEX ORBITS9
first quadrant, Q is a segment. Also, note that all nontrivial TC-orbits are not
closed. �
In [2], Björk found a typical situation where analytic annuli appear in the maxi-
mal ideal space MA of a commutative Banach algebra A which admits a nontrivial
action of T by automorphisms: this happens if T -invariant functions on MA do
not separate distinct T-orbits. In [7], it was noted that analytic strips and/or an-
nuli appear in Ĝv if the stable subgroup of v in GC does not coincide with the
complexification of the stable subgroup of v in G.
Proposition 3.2. The hulls Ĝv for orbits of G = Aut0(D
n) = SnT
n are distin-
guished by inequalities (0.5), where µk are defined by (0.6).
Proof. The approximation by decreasing sequences of hulls makes it possible to
reduce the proposition to the case of a generic v in (0.5). Then, applying to v =
(v1, . . . , vn) a suitable transformation in T
n, we may assume that
v1 > v2 > · · · > vn > 0.(3.6)
Moreover, we may use Proposition 2.2 with X = Snv, CT = − closRn+ (we keep
the notation of Proposition 2.2). Since X , QX , CT , PX , and µk are Sn-invariant,
Sn is transitive on X , by (0.6), (0.5), and (2.10), it is sufficient to prove that the
vectors ξk =
r=1 ǫr, k = 1, . . . , n, correspond to the faces of PX that meet at v,
are orthogonal to them, and look outside of PX .
Set η1 = ǫ2 − ǫ1, . . . , ηn−1 = ǫn − ǫn−1, ηn = −ǫn. Then {−ηk}nk=1 is a base in
Rn, which is dual to the base {ξk}nk=1. We claim that the cone of the polyhedron
PX at the vertex v is generated by {ηk}nk=1. This implies the assertion above
(note that both cones are simplicial). If τ ∈ Sn is a transposition (k, j), then
v − τv = (vk − vj)(ǫk − ǫj). If σ, κ ∈ Sn then v − σκv = (v − κv) + (κv − σκv).
Furthermore, Sn is generated by transpositions (k, k + 1), and vk − vk+1 > 0 by
(3.6), where k = 1, . . . , n− 1. Therefore, vectors η1, . . . , ηn−1 generate the cone of
QX at v. Since −ǫk =
r=0 ηn−r and CT is generated by −ǫk, k = 1, . . . , n, this
proves the proposition. �
Property (3.2) implies spanC Gv = spanC T
Cv. Hence, we may assume that (1.2)
is valid. Then, a generic ξ ∈ t has a simple spectrum. Any f ∈ F permutes
eigenvalues and eigenspaces. Thus, assuming (1.2) and identifying V with C(K)
in accordance with (1.3), we get that each element of F is a composition of a
permutation of K and a multiplication by a function on K. Further, (3.3) implies
that the stable subgroup of v in TC is trivial. Hence,
TRv ∼= TC/T ∼= tR,
where the identification of TRv and tR is realized by ξ → exp(ξ)v, ξ ∈ tR.
Lemma 3.3. Let G ⊂ GL(V ), a subgroup F ⊆ G, and v ∈ V satisfy (3.1)–
(3.3). Then TCv contains a G-invariant T -orbit. Moreover, there exists a mapping
f → tf , F → T , such that F̃ = {tff : f ∈ F} is a subgroup of G which has a fixed
point in TC and satisfies (3.1)–(3.3).
Proof. The group F naturally acts on TRv ∼= TC/T ∼= tR. Any g ∈ F is a composi-
tion of σ ∈ SK and a multiplication by a function in C(K). Since t acts on C(K)
by multiplication on linear functions and σ induces a linear transformation in tC,
the induced action of F on tR is affine. Since F is finite, it has a fixed point in tR.
10 V.M. GICHEV
Hence, TCv contains a G-invariant T -orbit. Let us fix a point u in it and define
tf by tffu = u; the choice is unique due to (3.3). Taken together with (3.1), this
implies that F̃ is a group, which obviously satisfies the lemma. �
According to Lemma 3.3, we may assume without loss of generality that
fv = v for all f ∈ F.(3.7)
In the following example we give a construction (associated with a given finite group
F ) for orbits with property (3.2).
Example 3.4. Let t be a real linear space, t∗ be the dual space to t, L be a lattice
in t, and L∗ ⊂ t∗ be the dual lattice to L. Set
λx(y) = y(x), where x ∈ t, y ∈ t∗.
Let K be a finite subset of L∗ that generates L∗ as a subgroup of the vector group
t∗. Then
t∗ = spanR K,(3.8)
L = {x ∈ t : λx(K) ⊂ Z}.(3.9)
Further, let F be a finite subgroup of GL(t) which keeps K. Set V = C(K). The
mapping
λ : x → iλx
(3.10)
is an embedding t → V , which has a natural extension to tC. Set
exp(x) = e2πiλx .(3.11)
Clearly, L = ker exp. Hence, exp defines an embedding of T = t/L and TC into the
group (C∗)n:
TC = exp(tC) ⊆ (C∗)n.
The group TC acts on C(K) by multiplication. The inclusion v ∈ C(K)F is the
same as (3.7); it implies (3.2). Furthermore, if v ∈ (C∗)n, then
spanC Tv = V.(3.12)
Indeed, the space spanC T is a subalgebra of C(K), which separates points of the
finite set K. Hence, it coincides with C(K). �
Theorem 3.5. Let a group G ⊂ GL(V ), a finite subgroup F ⊆ G, a torus T , and
a vector v ∈ V satisfy (3.1)–(3.3), (3.7), and (3.12). Then V,G, F, T, v can be
realized as in Example 3.4, where
v ∈ (C∗)n ∩ C(K)F .(3.13)
Conversely, if V,G, F, T, v are as in Example 3.4 and v satisfies (3.13), then (3.1)–
(3.3), (3.7), and (3.12) are true.
Proof. The group F acts in t and t∗ by the adjoint action. Let K ⊂ t∗ be the
collection of all weights for the representation of T in V ; clearly, K is F -invariant.
It follows from (3.12) and (3.2) that the weights are multiplicity free. This defines
an equivariant linear isomorphism between V and C(K), where the group T acts by
multiplication. Thus, λ and exp are well defined by (3.10) and (3.11). According
to (3.7) and (3.3), (3.13) is true; (3.8) holds since T ⊂ GL(V ) is compact and acts
effectively on V (note that the annihilator of spanR K in t acts trivially due to
ORBITS OF TORI EXTENDED BY FINITE GROUPS AND THEIR POLYNOMIAL HULLS: THE CASE OF CONNECTED COMPLEX ORBITS11
(3.11) and (3.13)). Let us define L by (3.9). Then L = ker exp by (3.11). Hence, L
is a lattice in t and the group L∗ generated by K is the dual lattice in t∗.
The converse was proved in Example 3.4. �
4. Finite extensions of T which keep generic TC-orbits
In what follows, we use the setting of Example 3.4. Let Z denote the centralizer
of G in GL(V ). We assume that (C∗)n acts in V = C(K) by multiplication.
Lemma 4.1. Z = C(K)F ∩ (C∗)n.
Proof. Since λ(t) separates points of K, Z ⊆ (C∗)n. The multiplication by u ∈
C(K) commutes with F if and only if u is F -invariant. �
In general, condition (3.2) does not hold for a generic vector v. Hence, there is
a natural problem: describe V and G such that generic orbits satisfy (3.2). The
following proposition contains a simple criterion.
Proposition 4.2. Let V,G be as in Example 3.4. Then G satisfies (3.2) for a
generic v ∈ V if and only if
C(K) = λ(tC) + C(K)F .(4.1)
In this case, each TC-orbit in (C∗)n intersects C(K)F .
Proof. It follows from Lemma 4.1 that the right-hand side of (4.1) is the tangent
space at 1 to the set TC Z. Clearly, G̃C = ZGC is a group, TCZ is the identity
component of G̃C, and the right-hand side of (4.1) is the tangent space to G̃C1.
Hence, (4.1) holds if and only if G̃C1 is open. Moreover, this is equivalent to the
equality TCZ = exp(λ(tC) + C(K)F ) = (C∗)n. Therefore, each TC-orbit in (C∗)n
intersects C(K)F , i.e., contains an F -fixed point. Thus, (4.1) implies (3.2) for
v ∈ (C∗)n.
Let (3.2) hold and letW be an F -invariant neighborhood of 1. IfW is sufficiently
small, then the condition log 1 = 0 defines a branch of log in W . We may assume
that logW is convex and symmetric. This makes it possible to define roots in W :
r = exp
. For v ∈ W 12 and f ∈ F , set gf =
, where r = cardF ,
and g =
f∈F gf . Then gv is F -fixed. If (3.2) holds for v, then gf ∈ TC for all
f ∈ F ; hence, gv ∈ TCv. Consequently, for all v ∈ W , TCv intersects C(K)F . Since
Z keeps this property of orbits, it follows that TCZ has a nonempty interior. This
implies (4.1). �
Theorem 4.3. Let G ⊂ GL(V ) be a semidirect product of a torus T and a finite
subgroup F , and let Z be the centralizer of G in GL(V ). Suppose that spanC Tv = V
for some v ∈ V . Then the following conditions are equivalent:
(i) Gv ⊂ TCv for a generic v ∈ V ;
(ii) GCZv is open in V for a generic v ∈ V .
Proof. By Theorem 3.5, we may use the construction of Example 3.4. According to
Lemma 4.1, (ii) is equivalent to (4.1), and the assertion follows from Proposition 4.2.
12 V.M. GICHEV
We shall give a constructive description of these spaces and groups. Set
C0(K) =
u ∈ C(K) :
u(q) = 0
Sometimes, we identify points in K with their characteristic functions.
Example 4.4. Let V = Cn = C(K), where K = {1, . . . , n}, let F be a subgroup
of Sn, and
K = K1 ∪ · · · ∪Kp(4.2)
be the partition of K into F -orbits. For k ∈ {1, . . . , p}, set Vk = C(Kk). Then
V = V1 ⊕ · · · ⊕ Vp. Set
t0k = C0(Kk) ∩ iRn,
= exp(t0
) ⊂ C(Kk),
where exp is defined by (3.11). Set t0 = t01 ⊕ · · · ⊕ t0p,
T 0 = exp(t0) = T 01 × · · · × T 0p .
Let T be an F -invariant torus such that
T 0 ⊆ T ⊆ Tn(4.3)
and set G = FT . Then generic GC-orbits satisfy (3.2). The group G is irreducible
if and only if F is transitive on K; in general, F -orbits in K define G-irreducible
components of V . There are two extreme cases in (4.3).
(A) If T = Tn, then there is one open orbit (C∗)n of the groupGC = FTC, which
evidently satisfies (3.2). If F is nontrivial, then there exist degenerate orbits
that do not satisfy (3.2); moreover, if F is transitive onK, then all non-open
GC-orbits, except for zero, are nontrivial finite unions of TC-orbits.
(B) If T = T 0, then generic orbits are closed. They have codimension p and
are distinguished by equations
zr = ck,
where ck ∈ C∗, k = 1, . . . , p.
Note that (A) and (B) are invariant under the Cartesian product (the group F
need not be the product of groups Fk of irreducible components but must have the
same orbits in K as F1 × · · · × Fp). In terms of Example 3.4: in (A), t = Rn, the
mapping λ : tC → C(K) is surjective, K = {ǫ1, . . . , ǫn}; in (B), t = iRn ∩ C0(K),
λ(tC) = C0(K), and the set K is the projection of {ǫ1, . . . , ǫn} into t∗ = t. In both
cases, K is the set of all vertices of a regular simplex. �
Theorem 4.5. Let V,G be as in Theorem 4.3 and let (i) hold. Then V,G can be
realized as in Example 4.4. Furthermore,
(1) V,G are of type (A) if and only if GC has an open orbit,
(2) (B) is equivalent to the assumption that the center of G is finite,
(3) if G is irreducible, then either (A) or (B) holds.
Let C(K)F+ be the cone of all nonnegative functions in C(K)
Lemma 4.6. Let G and V be as in Example 3.4. Then, the orbit GCv is closed
for a generic v ∈ V if and only if
R ∩ C(K)F+ = 0.(4.4)
ORBITS OF TORI EXTENDED BY FINITE GROUPS AND THEIR POLYNOMIAL HULLS: THE CASE OF CONNECTED COMPLEX ORBITS13
Proof. Clearly, GCv is closed if and only if TCv is closed. Let v ∈ (C∗)n. By
Proposition 2.2 and Corollary 2.3, TCv is not closed if and only if CT 6= 0. Since
CT is F -invariant by (1.4), it contains
f∈F fu for each u ∈ CT . Thus, CT = 0 is
equivalent to (4.4). �
Proof of Theorem 4.5. Suppose that G is irreducible or, equivalently, F is transi-
tive. Then Z = C∗ 1 according to Lemma 4.1. If 1 ∈ λ(tC), then TC ⊇ Z and
TCv is open for a generic v ∈ V by Theorem 4.3. If 1 /∈ λ(tC), then (4.4) is true;
by Lemma 4.6, TCv is closed for a generic v ∈ V . By Proposition 4.2, a generic
TC-orbit intersects C∗1. Consequently, we have
codimGCv = 1.(4.5)
Let 1 ∈ TC ∩ Z. The orthogonal projection of 1 into the tangent space T1TC1 is
F -fixed. Hence, it is proportional to 1; since 1 /∈ λ(tC), this implies 1 ⊥ λ(tC)1.
Therefore, T1T
C1 coincides with the tangent space to the hypersurface z1 . . . zn = 1
at 1; since the monomial on the left is an eigenfunction of TC, this group keeps it.
Due to (4.5), TC1 coincides with this hypersurface. Then, T = Tn ∩ SU(n), and
any TC-orbit that intersects Z is a hypersurface z1 . . . zn = c, for some c ∈ C∗. This
implies tC = C0(K) and T = T
Thus, the theorem is proved for all irreducible G. The projection onto each
irreducible component keeps the property (3.2) for generic orbits since it commutes
with G. Hence, (i) holds for all irreducible components. They correspond to F -
orbits Kk in the partition (4.2). Let t
, k = 1, . . . , p , be defined as in Example 4.4.
According to the arguments above, λ (t|Kk) ⊇ λ(t0k) for all k. If x ∈ t, then the
averaging
Ax = 1
f∈F fx, r = cardF,
distinguishes the F -fixed component of x (i.e., Ax ∈ C(K)F ∩ t and x− Ax ∈ t0);
since t is F -invariant, it contains both components. By Lemma 4.1, if G has a finite
center, then λ (t|Kk) = λ(t0k) for all k. It follows that
t ⊆ t0 = t01 ⊕ · · · ⊕ t0p.
On the other hand, (ii) and Lemma 4.1 imply codim t ≤ dimC(K)F = p. Hence,
the inclusion above is in fact the equality. Thus, we get (B) assuming that G has
a finite center. The converse is true since t0 does not contain a nontrivial F -fixed
element. The same arguments show that any F -invariant torus T includes T 0 if (i)
is true. This proves that V,G admit the realization of Example 4.4; (1) and (2) are
clear. �
Corollary 4.7. Let G be as in Theorems 4.5 and 4.3. Then G contains a closed
subgroup G0 such that
(1) each connected component of G contains a connected component of G0,
(2) G0 has a finite center,
(3) generic orbits of (G0)C are closed,
(4) Gv ∩ TRv = G0v ∩ (T 0)Rv for a generic v ∈ V .
Proof. By Theorem 4.5 and (4.3), G ⊇ T 0, where T 0 is as in (B). Clearly, F
normalizes T 0. Hence, G0 = FT 0 is a group, which satisfies the corollary. �
14 V.M. GICHEV
Proposition 2.2 makes it possible to find Ĝv for G as above. If T = Tn, then
T ⊃ Zn2 and generic T -orbits intersect Rn+; hence, we may assume v ∈ Rn+. Then
T̂ v ∩ Rn is a parallelepiped Πv = conv{(±v1, . . . ,±vn)}. Clearly, Πv = Zn2Π+v ,
where Π+v = Πv ∩ closRn+. Since Rn+ = TRv, we may use Proposition 2.2 with
X = Fv, CT = − closRn+, PX = conv(Fv) − Rn+:
Ĝv = ∪u∈exp(Qv)Dnu = T ∪u∈exp(Qv) Πu = T ∪u∈exp(Qv) Π+u ,
where Qv = convFv. For the description in the form (2.10), one has to know
normal vectors to faces of convFv. Since F may be an arbitrary subgroup of
Sn, they need not be proportional to rational vectors (for example, this is true
for the cyclic subgroup of order 3 in S3). We shall describe the situation where
they are locally independent of v; since they depend on v continuously, this is
equivalent to the condition that they are rational. Note that the vector which joins
two points in tR as in Lemma 2.1 is rational if and only if the strip reduces to an
annulus. In Example 4.4, F need not be the product of groups corresponding to
the irreducible components; we shall see that F possesses this property in the case
under consideration.
Let U be a real vector space and F ⊂ GL(U) be a finite group. Set
Cu = cone(u− Fu);
this is the cone at the vertex u of the polytope conv(Fu) (which may be degenerate).
We say that Cu is locally independent of u if, for a generic u ∈ U , Cu = Cw for all
w that are sufficiently close to u.
Lemma 4.8. Let U be a real vector space and F be a finite subgroup of GL(U).
Suppose that Cu is locally independent of u. Then F is generated by reflections in
hyperplanes in U .
Proof. We may assume without loss of generality that U is equipped with an inner
product and that F ⊆ O(U). Let R+(u − fu), f ∈ F , be an extreme ray of Cu.
The equality Cu = Cw for w in a neighborhood of u implies that this ray does not
change near u. Hence, dim(1 − f)U = 1. Since f is orthogonal and nontrivial, it
is a refection in a hyperplane. The stable subgroup of a generic u ∈ U is trivial
(hence, F acts freely on a generic orbit) and each vertex of conv(Fu) can be joined
with u by a chain of edges. Applying the above arguments repeatedly to u, fu, etc.,
we get that F is generated by reflections in hyperplanes. �
For any g ∈ Zn2Sn and k = 1, . . . , n, gǫk = ± ǫσ(k) for some σ ∈ Sn. The mapping
f → σ is a natural homomorphism Zn2Sn → Sn, which we denote by φ.
Lemma 4.9. Let F be a transitive subgroup of Sn acting in R
n by permutations
of coordinates and let a group H ⊆ Zn2Sn be generated by reflections in hyperplanes
in Rn. If φ(H) = F , then F = Sn.
Proof. Let ρ be a reflection in a hyperplane in Rn. If ρ ∈ Zn2Sn = BCn, then it
is conjugate to a reflection in a wall of the Weyl chamber that is distinguished by
the inequalities x1 > · · · > xn > 0. Hence, φ(ρ) is a transposition if it is nontrivial.
Since F = φ(H), F is generated by transpositions. It remains to note that any
subgroup of Sn, which is generated by transpositions, coincides with Sn if it is
transitive on {1, . . . , n} (consider the graph with the vertices {1, . . . , n} and edges
corresponding to transpositions and note that inclusions (k, l) ∈ F , (l,m) ∈ F
imply (k,m) ∈ F ; this makes it possible to use the induction). �
ORBITS OF TORI EXTENDED BY FINITE GROUPS AND THEIR POLYNOMIAL HULLS: THE CASE OF CONNECTED COMPLEX ORBITS15
We say that a pair (V,G) is standard if it is isomorphic to (A) or (B) in Ex-
ample 4.4 with F = SK . The product of pairs (Vk, Gk), k = 1, . . . ,m, is the pair
k=1 Vk,
k=1 Gk).
Theorem 4.10. Let G = FT be a compact subgroup of GL(n,C), where T ⊆ Tn
is a torus and F is a subgroup of Sn. Suppose that Gv ⊂ TCv for a generic v ∈ V
(1) either T = Tn or the center of G is finite,
(2) for a generic v ∈ Cn, Ĝv can be distinguished in closTCv by a family of
inequalities
|z1|s1 . . . |zn|sn ≤ ρs(v),
where ρs(v) ≥ 0 and vector s = (s1, . . . , sn) runs over a certain finite subset
of Rn which is independent of v.
Then (V,G) is isomorphic to the product of standard pairs. Moreover, if G is
irreducible, then (V,G) is standard.
Proof. Let G be irreducible. Then F is transitive and (V,G) are as in (A) or as
in (B) by Theorem 4.5. Suppose that (B) is the case. It follows from (2) and
Proposition 2.2 that the polytope QX ⊂ tR, where X = Gv ∩ TRv, for a generic
v, satisfies the assumption of Lemma 4.8. Therefore, F is generated by reflections
(we may assume that F ⊂ O(tR)). They extend to reflections in hyperplanes in
tR+R1 = Rn if we assume that they fix 1. Then, Lemma 4.9 implies F = Sn. The
case (A) can be reduced to (B): it is sufficient to replace Tn with T = SU(n) ∩ Tn
since F evidently keeps T and to note that (2) remains true due to Proposition 2.2.
Thus, (V,G) is standard.
Let the center of G be finite. According to Theorem 4.5, T may be identified
with the group T 0 in Example 4.4. In particular, GCv is closed for a generic v
and CT = 0 due to Proposition 2.2. By Proposition 4.2, generic orbits contain
F -fixed points. Applying the arguments above (which did not use the assumption
that G is irreducible), we get that the cones at the vertices of the convex polytope
QX , X = Gv ∩ TR ⊂ tR, are locally independent of v. Clearly, the same is true
for its projection into each space t0k corresponding to an irreducible component
Vk of V = C
n. This implies that all irreducible components are standard. Thus,
Fk = S(Kk), where k = 1, . . . , p and K = K1 ∪ · · · ∪Kp is the partition of K into
F -orbits. Due to Theorem 4.5, it is sufficient to prove that
F = F1 × · · · × Fp.(4.6)
By Lemma 4.8, F |t0 is generated by reflections in hyperplanes in t0; the condition
that they keep real F -invariant functions on K uniquely defines their extension to
Rn. Hence, F is generated by reflections in Rn. A permutation which induces a
reflection in a hyperplane in Rn is a transposition of a pair of coordinates; this pair
is necessarily contained in only one of the sets Kk, k = 1, . . . , p. This proves (4.6).
If T = Tn, then T is a product of tori in irreducible components. Thus, the case
T = Tn follows from the above case, since the assumptions of the theorem hold
true for the group T 0 if they hold for T in (4.3) in Example 4.4. �
5. Hulls of isotropy orbits of bounded symmetric domains
We start with a preliminary material on hermitian symmetric spaces following
[15] but adapting the exposition to our purpose in order to be as self contained as
16 V.M. GICHEV
possible. For a subset X of a Lie algebra g, z(X) = {z ∈ g : [z,X ] = 0} is the
centralizer of X . Let G be a simple real noncompact Lie group with a finite center,
K be its maximal compact subgroup, and g, k be their Lie algebras, respectively.
If the center z = z(k) of k is nontrivial, then g is called hermitian. Then k = z(z)
and dim z = 1 (note that K is irreducible in g/k). Let c be a Cartan subalgebra
of k. Then c is also a Cartan subalgebra of g and z ⊆ c. There exists k ∈ z such
that ad(k) has eigenvalues 0,±i (it is unique up to a sign; ker ad(k) = k). Then
κ = eπ ad(k) is the Cartan involution which defines the Cartan decomposition
g = k⊕ d,(5.1)
where k, d are eigenspaces for 1,−1, respectively. Furthermore, j = ad(k) is a
complex structure in d. This defines the structure of a hermitian symmetric space of
noncompact type in D = G/K. These spaces can be realized as bounded symmetric
domains in Cn with K = Aut0(D). Any irreducible bounded symmetric domain
admits such a realization. Let ∆ ⊆ ic∗ be the root system of gC. Each α ∈ ∆
corresponds to an sl2-triple hα, eα, fα such that ihα ∈ c. Thus, α(hα) = 2, [eα, fα] =
hα, and
[h, eα] = α(h)eα, [h, fα] = −α(h)fα(5.2)
for all h ∈ cC. We identify cC and (c∗)C equipping g with an Ad(K)-invariant
sesquilinear inner product and normalize it by the condition
max{|α| : α ∈ ∆} =
2.(5.3)
Then short roots must have length 1 (note that G2 has no real hermitian form).
The set ∆∨ = {hα : α ∈ ∆} is the dual root system. The above normalization
implies hα = α for long roots and hα = 2α for short ones. Since ad(h), h ∈ c, has
eigenvalues 0 and α(h), where α ∈ ∆, we get α(ik) = 0,±1, i.e., ik is a microweight
(of ∆∨). For s = 0,±1, set
∆s = {α ∈ ∆ : α(ik) = s}.(5.4)
Since k⊕ id is a compact real form of gC and spanR{ihα, eα − fα, i(eα + fα)} is the
su(2)-subalgebra corresponding to a root α ∈ ∆, we have
d = spanR{eα + fα, i(eα − fα) : α ∈ ∆1}.(5.5)
Set sα = spanR{ihα, eα + fα, i(eα − fα)}. Then sα is an sl(2,R)-subalgebra of gC
α ∈ ∆±1 ⇐⇒ sα ⊆ g.(5.6)
Let E be a maximal subset of pairwise orthogonal long roots in ∆1. Set
α∈E hα, e =
α∈E eα, f =
α∈E fα;
α∈E ⊕ sα.
Let α, β ∈ E, α 6= β. Since α, β are long and orthogonal, ±α± β /∈ ∆. Hence,
α, β ∈ E, α 6= β =⇒ [sα, sβ] = 0.(5.7)
It follows that h, e, f is an sl2-triple and s is a subalgebra of g. Set
θ = e
π ad(e−f),(5.8)
a = spanR θE.(5.9)
ORBITS OF TORI EXTENDED BY FINITE GROUPS AND THEIR POLYNOMIAL HULLS: THE CASE OF CONNECTED COMPLEX ORBITS17
Here is the standard realization of root systems Bn and Cn:
Bn = {±ǫk ± ǫl, ±ǫm : k, l,m = 1, . . . , n, k < l};
Cn = {±ǫk ± ǫl, ±2ǫm : k, l,m = 1, . . . , n, k < l}.
Then Cn = B
n , but Cn does not satisfy (5.3). These systems have microweights;
up to the action of the Weyl group, they are:
Bn : ǫ1;
(ǫ1 + · · ·+ ǫn).
There are no other irreducible root systems which have microweights and contain
roots of different lengths. Also, Bn and Cn have the sameWeyl group BCn = Z
Lemma 5.1. The space a is a maximal abelian subspace of d.
Proof. A straightforward calculation with 2-matrices shows that θh = e + f. By
(5.7),
θhα = eα + fα for all α ∈ E.(5.10)
It follows from (5.5) that a ⊆ d. Moreover, a is abelian due to (5.7). Set Ξ = ∆∩E⊥.
We claim that
Ξ ⊆ ∆0.(5.11)
Indeed, a root in ∆1 ∩ Ξ must be short. This may happen only in Bn or Cn, since
G2 and F4 have no microweights and other irreducible root systems have no roots
of different lengths. In Bn, k is a short root and all other short roots are orthogonal
to k. Hence, they do not belong to ∆1. In Cn, E = {2ǫ1, . . . 2ǫn}; then Ξ = ∅.
Since ∆−1 = −∆1, this proves (5.11).
Set b = E⊥ ∩ c and m = spanC{eα, fα : α ∈ Ξ}. It follows from (5.11) that
m ⊆ kC. Clearly, z(E) = cC ⊕ m. The space m is θ-invariant, because θ fixes roots
in Ξ. Due to (5.9), we get
z(a) = θz(E) = bC ⊕ aC ⊕m
Since bC ⊕m ⊆ kC, this implies z(a) ∩ d = a. �
The projection of θ∆ into a is the restricted root system ∆a (it is also the set of
roots for ad(a) in g). The group
W = {Ad(g) : g ∈ K, Ad(g)a = a}|a,
acting in a, is the Weyl group of a.
In what follows, we denote by v the complexification of a with respect to the
complex structure j (thus, v ⊂ d). The set θE is a base in v; enumerating it, we
identify v with Cn. Set t = spanR iE, T = exp t, H = WT . The torus T = T
n is a
maximal compact subgroup in the group exp s ⊆ G.
Proposition 5.2. The following assertions hold:
(1) ∆a is a root system of type BCn or Cn;
(2) the pair (v, H) is standard with T = Tn.
Proof. (1). Let ∆a \ θE contain a long root α. Then α = 12 (α1 + α2 + α3 +
α4) for some α1, . . . , α4 ∈ θE, since |α|2 = 2 and 〈α, β〉 = 0,±1 for all β ∈ E
due to the normalization (5.3) (note that α, β generate A2 if 〈α, β〉 6= 0). Roots
α, α1, . . . , α4 generate D4, since only A4 and D4 among irreducible systems of rank
18 V.M. GICHEV
4 consist of roots of equal length, but A4 does not contain an orthogonal base. Since
〈ik, β〉 = 1 for all β ∈ E, the projection of iθk into spanR D4 is a microweight ω
such that 〈ω, αk〉 = 1, k = 1, . . . , 4, but D4 has no microweight with this property
(in the realization above, D4 = B4 ∩ C4 and the microweights are either ±ǫk or
(±ǫ1± ǫ2± ǫ3± ǫ4)). Thus, E ∪ (−E) is the set of all long roots in ∆a. According
to the classification of irreducible root systems, only Cn and BCn = Bn ∪ Cn has
the property that linearly independent long roots are mutually orthogonal.
(2). The maximal compact subgroup of the group corresponding to s is Tn.
Hence, T = Tn ⊃ Zn2 . Systems Cn and BCn have the same Weyl group W = BCn.
Therefore, H = WT = SnT
Let D be a bounded symmetric domain in a complex linear space d (may be,
reducible) and v ⊆ d be the complex linear span of a maximal abelian subspace in
d (thus, we identify d with the corresponding space in the Cartan decomposition
(5.1), which is induced by the Cartan involutions in irreducible components). Let
Aut00(v, D) denote the subgroup of all linear transformations in Aut(D) which keep
v and each irreducible component of D.
Corollary 5.3. Let F be a subgroup of Sn, G = FT
n ⊂ GL(n,C). Then G
satisfies condition (2) of Theorem 4.10 if and only if (V,G) is isomorphic to a pair
(v,Aut00(v, D)) for a bounded symmetric domain D.
Proof. All pairs (Cn, SnT
n) appear as (v,Aut00(v, D)) for matrix balls D. It re-
mains to combine Theorem 4.10 and Proposition 5.2. �
It is possible now to describe hulls of K-orbits in d (with respect to the complex
structure j) it terms of Proposition 3.2. The key point is that K is polar in d: each
K-orbit meets a orthogonally (i.e., a is a Cartan subspace). This is true, since all
maximal abelian subspaces are conjugate in d by K, ad(a) is symmetric if a ∈ d
and, for a generic a ∈ a, ker ad(a) = a; hence,
[a, g] = a⊥.(5.12)
We may include the linear base in v into a base in d as the first n vectors of the
latter. Then z1, . . . , zn are coordinates in v and linear functions in d. The functions
µk in (0.6) admit a K-invariant extension to d:
µk(z) = sup{|(gz)1 . . . (gz)k| : g ∈ K},(5.13)
where k = 1, . . . , n. The following lemma shows that (5.13) is an extension indeed.
Lemma 5.4. For z ∈ v, (0.6) and (5.13) coincide.
Proof. It follows from (5.12) that any critical point of the linear function Re z1 on
the orbit Kz belongs to a. If the lemma is not true, then there exist z ∈ v and
k ∈ {1, . . . , n} such that |(gz)k| > |zk|. Transformations in Sn and T reduce the
problem to the case z1 > · · · > zn > 0 and k = 1, but then the assumption implies
that Re z1 attains its maximal value on Kz outside of a. �
Proposition 5.5. For any v ∈ d, K̂v = {z ∈ d : µk(z) ≤ µk(v), k = 1, . . . , n}.
Proof. Due to (5.13), each µk is a supremum of absolute values of holomorphic
polynomials. Hence, the right-hand side is polynomially convex. Thus, it includes
K̂v. The inverse inclusion holds, since each K-orbit intersects v by an H-orbit
and hulls of H-orbits are distinguished in v by the same inequalities according to
Proposition 3.2 and Lemma 5.4. �
ORBITS OF TORI EXTENDED BY FINITE GROUPS AND THEIR POLYNOMIAL HULLS: THE CASE OF CONNECTED COMPLEX ORBITS19
The functions µk can be written in more invariant terms. To do it, note that the
Weyl group of ∆a has the form Z
2Sn in the base θE by (5.9); thus, zk = αk(z),
k = 1, . . . , n, where αk ∈ θE and z ∈ a. Therefore, zk are eigenvalues of ad(z)
in the subspace generated by the corresponding root vectors. The problem is to
distinguish this subspace (in fact, we use a slightly different version). After that,
functions µk can be defined as norms of some operators according to the following
lemma (this observation was used in [12] in another context).
Lemma 5.6. Let V be a Euclidean space and A be a symmetric nonnegative oper-
ator in V with eigenvalues λ1 ≥ λ2 ≥ · · · ≥ λm ≥ 0, where m = dim V . Let A∧k be
its natural extension to the k-th exterior power V ∧k =
V . Then
‖A∧k‖V ∧k = λ1 . . . λk,
where ‖ ‖k is the operator norm with respect to the inner product in V ∧k.
Proof. The norm of a nonnegative symmetric operator is equal to its maximal
eigenvalue. �
Let v ∈ g be semisimple and π(v) denote the projection onto ker ad(v) along
other eigenspaces of ad(v) (note that π(v) is a function of ad(v), since it is the
residue at zero of the resolvent of ad(v)). Set
a(v) = ad([v, [v, k]])π(v) ad(k),
pk(v) = ‖a(v)∧k‖g∧k , k = 1, . . . , n.
The space d is a(v)-invariant and ker a(v) ⊇ k. We assume that g is equipped
with some K-invariant inner product, which extends the inner product in d. It
follows from the calculation below that a(v) is symmetric and has range ad(k)a.
Let n = dim a be the rank of the symmetric space D. It is equal to the codimension
of a generic K-orbit in d.
Theorem 5.7. For any v ∈ d,
K̂v = {z ∈ d : pk(z) ≤ pk(v), k = 1, . . . , n}.
Proof. It is sufficient to prove the assertion for a generic v ∈ d. Clearly, pk are
K-invariant. Hence, we may assume v ∈ a. Then, by (5.9) and (5.10),
vα(eα + fα),
where vα ∈ R. According to (5.2) and (5.4), [k, v] =
α∈E ivα(eα − fα). Thus,
[v, [v, k]] =
2iv2αhα
due to (5.7). Also, (5.7) implies that ad(ihα) keeps v and has eigenvalues 0,±2i
in it for each α ∈ E. Therefore, v is ad([v, [v, k]])-invariant and its eigenvalues are
±4v2αi, α ∈ E. Since ad(k)g = d and π(v)d = a for a generic v ∈ a, the space v is
a(v)-invariant; moreover, a(v)g = ad(k)a ⊆ v. Thus, a(v) has eigenvalues 0,±4v2α
in g. According to Lemma 5.6 and (0.6),
pk(v) = 4
kµ2k(v)(5.14)
for v ∈ v and k = 1, . . . , n. Since pk and µk are K-invariant, (5.14) holds for all
v ∈ d. The theorem follows from Proposition 5.5. �
20 V.M. GICHEV
Corollary 5.8. Functions pk, k = 1, . . . , n, are plurisubharmonic in d with respect
to the complex structure j = ad(k).
Proof. By (5.14) and (5.13),
pk(z) = 4
k sup{|(gz)21 . . . (gz)2k| : g ∈ K}.
The right-hand side is plurisubharmonic, since the functions z2
are j-holomorphic
and j is K-invariant. �
One can get the same functions pk by replacing g with d, endowed with the
complex structure j, and a(v) with ad([v, jv])(π(v) + π(jv)).
Acknowledgement. I thank Anton Pankratiev for helpful comments.
References
[1] J.T. Anderson, Polnomial hulls of sets invariant under an action of the special unitary group,
Can. J. Math., v. XL, (1988) no. 5, 1256-1271.
[2] J.-E. Björk, Compact groups operating on Banach algebras, Math. Ann., v. 205 (1973), no. 4,
281–297.
[3] J. Dadok, Polar coordinates induced by actions of compact Lie groups, Transactions Amer.
Math. Soc., 288 (1985), 125-137.
[4] A. Debiard, B. Gaveau, Equations de CauchyRiemann sur SU(2) et leurs enveloppes dholo-
morphie, Can. J. Math., v. 38 (1986), 1009-1024.
[5] T. Gamelin, Uniform Algebras, Prentice-Hall, Englewood Cliffs, N. J., 1969.
[6] V.M. Gichev, Maximal ideal spaces of invariant function algebras on compact groups, preprint,
46 pp., available at http://arXiv.org/abs/math/0603449
[7] V.M. Gichev, I.A. Latypov, Polynomially Convex Orbits of Compact Lie Groups, Transfor-
mation Groups, v. 6 (2001) no. 4, 321-331.
[8] J. Faraut, L. Bouattour, Enveloppes polynomiales densembles compacts invariants, Math.
Nachrichten 266 (2004), 20-26.
[9] W. Kaup, Bounded symmetric domains and polynomial convexity, Manuscr. Math. 114 (2004),
No.3, 391-398.
[10] W. Kaup, D. Zaitsev, On the CR-structure of compact group orbits associated with bounded
symmetric domains, Invent. Math., 153 (2003), 45-104.
[11] J. Kane, Maximal ideal spaces of U-algebras, Illinois J. Math., v.27 (1983), No 1, 1-13.
[12] B. Kostant, On convexity, the Weyl group and the Iwasawa decimposition, Ann. Sci. Ecole.
Norm. Super., 6 (1973), 413–455.
[13] V.L. Popov, E.B. Vinberg, Invariant theory, Itogi Nauki i Tekhniki, Sovr. Probl. Mat. Fund.
Napravl., v. 55, VINITI, Moscow 1989, pp. 137–309 (in Russian); English transl.: Algebraic
Geometry IV, Encyclopaedia of Math. Sciences, v. 55, Springer-Verlag, Berlin 1994, pp. 123-
[14] C. Sacre, Enveloppes polynomiales de compacts, Bull. Sci. math. 116 (1992), 129-144.
[15] J.A. Wolf, Fine Structure of Hermitian Symmetric Spaces, Symmetric spaces (Short Courses,
Washington Univ., St. Louis, Mo., 1969-1970), Pure Appl. Math., Vol. 8. New York: Dekker
1972, pp. 271-357.
Omsk Branch of Sobolev Institute of Mathematics, Pevtsova 13, 644099, Omsk, Rus-
E-mail address: gichev@ofim.oscsbras.ru
http://arXiv.org/abs/math/0603449
	Introduction
	1. Preliminaries
	2. Hulls of finite unions of T-orbits in a TC-orbit
	3. Finite extensions of T that keep a TC-orbit
	4. Finite extensions of T which keep generic TC-orbits
	5. Hulls of isotropy orbits of bounded symmetric domains
	Acknowledgement
	References
