Harmonic Analysis and the Trace Formula

5Abstracts
Symmetry Breaking Operators for Orthogonal Groups O(n, 1)
Toshiyuki Kobayashi
Given an irreducible representation π of a group G and a subgroup G′ , we may
think of π as a representation of the subgroup G′ (the restriction π|G′ ). A typicalexample is the tensor product representation π1 ⊗ π2 of two representations π1 andπ2 of a group H, which is obtained by the restriction of the outer tensor productπ1 ⊠ π2 of the direct product group G := H × H to its subgroup G′ := diag(H).
As branching problems, we wish to understand how the restriction π|G′ behaves
as a G′ -module. For reductive groups, this is a diﬃcult problem, partly because
the restriction π|G′ may not be well under control as a representation of G′ evenwhen G′ is a maximal subgroup of G. Wild behavior such as infinite multiplicities
may occur, for instance, already in the tensor product representation of SL3 (R).
The author proposed in [3] to go on successively, further steps in the study ofbranching problems via the following three stages:
Stage A: Abstract feature of the restriction π|G′Stage B: Branching laws.
Stage C: Construction of symmetry breaking operators (Definition 0.1).
Here, branching laws in Stage B ask an explicit decomposition of the restriction
into irreducible representations of the subgroup G′ when π is a unitary representation, and also ask the multiplicity m(π, τ ) := dim HomG′ (π|G′ , τ ) for irreducible
representations τ of G′ . The latter makes sense even when π and τ are nonunitary.
Stage C refines Stage B, by asking an explicit construction of SBOs when π andτ are realized geometrically.
Definition 0.1. An element in HomG′ (π|G′ , τ ) is called a symmetry breaking operator, SBO for short.
Stage A includes a basic question whether spectrum is discrete or not, see [1].
Another fundamental question in Stage A is an estimate of multiplicities. In [2, 6],
we discovered the following geometric creteria to control multiplicities:
Theorem 0.2 (geometric criteria for finite/bounded multiplicities).
(1) The dimension of HomG′ (π|G′ , τ ) is finite for any irreducible representations π of G and any τ of G′ iﬀ G × G′ /diag(G′ ) is real spherical.
(2) The dimension of HomG′ (π|G′ , τ ) is uniformly bounded with respect to πand τ iﬀ (GC × G′C )/diag(G′C ) is spherical.
Here we recall
Definition 0.3.
(1) A complex manifold XC with holomorphic action of acomplex reductive group GC is spherical if a Borel subgroup of GC has anopen orbit in XC .

6Oberwolfach Report 25/2017(2) A real manifold X with continuous action of a real reductive group G isreal spherical if a minimal parabolic subgroup of G has an open orbit inX.

The latter terminology was introduced in [2] in search for a broader frameworkfor global analysis on homogeneous spaces than the usual (e.g. group manifolds,symmetric spaces). That is, the function space C ∞ (G/H) (or L2 (G/H) etc.)
should be under control by representation theory if(1)dim HomG (π, C ∞ (G/H)) < ∞b adm ,for all π ∈ Gand hence we could expect to develop global analysis on G/H by using representation theory if (1) holds[2]. We discovered and proved that the geometric property“real spherical” characterizes exactly the representation-theoretic property (1):Fact 0.4 ([2, 6]). Let X = G/H where G ⊃ H are algebraic real reductive groups.
b adm ) iﬀ X is real spherical.
(1) dim HomG (π, C ∞ (X)) < ∞ (∀ π ∈ G(2) dim HomG (π, C ∞ (X)) is uniformly bounded iﬀ XC is spherical.
Theorem 0.2 follows from Fact 0.4.
The classification of the real spherical spaces of the form (G × G′ )/diag(G′ )was accomplished in [5] when (G, G′ ) is a reductive symmetric pair. This a priori
estimate in Stage A singles out the settings which would be potentially promisingfor Stages B and C of branching problems. One of such settings arises from adiﬀerent discipline, namely, from conformal geometry. The first complete solution
to Stage C obtained [7] is related to this geometric setting as below.
Given a Riemannian manifold (X, g), we write G = Conf(X, g) for the group of
conformal diﬀeomorphisms of X. Then there is a natural family of representations
πλ of G on C ∞ (X) for λ ∈ C given by(πλ (h)f )(x) = Ω(h−1 , x)λ f (h−1 · x)

for h ∈ G, x ∈ X.

We can extend this to a family of representations on the space E i (X) of diﬀerential(i)i-forms, to be denoted by πλ .
If Y is a submanifold of X, then there is a natural morphism
G′ := {h ∈ G : h · Y ⊂ Y } → Conf(Y, g|Y ).
Then we may compare two families of representations of the group G′ :(i)• the restriction πλ |G′ acting on E (i) (X),(j)• the representation π ′ ν acting on E (j) (Y ).
A conformally covariant SBO on diﬀerential forms is a linear map E i (X) → E j (Y )(i)(j)that intertwines πλ |G′ and π ′ ν . Here is a basic question arising from conformal
geometry:
Question 0.5. Let X be a Riemannian manifold X, and Y a hypersurface. Construct and classify conformally covariant SBOs from E i (X) to E j (Y ).

Harmonic Analysis and the Trace Formula

7

We are interested in “natural operators” D that persist for all pairs (X, Y ).
The larger Conf(X; Y ) is, the more constrains are on D, and hence, we first
focus on the model space with largest symmetries which is given by (X, Y ) =(S n , S n−1 ). In this case the pair (G, G′ ) of conformal groups is locally isomorphic
to (O(n+1, 1), O(n, 1)). It then turns out that the criterion in Theorem 0.2 (2) forStage A is fulfilled. Then Question 0.5 is regarded as Stages B and C of branchingproblems. Recently, we have solved completely Question 0.5 in the model space:• Continuous SBOs for i = j = 0 were constructed and classified in [7].
• Diﬀerential SBOs for general i and j were constructed and classified in [4].
• The final classification is announced in [8].
Here is a flavor of the complete classification:
Theorem 0.6. If HomG′ (πλ |G′ , π ′ ν ) ̸= {0} for some λ, ν ∈ C, then j ∈ {i −2, i − 1, i, i + 1} or i + j ∈ {n − 2, n − 1, n, n + 1}.
(i)

(j)

In the talk, I gave briefly the methods of the complete solution [4, 7, 8], some ofwhich are also applicable in a more general setting that Theorem 0.2 (an a priori
estimate for Stage A) suggests.
Finally, some applications of these results include:• an evidence of a conjecture of Gross and Prasad for O(n, 1), see [8];• periods of irreducible unitary representations with nonzero cohomologies;• a construction of discrete spectrum of the branching laws of complementary
series [7, Chap. 15].
References[1] T. Kobayashi, Discrete decomposability of the restriction of Aq (λ) with respect to reductive
subgroups and its applications, Invent. Math. 117 (1994), 181–205; Part II, Ann. of Math.
(2) 147 (1998), 709–729; Part III, Invent. Math. 131 (1998), 229–256.
[2] T. Kobayashi, Introduction to harmonic analysis on real spherical homogeneous spaces,
Proceedings of the 3rd Summer School on Number Theory “Homogeneous Spaces and Automorphic Forms” in Nagano (F. Sato, ed.), 1995, 22–41.
[3] T. Kobayashi, A program for branching problems in the representation theory of real reductive groups, In: Representations of Lie Groups, In Honor of David A. Vogan on his 60thBirthday, Progress in Mathematics 312, pp. 277–322. Birkhäuser, 2015.
[4] T. Kobayashi, T. Kubo, and M. Pevzner, Conformal Symmetry Breaking Diﬀerential Operators for Diﬀerential Forms on Spheres, Lecture Notes in Mathematics, vol. 2170, iv + 192
pages, 2016. ISBN: 978-981-10-2657-7.
[5] T. Kobayashi, T. Matsuki, Classification of finite-multiplicity symmetric pairs, Transform.
Groups 19 (2014), 457–493, Special issue in honor of Dynkin for his 90th birthday.
[6] T. Kobayashi and T. Oshima, Finite multiplicity theorems for induction and restriction,
Adv. Math. 248 (2013), 921–944.
[7] T. Kobayashi, B. Speh, Symmetry Breaking for Representations of Rank One OrthogonalGroups, Mem. Amer. Math. Soc., vol. 238, No. 1126, (2015), v+112 pages, ISBN: 978-14704-1922-6.
[8] T.Kobayashi, B.Speh, Symmetry breaking for orthogonal groups and a conjecture by B.
Gross and D. Prasad, submitted, arXiv:1702.00263.