arXiv:1704.04018v1 [math.RT] 13 Apr 2017

The Fourier transform on the group GL2(R) and
the action of the overalgebra gl4
Yu.A.Neretin1
We define a kind of ’operational calculus’ for GL2 (R). Namely, the group GL2 (R)
can be regarded as an open dense chart in the Grassmannian of 2-dimensional subspaces in R4 . Therefore the group GL4 (R) acts in L2 on GL2 (R). We transfer the
corresponding action of the Lie algebra gl4 to the Plancherel decomposition of GL2 (R),
the algebra acts by differential-difference operators with shifts in an imaginary direction. We also write similar formulas for the action of gl4 ⊕ gl4 in the Plancherel
decomposition of GL2 (C).

1

The statement of the paper

1.1. The group GL( R). Let GLn (R) be the group of invertible real matrices
of order n. We denote elements of GL2 (R) by


x11 x12
.
X=
x21 x22
The Haar measure on GL2 (R) is given by
(det X)−2 dX = (det X)−2 dx11 dx12 dx21 dx22 .
Recall some basic facts on representations of the group GL2 (R), for systematic exposition of the representation theory of SL2 (R), see, e.g., [5], [7]. Denote
by R× the multiplicative group of R. Let µ ∈ C and ε ∈ Z/2Z. We define the
function xµ//ε on R× by
xµ//ε := |x|µ sgn(x)ε .
Denote by Λ the set of all collections
(µ1 , ε1 ; µ2 , ε2 ),
i.e.,
Λ ≃ C × Z2 × C × Z2 .
For each element of Λ we define a representation Tµ,ε of GL2 (R) in the space of
functions on R by


x11 x12
ϕ(t) =
Tµ1 ,ε1 ;µ2 ,ε2
x21 x22
1/2+µ2 //ε2

 x + tx 
x
x12
12
22
.
· (x11 + tx21 )−1+µ1 −µ2 //ε1 −ε2 det 11
=ϕ
x21 x22
x11 + tx21
(1.1)
1 Supported

by the grant FWF, Project P28421.

1

A space of functions can be specified in various ways, it is convenient to
consider the space Cµ∞1 −µ2 ,ε1 −ε2 of C ∞ -functions on R such that2
ϕ(−1/t)(−t)−1+µ1 −µ2 //ε1 −ε2
also is C ∞ -smooth.
Thus, for any fixed X ∈ GL2 (R) we get an operator-valued function (µ1 , ε1 ; µ2 , ε2 ) 7→
Tµ1 ,ε1 ;µ2 ,ε2 on Λ holomorphic in the variables µ1 , µ2 .
Generators of the Lie algebra gl2 (R) act by formulas
d
+ (−1/2 + µ1 ),
dt
d
= −t2 + t(−1 + µ1 − µ2 ),
dt

d
,
dt
d
= t + (1/2 + µ2 ).
dt

L11 = −t

L12 =

(1.2)

L21

L22

(1.3)

The expressions for the generators do not depend on ε1 , ε2 . However the space
of the representation depends on ε1 − ε2 .
Consider integral operators
Aµ1 ,ε1 ;µ2 ,ε2 : Cµ∞1 −µ2 ,ε1 −ε2 → Cµ∞2 −µ1 ,ε1 −ε2
defined by
Aµ1 ,ε1 ;µ2 ,ε2 f (t) :=

Z

(t − s)−1−µ1 +µ2 //ε1 −ε2 f (s) ds.

R

The integral is convergent if Re(µ1 − µ2 ) < 0 and determines a function
holomorphic in µ1 , µ2 . As usual (see, e.g., [4], §I.3), the integral admits a
meromorphic continuation to the whole plane (µ1 , µ2 ) ∈ C2 with poles at µ1 −
µ2 = 0, 1, 2, . . . . The operators Aµ1 ,ε1 ;µ2 ,ε2 are intertwinig,
Aµ1 ,ε1 ;µ2 ,ε2 Tµ1 ,ε1 ;µ2 ,ε2 = Tµ2 ,ε2 ;µ1 ,ε1 Aµ1 ,ε1 ;µ2 ,ε2

(1.4)

(parameters (µ1 , ε1 ) and (µ2 , ε2 ) of T are transposed). If µ1 − µ2 6= ±1, ±2,
. . . , then representations Tµ1 ,ε1 ;µ2 ,ε2 are irreducible and operators Tµ1 ,ε1 ;µ2 ,ε2
are invertible.
Representations Tµ1 ,ε1 ;µ2 ,ε2 form a so-called (nonunitary) principal series of
representations. Recall description of some unitary irreducible representations
of GL2 (R).
Unitary principal series. If µ1 , µ2 ∈ iR, then the representation is unitary in
L2 (R). Denote by Λprincipal ⊂ Λ the subset consisting of tuples (is1 , ε1 ; is2 , ε2 )
with s1 > s2 .
Discrete series. Notice that the group GL2 (R) acts on the Riemann sphere
C = C ∪ ∞ by linear-fractional transformations
X : z 7→

x12 + zx22
,
x11 + zx21

(1.5)

2 This condition means that functions ϕ are smooth as sections of line bundles on the
projective line R ∪ ∞.

2

these transformations preserve the real projective line R ∪ ∞ and therefore
preserves its complement C \ R. If det X > 0, then (1.5) leave upper and lower
half-planes invariant, if det X < 0, then (1.5) permutes the half-planes.
Let n = 1, 2, 3, . . . . Consider the space Hn of holomorphic functions ϕ on
C \ R satisfying
Z
|ϕ(z)|2 | Im z|−n−1 d Re z d Im z < ∞.
C\R

In fact, ϕ is a pair of holomorphic functions ϕ+ and ϕ− determined on halfplanes Im z > 0 and Im z < 0. These functions have boundary values on R in
distributional sense (we omit a precise discussion since it is not necessary for our
purposes). The space Hn is a Hilbert space with respect to the inner product
Z
hϕ1 , ϕ2 i =
ϕ1 (z)ϕ2 (z) | Im z|−n−1 d Re z d Im z.
C\R

For s ∈ R, δ ∈ Z2 we define a unitary representation Dn,s of GL2 (R) in Hn by
Dn,s


x11
x21


 x + zx 
x12
12
22
ϕ(z) = ϕ
(x11 + zx21 )−1−n ×
x22
x11 + zx21

1/2+n/2+is//δ
x
x12
× det 11
.
x21 x22

In fact, we have operators (1.1) for
µ1 = −n/2 − is, ε1 = −1 + n + δ; µ2 = n/2 + is, ε2 = δ
restricted to the subspace generated by boundary values of functions f+ and
f− .
We denote by Λdiscrete the set of all parameters (n, is) of discrete series.
1.2. The Fourier transform. Let ϕ be contained in the space C0∞ (GL2 (R)
of compactly supported function on GL2 (R). We consider a function that sent
each µ
e = (µ1 , ε1 ; µ2 , ε2 ) to an operator in Cµ∞1 −µ2 ,ε1 −ε2 given by
Z
dX
F (X) Tµ1 ,ε1 ;µ2 ,ε2 (X)
.
Tµ1 ,ε1 ;µ2 ,ε2 (F ) =
2
det(X)
GL2 (R)
Next, we define a subset Λtempered ⊂ Λ by
Λtempered := Λprincipal ∪ Λdiscrete .
Let us define Plancherel measure dP(e
µ) on Λtempered. On the piece Λprincipal it
is given by
1
(s1 − s2 ) tanh π(s1 − s2 )/2 ds1 ds2 ;
16π 3
1
dP(is1 , 1; is2 , ε2 ) =
(s1 − s2 ) coth π(s1 − s2 )/2 ds1 ds2 .
16π 3

dP(is1 , 0; is2 , ε2 ) =

3

On n-th piece of Λdiscrete it is given by
dP =

n
ds.
8π 3

Consider the space L2 of functions Q on Λtempered taking values in the space
of Hilbert–Schmidt operators in the corresponding Hilbert spaces and satisfying
the condition
Z


tr Q(e
µ)Q∗ (e
µ) dP(e
µ) < ∞.
Λtempered

This is a Hilbert space with respect to the inner product
Z


tr Q1 (e
µ)Q∗2 (e
µ) dP(e
µ).
hQ1 , Q2 iL2 :=
Λtempered

According the Plancherel theorem, for any F1 , F2 ∈ C0∞ (GL2 (R)) we have
hF1 , F2 iL2 (GL2 (R)) = hT (F1 ), T (F2 )iL2 (Λtempered ) .
Moreover, the map F → T (F ) extends to a unitary operator L2 (GL2 (R)) → L.
1.3. Overgroup. Let Mat2 (R) be the space of all real matrices of order 2.
By Gr4,2 (R) we denote the Grassmannian of 2-dimensional subspaces in R2 ⊕R2 .
For any operator R2 → R2 its graph is an element Gr4,2 (R). The set Mat2 (R)
of such operators is an open dense chart in Gr4,2 (R).
The group GL4 (R) acts in a natural way in R4 and therefore on the Grassmannian. In the chart Mat2 (R) the action is given by the formula (see, e.g.,
[18], Theorem 2.3.2)


A B
: X 7→ (A + XC)−1 (B + XD),
C D


A B
where
is an element of GL4 (R) written as a block matrix of size 2 + 2.
C D
The Jacobian of this transformation is (see, e.g., [18], Theorem 2.3.2)
−4

det(A + XC)



A
det
C

B
D

2

.

For σ ∈ iR we define a unitary representation of GL4 (R) in L2 (Mat2 (R)) by
Rσ



A
C



B
F (X) = F (A + XC)−1 (B + XD) ×
D
× | det(A + XC)|−2+2σ det



A
C

(these representations are contained in degenerate principal series).

4

B
D



1−σ

The group GL2 (R) is an open dense subset in Mat2 (R). Therefore, we can
identify the spaces L2 on GL2 (R) and Mat2 (R). For this we consider a unitary
operator


J : L2 Mat2 (R) → L2 GL2 (R)

given by

Jσ F (X) = F (X) · | det X|1−σ


This determines a unitary representation Uσ := Jσ Rσ Jσ−1 of GL4 (R) in L2 GL2 (R) ,
the explicit expression is
Uσ



A
C



B
F (X) = F (A + XC)−1 (B + XD) ×
D
det(A + XC) det(B + XD)


×
A B
det X det
C D

−1+σ

. (1.6)

a subgroup GL2 (R) × GL2 (R) ⊂ GL4 (R) consisting of matrices
 Consider


A 0
. For this subgroup we get the usual left-right action in L2 GL2 (R) ,
0 D
Uσ




A 0
F (X) = F (A−1 XD).
0 D

Formula (1.6) extends this formula to the whole group GL4 (R). The Lie algebra
gl4 acts in the space of functions on Mat2 (R) by first order differential operators,
which can be easily written; a list of formulas for all generators ekl , where k,
l = 1, 2, 3, 4, is given below in Subs. (2.5). We restrict this action to the space
of smooth compactly supported functions on GL2 (R). Notice that the operators
i · ekl are symmetric on this domain, but some of them are not essentially selfadjoint.
Our purpose is to write explicitly the images Ekl of operators ekl under the
Fourier transform.
1.4. Formulas. Operators Tµ1 ,ε1 ;µ2 ;ε2 (F ) have the form
Z ∞
K(t, s|µ1 , ε1 ; µ2 , ε2 ) ϕ(t) dt.
Tµ1 ,ε1 ;µ2 ;ε2 (F )ϕ(t) =

(1.7)

−∞

Recall that functions K are holomorphic in µ1 , µ2 .
We wish to write operators Ekl on kernels K. The complete list is contained
below in Subsect. 2.5, here we present two basic expressions.
The algebra gl4 (R) can be decomposed as a linear space into a direct sum
of four subalgebras a, b, c, d consisting of matrices of the form








∗ 0
0 0
0 0
0 0
,
,
,
0 0
0 ∗
∗ 0
0 ∗

5

The subalgebras a and d are isomorphic to gl2 (R), subalgebras b and c are
Abelian.
Formulas for the action of a and d immediately follow from the definition of
the Fourier transform. To obtain formulas for the whole gl4 , it is sufficient to
write expressions for one generator of b, say E14 , and one generator of c, say
E32 . After this other generators can be obtained by evaluation of commutators.
We define shift operators V1+ , V1− , V2+ , V2− by
V1± K(t, s|µ1 , ε1 ; µ2 , ε2 ) = K(t, s|µ1 ± 1, ε1 + 1; µ2 , ε2 );
V2± K(t, s|µ1 , ε1 ; µ2 , ε2 ) = K(t, s|µ1 , ε1 ; µ2 ± 1, ε2 + 1).
In this notation,
E14 =

−1/2 − σ + µ1 ∂ − −1/2 − σ + µ2 ∂ −
V +
V ;
µ1 − µ2
∂s 1
µ1 − µ2
∂t 2

E32 =

1/2 + µ1 + σ ∂ + 1/2 + µ2 + σ ∂ +
V +
V .
µ1 − µ2
∂t 1
µ1 − µ2
∂s 2

1.5. Remarks on a general problem. In [17] the author formulated the
following question: Assume that we know the explicit Plancherel formula for
the restriction of a unitary representation ρ of a group G to a subgroup H. Is
it possible to write the action of the Lie algebra of G in the direct integral of
representations of H?
Now it seems that an answer to this question is affirmative.
The initial paper [17] contains a solution for a tensor product3 of a highest
and lowest weight representations of SL2 (R). In this case the overalgebra acts
by differential-difference operators in the space L2 (S 1 × R+ ) having the form
Lf (ϕ, s) = D1 f (ϕ, s + i) + D2 f (ϕ, s) + D3 f (ϕ, s − i),
where Dj are second order differential operators in the variable ϕ.
In [10]–[13] Molchanov solved several rank 1 problems of this type, expressions are similar, but there appear differential operators of order 4. In [20]
the action of the overalgebra in restrictions from GLn+1 (C) to GLn (C), in this
case differential operators have order n. In all the cases examined by now shift
operators in the imaginary direction are present.
In the present paper, we write the action of the overalgebra in the restriction
of a degenerate principal series of the group GL4 (R) to GL2 (R). Notice that
canonical overgroups exist for all 10 series of real classical groups4 . Moreover
overgroups exist for all 52 series of classical semisimple symmetric spaces G/H,
see [9], [15], see also [18], Addendum D.6. So the problem makes sense for all
classical symmetric spaces.
3 The problem of a decomposition of a tensor product ρ ⊗ ρ of representations of a group
1
2
G is a special case of a decomposition of restrictions. Indeed, ρ1 ⊗ ρ2 is a representation of
the group G × G, and we must restrict this representation to the diagonal G.
4 Emphasize that in the present paper we consider groups GL and not SL, also we must
consider groups U(p, q) and not SU(p, q).

6

Shturm–Liouville problems for difference operators in L2 (R) in the imaginary
direction
λf (s) = a(s)f (s + i) + b(s)f (s) + c(s)f (s − i)
arise in a natural way in the theory of hypergeometric orthogonal polynomials,
see, e.g., [1], [8], apparently a first example (the Meixner–Pollaszek system)
was discovered by J. Meixner in 1930s. On such operators with continuous
spectra see [16], [6], [19]. See also a multi-rank work of I. Cherednik [2] on
Harish-Chandra spherical transforms.
1.6. The Fourier transform on GL2 (C). For a detailed exposition of
representations of the Lorentz group, see [14]. For ν, ν ′ ∈ C satisfying ν −ν ′ ∈ Z
′
we define the function z νkν on the multiplicative group of C by
′

′

z νkν := z ν z ν := |z|2ν z ν

′

−ν

.

Denote by ∆ the set of all (µ1 , µ′1 ; µ2 , µ′2 ) ∈ C4 such that µ1 −µ′1 ∈ Z, µ2 −µ′2 ∈ Z.
For (µ1 , µ′1 ; µ2 , µ′2 ) ∈ ∆ we define a representation of GL2 (C) in the space of
functions on C by


x11 x12
ϕ(t) =
Tµ1 ,µ′1 ;µ2 ,µ′2
x21 x22

 x + tx 
x
12
22
−1+µ1 −µ2 k−1+µ′1 −µ′2
det 11
(x11 +tx21 )
=ϕ
x21
x11 + tx21

x12
x22

1/2+µ2 k1/2+µ′2

.

We consider the space of C ∞ -smooth functions ϕ on C such that
′

′

ϕ(−t−1 )t−1+µ1 −µ2 k−1+µ1 −µ2
is C ∞ -smooth at 0. These representations form the (nonunitary) principal series.
It is convenient to complexify the Lie algebra of GL2 (C),
gl2 (C)C ≃ gl2 (C) ⊕ gl2 (C).
Under this isomorphism, the operators of the Lie algebra act in our representation by
d
+ (−1/2 + µ1 ),
dt
d
= −t2 + t(−1 + µ1 − µ2 ),
dt
d
= −t + (−1/2 + µ′1 ),
dt
2 d
= −t
+ t(−1 + µ′1 − µ′2 ),
dt

d
,
dt
d
= t + (1/2 + µ2 ),
dt
d
= ,
dt
d
= t + (1/2 + µ′2 ).
dt

L11 = −t

L12 =

(1.8)

L21

L22

(1.9)

L11
L21

L12
L22

Formally, we have duplicated expressions (1.2)–(1.3).
7

(1.10)
(1.11)

If
Re(µ1 + µ′1 ) = 0,

Re(µ2 + µ′2 ) = 0,

(1.12)

2

then the representation Tµ1 ,µ′1 ;µ2 ,µ′2 is unitary in L . Denote by ∆tempered the
set of such tuples (µ1 , µ′1 ; µ2 , µ′2 ), it is a union of a countable family of parallel
2-dimensional real planes in C4 , we equip it by a natural Lebesgue measure
dλ(µ).
For any compactly supported smooth function F on GL2 (C) we define its
Fourier transform as an operator-valued function on ∆ given by
Z
Y
F (X) | det X|−4
Tµ1 ,µ′1 ;µ2 ,µ′2 (F ) =
d Re xkl d Im xkl .
GL2 (C)

k,l=1,2

The Plancherel formula is the following identity
Z
∗
 = −C ·
tr Tµ1 ,µ′1 ;µ2 ,µ′2 (F1 )Tµ1 ,µ′1 ;µ2 ,µ′2 (F2 ) ×
hF1 , F2 i 2
L

GL2 (C)

∆tempered

× (µ1 − µ2 )(µ′1 − µ′2 )dλ(µ),

where C is an explicit constant.
Denote by K(t, s|µ1 , µ′1 ; µ2 , µ′2 ) the kernel of the operator Tµ1 ,µ′1 ;µ2 ,µ′2 (F ),
Z
Tµ1 ,µ′1 ;µ2 ,µ′2 (F )ϕ(t) =
K(t, s|µ1 , µ′1 ; µ2 , µ′2 )ϕ(s) ds.
C

1.7. Overgroup for GL2 (C). Consider the complex Grassmannian Gr4,2 (C)
of 2-dimensional planes in C4 , again the set Mat2 (C) is an open dense set on
′
Gr4,2 (C). For
 σ, σ ∈ iR consider a unitary representation Rσ,σ′ of GL4 (C) in
2
L Mat2 (C) given by




B
Rσ,σ′ F (X) =
F (X) = F (A + XC)−1 (B + XD) ×
D

 1−σk1−σ′
A B
−2+2σk−2+2σ′
× | det(A + XC)|
det
.
C D


We define a unitary operator J : L2 Mat2 (C) → L2 Mat2 (C) by
A
C

′

Jσ,σ′ F (X) = F (X)(det X)−1+σk−1+σ .

−1
In this way we get a unitary representation Uσ,σ′ = Jσ,σ′ Rσ,σ′ Jσ,σ
′ of GL4 (C)

2
in L Mat2 (C) :

Uσkσ′



A
C



B
F (X) = F (A + XC)−1 (B + XD) ×
D

det(A + XC) det(B + XD)


×
A B
det X det
C D
8

−1+σk−1+σ′

.

1.8. Formulas for GL2 (C). We wish to write the action of the Lie algebra
gl4 (C)C ≃ gl4 (C) ⊕ gl4 (C)
in the Plancherel decomposition of GL2 (C). Denote the standard generators of
gl4 (C) ⊕ 0 and 0 ⊕ gl4 (C) by Ekl and E kl respectively. Define the following shift
operators
V1 K(t, s; µ1 , µ′1 , µ2 , µ′2 ) = K(t, s; µ1 + 1, µ′1 , µ2 , µ′2 );
V1′ K(t, s; µ1 , µ′1 , µ2 , µ′2 ) = K(t, s; µ1 , µ′1 + 1, µ2 , µ′2 ),
and similar operators V2 and V2′ shifting µ2 and µ′2 .
Then
−1/2 − σ + µ1 ∂ −1 −1/2 − σ + µ2 ∂ −1
V
V ;
−
µ1 − µ2
∂s 1
µ1 − µ2
∂t 2
−1/2 − σ ′ + µ′2 ∂
−1/2 − σ ′ + µ′1 ∂
′ −1
(V
)
−
(V ′ )−1 ;
=
1
µ′1 − µ′2
∂s
µ′1 − µ′2
∂t 2
1/2 + µ2 + σ ∂
1/2 + µ1 + σ ∂
V1 +
V2 ;
=
µ1 − µ2
∂t
µ1 − µ2
∂s
1/2 + µ′1 + σ ′ ∂ ′ 1/2 + µ′2 + σ ′ ∂ ′
V .
V +
=
µ′1 − µ′2
µ′1 − µ′2
∂s 2
∂t 1

E14 =
E 14
E32
E 32

2

Calculations

2.1. The expression for kernel.
Lemma 2.1 The kernel K(·) of an integral operator Tµ1 ,ε1 ;µ2 ,ε2 (F ) is given by
the formula
K(t, s|µ1 , ε1 ; µ2 , ε2 ) =
ZZZ

F u − tv, su − stv − tw, v, sv + w u−3/2+µ1 //ε1 w−3/2+µ2 //ε2 du dv dw.
=
R3

(2.1)

For F ∈ C0∞ GL2 (R) the integration is actually taken over a bounded domain.

The integral converges if Re µ1 > 1/2, Re µ2 > 1/2. For fixed ε1 , ε2 it has
a meromorphic continuation to the whole complex plane (µ1 , µ2 ) with poles on
the hyperplanes µ1 = −1/2 − k, µ2 = −1/2 − k, where k = 0, 1, 2, . . . (see, e.g.,
[4], §I.3).

9

Proof. By the definition
Z
Tµ1 ,ε1 ;µ2 ,ε2 (F )ϕ(t) ψ(t) dt =
R

=

Z

Z

R Mat2 (R)

 x + tx 
12
22
F (x11 , x12 , x21 , x22 )ϕ
(x11 + tx21 )−1+µ1 −µ2 //ε1 −ε2 ×
x21 + tx12
× (x11 x22 − x12 x21 )1/2+µ2 //ε2

dx11 dx12 dx21 dx22
dt.
(x11 x22 − x12 x21 )2

In the interior integral, we pass from the variables x11 , x12 , x21 , x22 to new
variables u, v, w, s defined by

 



1 −t
x11 x12
u 0
1 s
=
(2.2)
0 1
v w
x21 x22
0 1
or
x11 = u − tv,

x12 = su − stv − tw,

x21 = v,

x22 = sv + w.

The Jacobi matrix of this transformation is triangular, and the Jacobian is |u|.
The inverse transformation is
x11 x22 − x12 x21
x12 + tx22
u = x11 + tx21 , v = x21 , w =
,
s=
.
x11 + tx21
x11 + tx21
We also have x11 x22 − x12 x21 = uw.
After the change of variables we come to
ZZ
K(t, s|µ1 , ε1 ; µ2 , ε2 ) ϕ(s) ψ(t) ds dt,
R2

where K(·) is given by (2.1).
A function F has a compact support in R4 \ {x11 x22 − x12 x21 = 0}. So,
actually, x21 = v, x11 = u − tv, x22 = w + sv are contained in a bounded
domain. This implies the second claim of the lemma.

2.2. Preliminary remarks. Below F denotes
F := F (u − tv, su − stv − tw, v, sv + w).
Also, ∂11 F , ∂12 F , etc. denote
∂11 F :=

∂
F (x11 , x12 , x21 , x22 )
∂x11
x11 = u − tv
x21 = v,

x12 = su − stv − tw,
x22 = sv + w

etc. Partial derivatives of F are
∂
F = ∂11 F + s ∂12 F ;
∂u
∂
F = −t ∂11 F − st ∂12 F + ∂21 F + s ∂22 F ;
∂v
∂
F = −t ∂12 F + ∂22 F,
∂w
10

(2.3)
(2.4)
(2.5)

and
∂
F = (u − vt) ∂12 F + v ∂22 F ;
∂s
∂
F = −v ∂11 F − (w + sv) ∂12 F.
∂t
Also, notice that



y ν//δ

′

(2.6)
(2.7)

= νy ν−1//δ+1 .

2.3. A verification of the formula for E14 . It is easy to verify that the
operator e14 in C0∞ GL2 (R) is given by
e14 =

∂
x21
− (−1 + σ)
.
∂x12
x11 x22 − x12 x21

Therefore,
Tµ1 ,ε1 ;µ2 ,ε2 (e14 F ) =
ZZZ
=
∂12 F u−3/2+µ1 //ε1 w−3/2+µ2 //ε2 du dv dw−
ZZZ
v −3/2+µ1 //ε1 −3/2+µ2 //ε2
− (−1 + σ)
F
du dv dw,
w
u
uw
(the integration is taken over R3 on default).
We must verify that (2.8) coincides with
 −1/2 − σ + µ ∂
−1/2 − σ + µ2 ∂ − 
1
V1− +
V K
E14 K =
µ1 − µ2
∂s
µ1 − µ2
∂t 2
Below we establish two formulas
ZZZ
∂ −
∂12 F u−3/2+µ1 //ε1 w−3/2+µ2 //ε2 du dv dw −
V K=
∂s 1
ZZZ
v −3/2+µ1 //ε1 −3/2+µ2 //ε2
du dv dw,
w
u
= (−3/2 + µ2 )
F
uw
ZZZ

∂ −
∂12 F u−3/2+µ1 //ε1 w−3/2+µ2 //ε2 du dv dw +
V K=
∂t 2
ZZZ
v −3/2+µ1 //ε1 −3/2+µ2 //ε2
du dv dw.
w
u
= (−3/2 + µ1 )
F
uw

1
Considering the sum of (2.10) and (2.11) with coefficients −1/2−σ+µ
and
µ1 −µ2
we get coincidence of (2.8) and (2.9); for this, we use he identities

(2.8)

(2.9)

(2.10)

(2.11)
−1/2−σ+µ2
µ1 −µ2

−1/2 − σ + µ2
−1/2 − σ + µ1
−
= 1;
µ1 − µ2
µ1 − µ2
−1/2 − σ + µ1
−1/2 − σ + µ2
(−3/2 + µ2 ) ·
− (−3/2 + µ1 ) ·
= 1 − σ.
µ1 − µ2
µ1 − µ2
11

Now let us check (2.10). The following identity can be verified by a straightforward calculation (with (2.6) and (2.5)):
∂12 F −

1 ∂
v ∂
F =−
F.
u ∂s
u ∂w

Therefore the left-hand side of (2.10) equals to
ZZZ h
v ∂ i −3/2+µ1 //ε1 −3/2+µ2 //ε2
−
du dv dw.
w
F ·u
u ∂w
We integrate this expression by parts in the variable w and come to (2.10).
To check (2.11), we verify the identity
∂12 F +

1 ∂
v ∂
F =
F
v ∂t
w ∂u

and after this integrate by parts as above.
2.4. A verification of the formula for E32 . We have

∂
∂ 
∂
∂
+ (−1 + σ)x21 .
+ x11 x22
+ x221
+ x21 x22
e32 = − x11 x21
∂x11
∂x12
∂x21
∂x22
Therefore,
T (e32 K) =

ZZZ


G + (−1 + σ)vF · u−3/2+µ1 //ε1 w−3/2+µ2 //ε2 du dv dw, (2.12)

where
G = (u − vt)v ∂11 F + (u − vt)(w + vs) ∂12 F + v 2 ∂21 F + v(w + vs) ∂22 F.
On the other hand,
ZZZ
∂
1/2 + µ1 + σ
u F · u−3/2+µ1 //ε1 w−3/2+µ2 //ε2 du dv dw+
µ1 − µ2
∂t
ZZZ
1/2 + µ2 + σ
∂
+
w F · u−3/2+µ1 //ε1 w−3/2+µ2 //ε2 du dv dw. (2.13)
µ1 − µ2
∂s

E32 K =

We must verify that (2.12) and (2.12) are equal. As in the previous subsection,
this statement is reduced to a pair of identities
ZZZ
∂ 
G + (−1 + σ)vF − u F · u−3/2+µ1 //ε1 w−3/2+µ2 //ε2 du dv dw =
∂t
ZZZ
= (1/2 + µ2 + σ)
vF · u−3/2+µ1 //ε1 w−3/2+µ2 //ε2 du dv dw; (2.14)
ZZZ

∂ 
G + (−1 + σ)vF + w F · u−3/2+µ1 //ε1 w−3/2+µ2 //ε2 du dv dw =
∂s
ZZZ
= (1/2 + µ1 + σ)
vF · u−3/2+µ1 //ε1 w−3/2+µ2 //ε2 du dv dw. (2.15)
12

Let us verify (2.14). It can be easily checked (with (2.7), (2.4), (2.5)) that
G−u

∂
∂
∂
F = −v 2 F − vw
F.
∂t
∂v
∂w

We substitute this to the left-hand side of (2.14) and come to
ZZZ

∂
∂
F + (−1 + σ)vF ×
−v 2 F − vw
∂v
∂w

× u−3/2+µ1 //ε1 w−3/2+µ2 //ε2 du dv dw.

(2.16)

Integrating by parts, we get
ZZZ
∂  2  −3/2+µ1 //ε1 −3/2+µ2 //ε2
du dv dw+
w
v ·u
F·
∂v
ZZZ
∂  −1/2+µ2 //ε2 +1 
+
vF · u−3/2+µ1 //ε1
du dv dw+
w
∂v
ZZZ
+ (−1 + σ)
vF · u−3/2+µ1 //ε1 w−3/2+µ2 //ε2 du dv dw.
After a summation we come to the right-hand side of (2.14).
A proof of (2.15) is similar, we use the identity
G+v

∂
∂
∂
F = −v 2 F − −uv F
∂s
∂v
∂u

and repeat the same steps.
2.5. Table of formulas. First, we present formulas for the action of the
Lie algebra gl4 corresponding to Uσ , see (1.6). Denote generators of gl4 by ekl ,
where 1 6 k, l 6 4. Denote by ∂pq the partial derivatives ∂x∂pq , where p, q = 1, 2.
The generators ekl naturally split into 4 groups corresponding to blocks A, B,
C, D in (1.6).
a) Generators corresponding to the block A form a Lie algebra gl2 :
e11 = −x11 ∂11 − x12 ∂12 ,

e12 = −x21 ∂11 − x22 ∂12 ,

e21 = −x11 ∂21 − x12 ∂22 ,

e22 = −x21 ∂21 − x22 ∂22 .

b) Generators corresponding to the block D also form a Lie algebra gl2 :
e33 = x11 ∂11 + x21 ∂21 ,
e43 = x12 ∂11 + x22 ∂21 ,

e34 = x11 ∂12 + x21 ∂22 ,
e44 = x12 ∂12 + x22 ∂22 .

c) Elements corresponding to the block B form a 4-dimensional Abelian Lie
algebra:
x22
,
det X
x12
= ∂21 − (−1 + σ)
,
det X

x21
,
det X
x11
= ∂22 + (−1 + σ)
.
det X

e13 = ∂11 + (−1 + σ)

e14 = ∂12 − (−1 + σ)

e23

e24

13

c) Elements corresponding to the block C also form a 4-dimensional Abelian
Lie algebra:
e31 = −(x211 ∂11 + x11 x12 ∂12 + x11 x21 ∂21 + x12 x21 ∂22 ) + (−1 + σ)x11 ,
e32 = −(x11 x21 ∂11 + x11 x22 ∂12 + x221 ∂21 + x21 x22 ∂22 ) + (−1 + σ)x21 ,
e41 = −(x11 x12 ∂11 + x212 ∂12 + x11 x22 ∂21 + x12 x22 ∂22 ) + (−1 + σ)x12 ,
e42 = −(x12 x21 ∂11 + x12 x22 ∂12 + x21 x22 ∂21 + x222 ∂22 ) + (−1 + σ)x22 .
Denote by Ekl the corresponding operators Ekl on kernels K. Formulas for
operators of groups a), b) immediately follow from the definition of the Fourier
transform,
∂
− (1/2 − µ1 ),
∂t
∂
= −t2 + (−1 + µ1 − µ2 )t,
∂t

∂
,
∂t
∂
= t + (1/2 + µ2 ),
∂t

E11 = −t

E12 =

E21

E22

and
∂
− (1/2 + µ1 ),
∂s
∂
+ (−1 − µ1 + µ2 )s,
= −s2
∂s

∂
,
∂s
∂
=s
+ (1/2 − µ2 ).
∂s

E33 = −s

E34 =

E43

E44

Next,
∂  −
1/2 − µ2 + σ ∂ − 1/2 − µ1 + σ 
V ,
µ1 − µ2 + s
s V2 +
µ1 − µ2
∂t
µ1 − µ2
∂s 1
1/2 + σ − µ1 ∂ − 1/2 + σ − µ2 ∂ −
=−
V −
V ,
µ1 − µ2
∂s 1
µ1 − µ2
∂t 2
1/2 − µ2 + σ 
∂ −
:=
V +
s −(µ1 − µ2 ) + t
µ1 − µ2
∂t 2
∂  −
1/2 − µ1 + σ 
V ,
t µ1 − µ2 + s
µ1 − µ2
∂s 1
1/2 + σ − µ1 ∂ − 1/2 + µ2 − σ 
∂  −
=−
V .
µ1 − µ2 + t
t
V1 +
µ1 − µ2
∂s
µ1 − µ2
∂s 2

E13 :=
E14
E23

E24
and

∂  + 1/2 + µ2 + σ ∂ +
1/2 + µ1 + σ 
V −
µ1 − µ2 − t
t V2 ,
µ1 − µ2
∂t 1
µ1 − µ2
∂s
1/2 + µ1 + σ ∂ + 1/2 + µ2 + σ ∂ +
V +
V ,
=
µ1 − µ2 ∂t 1
µ1 − µ2 ∂s 2
∂  + 1/2 + µ2 + σ 
∂  +
1/2 + µ1 + σ 
V1 −
V ,
s µ1 − µ2 − t
t µ1 − µ2 + s
=
µ1 − µ2
∂t
µ1 − µ2
∂s 2
1/2 + µ1 + σ ∂ + 1/2 + µ2 + σ 
∂  +
=
V .
µ1 − µ2 + s
s V1 +
µ1 − µ2
∂t
µ1 − µ2
∂s 2

E31 :=
E32
E41
E42

14

2.6. The case of GL2 (C). Notice that formulas in Subsections 1.1–1.4
for SL2 (R) and in Subsections 1.6–1.8 are very similar, except the Plancherel
formulas.
The analog of formula (2.1) is
K(t, s|µ1 , µ′1 ; µ2 , µ′2 ) =
ZZZ

′
′
F u − tv, su − stv − tw, v, sv + w u−3/2+µ1 k−3/2+µ1 w−3/2+µ2 k−3/2+µ2 ×
=
C3

× d Re u d Im u d Re v d Im v d Re w d Im w.

(2.17)

Its derivation is based on the same change of variables (2.2), its real Jacobian
is uu. A further calculation one-to-one follows the calculation for GL2 (R).

References
[1] Andrews G. R., Askey R., Roy R., Special functions, Cambridge Univ.
Press, Cambridge, 1999
[2] Cherednik I., Inverse Harish-Chandra transform and difference operators.
Int. Math. Res. Not. 1997, No.15, 733-750 (1997).
[3]
[4] Gelfand, I. M.; Shilov, G. E. Generalized functions. Vol. 1: Properties and
operations. Providence, RI: AMS Chelsea Publishing (2016).
[5] Gelfand, I. M.; Graev, M. I.; Vilenkin, N. Ya. Generalized functions. Vol. 5:
Integral geometry and representation theory. Providence, RI: AMS Chelsea
Publishing, 2016.
[6] Groenevelt, W. The Wilson function transform. Int. Math. Res. Not. 2003,
no. 52, 2779-2817.
[7] Howe, R.; Tan, Eng Chye Non-Abelian harmonic analysis. Applications of
SL(2, R). Springer-Verlag, (1992).
[8] Koekoek R., Swarttouw R. F. The Askey-scheme of hypergeometric orthogonal polynomials and its q-analogue. Delft University of Technology
Faculty of Information Technology and Systems. Department of Technical Mathematics and Informatics Report no. 98-17 1998. Available via
http://homepage.tudelft.nl/11r49/askey.html
[9] Makarevich, B. O. Open symmetric orbits of reductive groups in symmetric
R-spaces, Math. USSR-Sb., 20:3 (1973), 406-418
[10] Molchanov, V. F. Canonical representations and overgroups for hyperboloids of one sheet and Lobachevsky spaces. Acta Appl. Math. 86 (2005),
no. 1-2, 115-129.
15

[11] Molchanov, V. F. Canonical representations and overgroups for hyperboloids. Funct. Anal. Appl. 39 (2005), no. 4, 284-295
[12] Molchanov, V. F. Canonical representations on Lobachevsky spaces: an
interaction with an overalgebra. Acta Appl. Math. 99 (2007), no. 3, 321337.
[13] Molchanov, V. F. Fourier and Poisson transforms for tensor products.
Functional Analysis and Its Applications, 2015, 49:4, 279-288.
[14] Naimark M. A. Linear representations of Lorentz group. Pergamon Press,
1964.
[15] Neretin, Yu. A. Pseudo-Riemannian symmetric spaces: uniform realizations and open embeddings into Grassmannians, Zap. Nauchn. Sem. POMI,
256, POMI, St. Petersburg, 1999, 145-167; J. of Math. Sciences (New York),
2001, 107:5, 4248-4264.
[16] Neretin, Yu. A. Index hypergeometric transform and imitation of analysis of
Berezin kernels on hyperbolic spaces, Sbornik Math., 192:3 (2001), 403–432
[17] Neretin, Yu. A. The action of an overalgebra in the Plancherel decomposition and shift operators in an imaginary direction. Izv. Math. 66 (2002),
no. 5, 1035-1046.
[18] Neretin, Yu. A. Lectures on Gaussian integral operators and classical
groups. European Mathematical Society (EMS), Zürich, 2011.
[19] Neretin, Yu. A. Difference Sturm-Liouville problems in the imaginary direction. J. Spectr. Theory 3 (2013), no. 3, 237-269.
[20] Neretin, Yu. A. Restriction of representations of GL(n + 1, C) to GL(n, C)
and action of the Lie overalgebra. Preprint, arXiv:1510.03611
Math. Dept., University of Vienna;
Institute for Theoretical and Experimental Physics (Moscow);
MechMath Dept., Moscow State University;
Institute for Information Transmission Problems.
URL: http://mat.univie.ac.at/∼neretin/

16

