# Shintani Functions onSL(3,R)

Authors: Keiju Sono
Journal: International Journal of Mathematics and Mathematical Sciences
 (2011)

---

## Abstract

We investigate the Shintani functions attached to the spherical and nonspherical principal series representations ofSL(3,R). We give the explicit formulas of the radial part of Shintani functions and evaluate
the dimension of the space of Shintani functions.


---

## Body

## 1. Introduction

Shintani function is originally introduced by Shintani forp-adic linear group GL(n,k), where k is a finite extension of the p-adic field Qp [1]. He defined some “Whittaker function” on GL(n,k) and obtained the explicit formulas of them. Moreover, he proved the uniqueness of his function. Later, a more detail study of Shintani functions for GL(n) was done by Murase and Sugano [2] (see also [3]). They obtained new kinds of integral formulas for the L-functions in terms of the global Shintani functions and proved the multiplicity one theorem of the local one at the finite primes.On the other hand, the multiplicity and explicit formulas of the Archimedean Shintani functions were more recently investigated by some mathematicians. For example, Hirano studied the Shintani functions onGL(2,R) [4] and GL(2,C) [5], Tsuzuki on SU(1,1) [6] and U(n,1) [7], and Moriyama on Sp(2,R) [8, 9]. They constructed the differential equations satisfied by the radial part of the Shintani functions and obtained the explicit formulas by solving them. Most of them are expressed by some linear combinations of the Gaussian hypergeometric functions. Moreover, the dimensions of the spaces of Shintani functions are obtained, which are sometimes bigger than 1.In this paper, we investigate the Shintani functions onG=SL(3,R), attached to the principal series representations of G. Now we explain the definition of the Shintani functions on G. We take(1.1)H={(H100h2)∈G∣(H1,h2)∈GL(2,R)×GL(1,R)}
as a subgroup of G and take K=SO(3) as a maximal compact subgroup of G. Let π be an arbitrary irreducible unitary representation of G and (η,Vη) an irreducible unitary representation of H, and let Cη∞(H∖G) be the space of smooth functions F:G→Vη satisfying F(hg)=η(h)F(g)((h,g)∈H×G). We consider the intertwining space Iη,π=Hom(𝔤C,K)(π,Cη∞(H∖G)) and its restriction(1.2)Iη,π⟶HomK(τ,Cη∞(H∖G))≅Cη,τ*(L)
to the minimal K-type (τ,Vτ) of π, where (τ*,Vτ*) is the contragredient representation of (τ,Vτ), L≔H∖G/K and Cη,τ*∞(L) is the space of smooth functions F:G→Vη⊗Vτ* satisfying F(hgk)=(η(h)⊗τ*(k-1))F(g) for (h,g,k)∈H×G×K. The function which belongs to the image of above map is called the Shintani function. In this paper, we assume that π is the irreducible unitary principal series representation of G and η is the unitary character of H. The study of Shintani functions for the general unitary representation η of H is a further problem.In Section4, we investigate the Shintani functions attached to the spherical (or class one) principal series representations. These representations have unique K-fixed vector, and hence the minimal K-type is one-dimensional. In this case, the explicit formulas of Shintani functions are obtained by solving two Casimir equations which are characterized by the action of the center of universal enveloping algebra. We also obtain the necessary condition of the existence of nonzero Shintani functions and prove that the dimension of the space of Shintani functions is equal to or less than 1 (Theorem 4.8).On the other hand, in Section5, we investigate the Shintani functions attached to the nonspherical principal series representations, whose minimal K-type is three-dimensional representation of K. In this case, we construct two kinds of differential equations. One is the Casimir equation we used in Section 4, and the other is the gradient equation. The key point is as follows. We have three different nonspherical principal series with the same infinitesimal characters Z(𝔤)→C. We cannot distinguish them only by the elements of Z(𝔤). This is the reason we need the gradient operator which has distinct eigenvalues for different nonspherical principal series. By combining these equations, we obtain the explicit formulas of the Shintani functions, the necessary condition of the existence of nonzero Shintani functions, and prove that the dimension of the space of Shintani functions is equal to or less than 1 (Theorem 5.7).As an application of the results of this paper, our explicit formulas for Shintani functions will be useful to compute the local zeta integral in the theory of Murase and Sugano ([2, 3]; (see also [10], in the case of U(n,1)). Furthermore, the author thinks these results are interesting themselves in view of the harmonic analysis on homogeneous spaces.


## 2. Preliminaries

### 2.1. Groups and Algebras

LetG be the real reductive Lie group SL(3,R) and 𝔤=𝔰𝔩(3,R) its Lie algebra. The Cartan involution θ:G→G is defined by θ(g)=(gt)-1(g∈G), and its differential dθ:𝔤→𝔤 is given by dθ(X)=-Xt(X∈𝔤), where t means the transposition of matrices. Then the fixed subgroup of θ in G is equal to K=SO(3), which is the maximal compact subgroup of G. Next, we define another involutive automorphism σ of G by σ(g)=JgJ(g∈G), where J=diag(-1,-1,1). Its differential dσ:𝔤→𝔤 is given by dσ(X)=JXJ(X∈𝔤). The fixed subgroup H of σ in G is isomorphic to GL(2,R), that is,  (2.1)H={(a11a120a21a22000a33)∈G}≅GL(2,R).
We define +1 and −1 eigenspaces of dθ, dσ in 𝔤 by (2.2)k={X∈g∣dθ(X)=X},p={X∈g∣dθ(X)=-X},h={X∈g∣dσ(X)=X},q={X∈g∣dσ(X)=-X}.
Then, 𝔨,𝔥 are the Lie algebras of K,H, respectively. We have 𝔤=𝔨⊕𝔭=𝔥⊕𝔮. Let Eij∈M(3,R) be the matrix whose (i,j)-component is 1 and the other components are 0 (1≤i,j≤3). For 1≤i<j≤3, we put Kij=Eij-Eji,Xij=Eij+Eji,Hij=Eii-Ejj. Then, we have(2.3)k=⨁i<jRKij,p=⨁i<jRXij⊕RH12⊕RH23.
Next, we take(2.4)Y1=(10000000-1),Y2=(010100000),Y3=(0-10100000),Y4=(00001000-1)
as a basis of 𝔥. We have 𝔭∩𝔮=RX13⊕RX23, and we take 𝔞=RX13 as a maximal Abelian subspace of 𝔭∩𝔮. We define a subgroup A of G by(2.5)A=exp⁡(a)={at:=(cosh⁡⁡t0sinh⁡⁡t010sinh⁡⁡t0cosh⁡⁡t)∣t∈R}.
Then, G has a decomposition G=HAK. Throughout this paper, we put L≔H∖G/K.For a Lie algebra𝔩, we denote its complexification by 𝔩C, that is, 𝔩C=𝔩⊗RC.


### 2.2. The Principal Series Representations

As a representation ofG, we take the principal series representation defined as follows. Let P0 be a minimal parabolic subgroup of G given by the upper triangular matrices in G and P0=MAP0N the Langlands decomposition of P0 with(2.6)AP0={diag⁡(a1,a2,a3)∣ai>0,a1a2a3=1},M=K∩{diagonalsinG},N={(1x1x201x3001)∈G∣x1,x2,x3∈R}.
To define a principal series representation with respect to the minimal parabolic subgroup P0 of G, we firstly fix a character σ of M and a linear form ν∈𝔞P0*⊗RC=HomR(𝔞P0,C), where 𝔞P0 is the Lie algebra of AP0. We write(2.7)ν(diag⁡(t1,t2,t3))=ν1t1+ν2t2
for diag(t1,t2,t3)∈𝔞P0. Then, we can define a representation σ⊗aν of MAP0 and extend this to P0 by the identification P0/N≃MAP0, taking the trivial representation 1N as the representation of N. Then, the induced representation(2.8)πσ,ν:=C∞IndP0G(σ⊗aν+ρ⊗1N)
is called the principal series representation of G. Here, ρ is the half sum of positive roots of (𝔤,𝔞) given by aρ=a12a2, for a=diag(a1,a2,a3)∈AP0.Concretely, the representation space is given by(2.9)C(M,σ)∞(K):={f∈C∞(K)∣f(mk)=σ(m)f(k),m∈M,k∈K},
and the action of G is defined by(2.10)(πσ,ν(x)f)(k)=a(kx)ν+ρf(κ(kx))(x∈G,k∈K).
Here, for g∈G,g=n(g)a(g)κ(g)(n(g)∈N,a(g)∈AP0,κ(g)∈K) is the Iwasawa decomposition. Throughout this paper, we assume that the representation πσ,ν is irreducible. Moreover, we assume that ν1,ν2 are the elements of -1R. Then, this representation becomes unitary.Next, we define charactersσj(j=0,1,2,3) of M as follows. The group M consisting of four elements is a finite Abelian group of (2,2)-type, and its elements except for the unity are given by(2.11)m1=diag⁡(1,-1,-1),m2=diag⁡(-1,1,-1),m3=diag⁡(-1,-1,1).
The set M̂ consists of 4 characters {σj∣j=0,1,2,3}, where σ0 is the trivial character of M and σ1,σ2,σ3 are defined by Table 1.Table 1m1m2m3σ11−1−1σ2−11−1σ3−1−11The following proposition (see [11, Proposition 1.1]) gives the correspondence between the character σ of M and the minimal K-type of the principal series representation πσ,ν of G.Proposition 2.1.
(1) Ifσ is the trivial character of M, the representation πσ,ν is spherical or class one. That is, it has a unique K-invariant vector in Hσ,ν.
(2) Ifσ is not trivial, the minimal K-type of the restriction πσ,ν|K to K is a 3-dimensional representation of K, which is isomorphic to the unique standard one (τ2,V2). The multiplicity of this minimal K-type is one:
(2.12)dim⁡CHomK(τ2,Hσ,ν)=1.


## 2.1. Groups and Algebras

LetG be the real reductive Lie group SL(3,R) and 𝔤=𝔰𝔩(3,R) its Lie algebra. The Cartan involution θ:G→G is defined by θ(g)=(gt)-1(g∈G), and its differential dθ:𝔤→𝔤 is given by dθ(X)=-Xt(X∈𝔤), where t means the transposition of matrices. Then the fixed subgroup of θ in G is equal to K=SO(3), which is the maximal compact subgroup of G. Next, we define another involutive automorphism σ of G by σ(g)=JgJ(g∈G), where J=diag(-1,-1,1). Its differential dσ:𝔤→𝔤 is given by dσ(X)=JXJ(X∈𝔤). The fixed subgroup H of σ in G is isomorphic to GL(2,R), that is,  (2.1)H={(a11a120a21a22000a33)∈G}≅GL(2,R).
We define +1 and −1 eigenspaces of dθ, dσ in 𝔤 by (2.2)k={X∈g∣dθ(X)=X},p={X∈g∣dθ(X)=-X},h={X∈g∣dσ(X)=X},q={X∈g∣dσ(X)=-X}.
Then, 𝔨,𝔥 are the Lie algebras of K,H, respectively. We have 𝔤=𝔨⊕𝔭=𝔥⊕𝔮. Let Eij∈M(3,R) be the matrix whose (i,j)-component is 1 and the other components are 0 (1≤i,j≤3). For 1≤i<j≤3, we put Kij=Eij-Eji,Xij=Eij+Eji,Hij=Eii-Ejj. Then, we have(2.3)k=⨁i<jRKij,p=⨁i<jRXij⊕RH12⊕RH23.
Next, we take(2.4)Y1=(10000000-1),Y2=(010100000),Y3=(0-10100000),Y4=(00001000-1)
as a basis of 𝔥. We have 𝔭∩𝔮=RX13⊕RX23, and we take 𝔞=RX13 as a maximal Abelian subspace of 𝔭∩𝔮. We define a subgroup A of G by(2.5)A=exp⁡(a)={at:=(cosh⁡⁡t0sinh⁡⁡t010sinh⁡⁡t0cosh⁡⁡t)∣t∈R}.
Then, G has a decomposition G=HAK. Throughout this paper, we put L≔H∖G/K.For a Lie algebra𝔩, we denote its complexification by 𝔩C, that is, 𝔩C=𝔩⊗RC.


## 2.2. The Principal Series Representations

As a representation ofG, we take the principal series representation defined as follows. Let P0 be a minimal parabolic subgroup of G given by the upper triangular matrices in G and P0=MAP0N the Langlands decomposition of P0 with(2.6)AP0={diag⁡(a1,a2,a3)∣ai>0,a1a2a3=1},M=K∩{diagonalsinG},N={(1x1x201x3001)∈G∣x1,x2,x3∈R}.
To define a principal series representation with respect to the minimal parabolic subgroup P0 of G, we firstly fix a character σ of M and a linear form ν∈𝔞P0*⊗RC=HomR(𝔞P0,C), where 𝔞P0 is the Lie algebra of AP0. We write(2.7)ν(diag⁡(t1,t2,t3))=ν1t1+ν2t2
for diag(t1,t2,t3)∈𝔞P0. Then, we can define a representation σ⊗aν of MAP0 and extend this to P0 by the identification P0/N≃MAP0, taking the trivial representation 1N as the representation of N. Then, the induced representation(2.8)πσ,ν:=C∞IndP0G(σ⊗aν+ρ⊗1N)
is called the principal series representation of G. Here, ρ is the half sum of positive roots of (𝔤,𝔞) given by aρ=a12a2, for a=diag(a1,a2,a3)∈AP0.Concretely, the representation space is given by(2.9)C(M,σ)∞(K):={f∈C∞(K)∣f(mk)=σ(m)f(k),m∈M,k∈K},
and the action of G is defined by(2.10)(πσ,ν(x)f)(k)=a(kx)ν+ρf(κ(kx))(x∈G,k∈K).
Here, for g∈G,g=n(g)a(g)κ(g)(n(g)∈N,a(g)∈AP0,κ(g)∈K) is the Iwasawa decomposition. Throughout this paper, we assume that the representation πσ,ν is irreducible. Moreover, we assume that ν1,ν2 are the elements of -1R. Then, this representation becomes unitary.Next, we define charactersσj(j=0,1,2,3) of M as follows. The group M consisting of four elements is a finite Abelian group of (2,2)-type, and its elements except for the unity are given by(2.11)m1=diag⁡(1,-1,-1),m2=diag⁡(-1,1,-1),m3=diag⁡(-1,-1,1).
The set M̂ consists of 4 characters {σj∣j=0,1,2,3}, where σ0 is the trivial character of M and σ1,σ2,σ3 are defined by Table 1.Table 1m1m2m3σ11−1−1σ2−11−1σ3−1−11The following proposition (see [11, Proposition 1.1]) gives the correspondence between the character σ of M and the minimal K-type of the principal series representation πσ,ν of G.Proposition 2.1.
(1) Ifσ is the trivial character of M, the representation πσ,ν is spherical or class one. That is, it has a unique K-invariant vector in Hσ,ν.
(2) Ifσ is not trivial, the minimal K-type of the restriction πσ,ν|K to K is a 3-dimensional representation of K, which is isomorphic to the unique standard one (τ2,V2). The multiplicity of this minimal K-type is one:
(2.12)dim⁡CHomK(τ2,Hσ,ν)=1.


## 3. The Space of Shintani Functions

### 3.1. The Definition of Shintani Functions

As a representation ofH, we take the unitary character η=ηs,k:H→C×(s∈-1R,k∈{0,1}) defined by (3.1)η((h11h120h21h22000h1))=det⁡(H1)k|det⁡(H1)|s-k,H1=(h11h12h21h22)∈GL(2,R).
Let η=ηs,k be the unitary character of H defined previously. We consider the induced representation C∞IndHG(η) with the representation space (3.2)Cη∞(H∖G)={F∈C∞(G)∣F(hg)=η(h)F(g),∀h∈H,∀g∈G}.
G acts on this space by right translation. Let πσ,ν be the principal series representation of G. We consider the intertwining space (3.3)Iη,πσ,ν:=Hom(gC,K)(πσ,ν,Cη∞(H∖G)).
We denote its image by Sη,πσ,ν, that is, (3.4)Sη,πσ,ν:=⋃T∈Iη,πσ,νImage(T).
We call the element of Sη,πσ,ν the Shintani function of type (η,πσ,ν). Let (τ,Vτ) be the K-type of the principal series representation πσ,ν, and let ι:τ→πσ,ν be the K-embedding of τ and ι* the pullback via ι. Then, the map (3.5)ι*:Iη,πσ,ν⟶HomK(τ,Cη∞(H∖G))≅Cη,τ*∞(L)
gives the restriction of T∈Sη,πσ,ν to τ, where τ* is the contragredient representation of τ and the space Cη,τ*∞(L) is defined by (3.6)Cη,τ*∞(L)={F:G⟶Vτ*∣F(hgk-1)=η(h)τ*(k)F(g),∀(h,g,k)∈H×G×K}.
We denote the image of Iη,πσ,ν in Cη,τ*∞(L) by Cη,τ*∞(L)πσ,ν, and the element of this space is called the Shintani function of type (πσ,ν,η,τ).


### 3.2. TheA-Radial Part

BecauseG has the decomposition G=HAK, the element of Cη,τ∞(L) is characterized by its restriction to A. We denote the centralizer and the normalizer of A in K∩H by ZK∩H(A), NK∩H(A), respectively. It is easy to verify that K∩H, ZK∩H(A), and NK∩H(A) are given as follows.Lemma 3.1.
One has(3.7)K∩H={(cos⁡⁡θsin⁡⁡θ0-sin⁡⁡θcos⁡⁡θ0001)∣θ∈R}⨆{(cos⁡⁡θsin⁡⁡θ0sin⁡⁡θ-cos⁡⁡θ000-1)∣θ∈R},ZK∩H(A)={I,(-10001000-1)},NK∩H(A)={I,(-1000-10001),(-10001000-1),(1000-1000-1)}.
Here, I is the unit element of GL(3,R).Letw0:=diag(-1,-1,1). Then, w0ZK∩H(A) is the unique nontrivial element of W=NK∩H(A)/ZK∩H(A). Let η:H→C be the unitary character of H and (τ,Vτ) the finite-dimensional representation of K. We denote by CW∞(A;η,τ) the space of smooth functions F:A→Vτ satisfying the following conditions: (3.8)(1)η(m)τ(m)F(a)=F(a)(∀m∈ZK∩H(A),∀a∈A),(2)η(w0)τ(w0)F(a)=F(a-1)(∀a∈A),(3)η(l)τ(l)F(1)=F(1)(∀l∈K∩H).
The following lemma is proved by Flensted and Jensen (see [12, Theorem 4.1]).Lemma 3.2.
The restriction toA gives the following isomorphism:
(3.9)Cη,τ∞(L)≅CW∞(A;η,τ).Through this isomorphism, we denote the image ofCη,τ∞(L)πσ,ν in CW∞(A;η,τ) by CW∞(A;η,τ)πσ,ν. This is our target space in this paper. The following two lemmas are obvious.Lemma 3.3.
Fort≠0, one has
(3.10)g=Ad(at-1)h⊕a⊕k.Lemma 3.4.
LetF∈Cη,τ∞(L). For X∈𝔨, Y∈𝔥, Z∈𝔞, one has
(3.11)R((Ad(at-1)Y)ZX)F(at)=η(Y)τ(-X)(Zf)(at).


## 3.1. The Definition of Shintani Functions

As a representation ofH, we take the unitary character η=ηs,k:H→C×(s∈-1R,k∈{0,1}) defined by (3.1)η((h11h120h21h22000h1))=det⁡(H1)k|det⁡(H1)|s-k,H1=(h11h12h21h22)∈GL(2,R).
Let η=ηs,k be the unitary character of H defined previously. We consider the induced representation C∞IndHG(η) with the representation space (3.2)Cη∞(H∖G)={F∈C∞(G)∣F(hg)=η(h)F(g),∀h∈H,∀g∈G}.
G acts on this space by right translation. Let πσ,ν be the principal series representation of G. We consider the intertwining space (3.3)Iη,πσ,ν:=Hom(gC,K)(πσ,ν,Cη∞(H∖G)).
We denote its image by Sη,πσ,ν, that is, (3.4)Sη,πσ,ν:=⋃T∈Iη,πσ,νImage(T).
We call the element of Sη,πσ,ν the Shintani function of type (η,πσ,ν). Let (τ,Vτ) be the K-type of the principal series representation πσ,ν, and let ι:τ→πσ,ν be the K-embedding of τ and ι* the pullback via ι. Then, the map (3.5)ι*:Iη,πσ,ν⟶HomK(τ,Cη∞(H∖G))≅Cη,τ*∞(L)
gives the restriction of T∈Sη,πσ,ν to τ, where τ* is the contragredient representation of τ and the space Cη,τ*∞(L) is defined by (3.6)Cη,τ*∞(L)={F:G⟶Vτ*∣F(hgk-1)=η(h)τ*(k)F(g),∀(h,g,k)∈H×G×K}.
We denote the image of Iη,πσ,ν in Cη,τ*∞(L) by Cη,τ*∞(L)πσ,ν, and the element of this space is called the Shintani function of type (πσ,ν,η,τ).


## 3.2. TheA-Radial Part

BecauseG has the decomposition G=HAK, the element of Cη,τ∞(L) is characterized by its restriction to A. We denote the centralizer and the normalizer of A in K∩H by ZK∩H(A), NK∩H(A), respectively. It is easy to verify that K∩H, ZK∩H(A), and NK∩H(A) are given as follows.Lemma 3.1.
One has(3.7)K∩H={(cos⁡⁡θsin⁡⁡θ0-sin⁡⁡θcos⁡⁡θ0001)∣θ∈R}⨆{(cos⁡⁡θsin⁡⁡θ0sin⁡⁡θ-cos⁡⁡θ000-1)∣θ∈R},ZK∩H(A)={I,(-10001000-1)},NK∩H(A)={I,(-1000-10001),(-10001000-1),(1000-1000-1)}.
Here, I is the unit element of GL(3,R).Letw0:=diag(-1,-1,1). Then, w0ZK∩H(A) is the unique nontrivial element of W=NK∩H(A)/ZK∩H(A). Let η:H→C be the unitary character of H and (τ,Vτ) the finite-dimensional representation of K. We denote by CW∞(A;η,τ) the space of smooth functions F:A→Vτ satisfying the following conditions: (3.8)(1)η(m)τ(m)F(a)=F(a)(∀m∈ZK∩H(A),∀a∈A),(2)η(w0)τ(w0)F(a)=F(a-1)(∀a∈A),(3)η(l)τ(l)F(1)=F(1)(∀l∈K∩H).
The following lemma is proved by Flensted and Jensen (see [12, Theorem 4.1]).Lemma 3.2.
The restriction toA gives the following isomorphism:
(3.9)Cη,τ∞(L)≅CW∞(A;η,τ).Through this isomorphism, we denote the image ofCη,τ∞(L)πσ,ν in CW∞(A;η,τ) by CW∞(A;η,τ)πσ,ν. This is our target space in this paper. The following two lemmas are obvious.Lemma 3.3.
Fort≠0, one has
(3.10)g=Ad(at-1)h⊕a⊕k.Lemma 3.4.
LetF∈Cη,τ∞(L). For X∈𝔨, Y∈𝔥, Z∈𝔞, one has
(3.11)R((Ad(at-1)Y)ZX)F(at)=η(Y)τ(-X)(Zf)(at).


## 4. Shintani Functions Attached to the Spherical Principal Series Representations

Throughout this section, as a character ofM, we take the trivial character σ=σ0. Then, the principal series representation πσ0,ν is the spherical or class one principal series representation whose minimal K-type is the one-dimensional trivial representation (1,V1) of K, which occurs of multiplicity one in πσ0,ν|K (Proposition 2.1).


### 4.1. The Capelli Elements

LetZ(𝔤) be the center of the universal enveloping algebra U(𝔤) of 𝔤. Z(𝔤) has two independent generators, and they are obtained as the Capelli elements because 𝔤=𝔰𝔩3 is of type A2 (see [13]). For i=1,2,3, we put (4.1)Eii′=Eii-13(∑k=13Ekk).
The following proposition gives the explicit description of the independent generators of Z(𝔤) (see [11]).Proposition 4.1.
The independent generators{Cp2,Cp3} of Z(𝔤) are given as follows:
(4.2)Cp2=(E11′-1)E22′+E22′(E33′+1)+(E11′-1)(E33′+1)-E23E32-E13E31-E12E21,Cp3=(E11′-1)E22′(E33′+1)+E12E23E31+E13E21E32-(E11′-1)E23E32-E13E22′E31-E12E21(E33′+1).SinceCp2,Cp3 are the elements of Z(𝔤), they act on πσ0,ν as the scalar operators. And since the space of Shintani functions is the image of the (𝔤C,K)-homomorphism of πσ0,ν, they act on the space of Shintani functions as the same scalar operators, respectively.


### 4.2. Eigenvalues ofCp2, Cp3

In order to construct the partial differential equations satisfied by spherical functions attached to the spherical principal series, we have to compute the eigenvalues of the actions of the Capelli elementsCp2, Cp3. For the spherical principal series representation, σ=σ0 is the trivial character of M. Let f0 be the generator of the minimal K-type in Hσ0,ν normalized such that f0∣K≡1. The actions of Cp2,Cp3 on f0 are computed in [11], and the result is as follows.Proposition 4.2.
The Capelli elementsCp2,Cp3 act on f0 by scalar multiples, and the eigenvalues are given as follows:
(4.3)Cp2f0=-13(ν12-ν1ν2+ν22)f0,Cp3f0=-127(2ν1-ν2)(2ν2-ν1)(ν1+ν2)f0.


### 4.3. Construction of the Casimir Equations

Next, we compute the actions ofCp2, Cp3 on F(at)∈Cη,1∞(L)|A. Here, η=ηs,k is the unitary character of H.Lemma 4.3.
ForF(at)∈Cη,τ∞(L)|A, one has
(4.4)R(Ad(at-1)Yi)F(at)=sF(at)(i=1,4),R(Ad(at-1)Yi)F(at)=0(i=2,3),R(X13)F(at)=dFdt(at).Proof.
By definition, we have(4.5)R(Ad(at-1)Yi)F(at)=dduF(atexp⁡(uAd(at-1)Yi))|u=0=dduF(exp⁡(uYi)at)|u=0=dduη(exp⁡(uYi))|u=0F(at).
Since exp(uY1)=diag(eu,1,e-u), we have η((exp(uY1))=eku·e(s-k)u=esu.
Therefore, we have(4.6)R(Ad(at-1)Y1)F(at)=sF(at).
Next, since
(4.7)exp⁡⁡(uY2)=(cosh⁡⁡usinh⁡⁡u0sinh⁡⁡ucosh⁡⁡u0001),
η(exp(uY2))=1k·|1|s-k=1. Therefore, we have
(4.8)R(Ad(at-1)Y2)F(at)=0.
The computations of the actions of Ad(at-1)(Yi)(i=3,4) are similar. Finally, since exp(uX13)=au, we have
(4.9)R(X13)F(at)=dduF(at+u)|u=0=dFdt(at).By simple computations of matrices, we have the following expressions of the elements inM(3,R).Lemma 4.4.
One has(4.10)E11′=cosh⁡2t+13cosh⁡⁡(2t)Ad(at-1)Y1-13Ad(at-1)Y4-12tanh⁡(2t)K13,E22′=13(-H12+H23),E33′=sinh⁡2t-13cosh⁡⁡(2t)Ad(at-1)Y1-13Ad(at-1)Y4+12tanh⁡(2t)K13,E23=12sinh⁡⁡tAd(at-1)Y3+12tanhtK12+12K23,E13=12X13+12K13,E12=12cosh⁡⁡tAd(at-1)Y2-12tanh⁡tK23+12K12.To make use of Lemma3.4, we have to rewrite Cp2, Cp3 in the form of linear combinations of the elements in (Ad(at-1)𝔥)𝔞𝔨. To do this, we use the following formulas which can be obtained by direct computation.Lemma 4.5.
One has(4.11)[K13,Ad(at-1)Y1]=-2cosh⁡⁡(2t)X13,[K13,Ad(at-1)Y4]=-cosh⁡⁡(2t)X13,[K12,Ad(at-1)Y3]=sinh⁡⁡tX13,[K23,Ad(at-1)Y3]=-2sinh⁡3tcosh⁡⁡(2t)Ad(at-1)Y1+2sinh⁡⁡tAd(at-1)Y4+cosh⁡⁡tcosh⁡⁡(2t)K13,[K13,X13]=2cosh⁡⁡(2t)Ad(at-1)Y1-2tanh⁡(2t)K13,[K23,Ad(at-1)Y2]=-cosh⁡⁡tX13,[K12,Ad(at-1)Y2]=2cosh⁡3tcosh⁡⁡(2t)Ad(at-1)Y1-2cosh⁡⁡tAd(at-1)Y4-sinh⁡⁡tcosh⁡⁡(2t)K13,[K12,X13]=-1sinh⁡⁡tAd(at-1)Y3-1tanh⁡tK12,[K23,X13]=1cosh⁡⁡tAd(at-1)Y2-tanh⁡tK23,[X13,Ad(at-1)Y2]=-tanh⁡tAd(at-1)Y2-1cosh⁡⁡tK23,[K13,Ad(at-1)Y2]=-1tanh⁡tAd(at-1)Y3-cosh⁡⁡(2t)sinh⁡⁡tK12,[K13,Ad(at-1)Y3]=tanh⁡tAd(at-1)Y2-cosh⁡⁡(2t)cosh⁡⁡tK23,[X13,Ad(at-1)Y1]=-2tanh⁡(2t)Ad(at-1)Y1-2cosh⁡⁡(2t)K13,[X13,Ad(at-1)Y4]=-tanh⁡(2t)Ad(at-1)Y1-1cosh⁡⁡(2t)K13,[K12,Ad(at-1)Y1]=-cosh⁡⁡(2t)cosh⁡⁡tAd(at-1)Y2-tanh⁡tK23,[K12,Ad(at-1)Y4]=1-sinh⁡2tcosh⁡⁡tAd(at-1)Y2-2tanh⁡tK23,[K23,Ad(at-1)Y1]=-cosh⁡⁡(2t)sinh⁡⁡tAd(at-1)Y3-1tanh⁡tK12,[K23,Ad(at-1)Y4]=-1+cosh⁡2tsinh⁡⁡tAd(at-1)Y3-2tanh⁡tK12.
Here, [X,Y]:=XY-YX is the Lie bracket on 𝔤.By using Lemma4.5, we can rewrite Cp2, Cp3 as we wished. Now, since for all F(at)∈Cη,1∞(L)|A is annihilated by the action of U(𝔤)𝔨 and Ad(at-1)Y2U(𝔤)Ad(at-1)Y3U(𝔤), and the actions of Ad(at-1)Y1 and Ad(at-1)Y4 on F are the same (the multiplication by s), we may regard Cp2, Cp3 as the elements in U(𝔤)(mod𝔓), where 𝔓 is the subalgebra of U(𝔤) defined by (4.12)P=(Ad(at-1)Y1-Ad(at-1)Y4)U(g)+Ad(at-1)Y2U(g)+Ad(at-1)Y3U(g)+U(g)k.Lemma 4.6.
One has the congruences(4.13)Cp2≡-14X132-12(tanh⁡(2t)+1tanh⁡(2t))X13+(14tanh2(2t)-13)(Ad(at-1)Y1)2-1(mod⁡P),Cp3≡-112(Ad(at-1)Y1)X132-16(tanh⁡(2t)+1tanh⁡(2t))(Ad(at-1)Y1)X13+(-127+112tanh2(2t))(Ad(at-1)Y1)3-13(Ad(at-1)Y1)(mod⁡P).By combining Proposition4.2, and Lemmas 4.3 and 4.6, we have the following two differential equations.Theorem 4.7.
The Shintani functionF(at)∈Cη,1∞(L)πσ0,ν|A satisfies the following equations:
(1) the differential equation obtained from the action ofCp2:
(4.14)-14d2Fdt2(at)-12(tanh⁡(2t)+1tanh⁡(2t))dFdt(at)+{(-13+14tanh2(2t))s2-1}F(at)=λ2F(at);
(2) the differential equation obtained from the action ofCp3:
(4.15)-112sd2Fdt2(at)-16(tanh⁡(2t)+1tanh⁡(2t))sdFdt(at)+{(-227+112tanh2(2t))s3-13s}F(at)=λ3F(at).
Here,
(4.16)λ2=-13(ν12-ν1ν2+ν22),λ3=-127(2ν1-ν2)(2ν2-ν1)(ν1+ν2)
are the eigenvalues of the Capelli elements on principal series representations.
Equations (4.15)−(4.14) ×(1/3)s give
(4.17){127s3+13λ2s-λ3}F(at)=0.Therefore, ifF(at) is not identically zero, we have (4.18)s3-3(ν12-ν1ν2+ν22)s+(2ν1-ν2)(2ν2-ν1)(ν1+ν2)=0.
By solving this equation, we have (4.19)s=2ν1-ν2,2ν2-ν1,-ν1-ν2.
Therefore, one of the necessary conditions of the existence of nontrivial Shintani functions is that the parameter s is one of the above three values. Now, we assume that s satisfies this condition. We put x=tanh(2t),F̃(x):=F(at) in (4.14). Then, we have (4.20)-4x(1-x)2d2F̃dx2(x)-4(1-x)2dF̃dx(x)+{(-13+14x)s2-1-λ2}F̃(x)=0.
Next, we put F̃(x)=(1-x)μG0(x)(μ∈C). Then, G0 satisfies (4.21)-4x(1-x)μ+2d2G0dx2(x)+(-4+(8μ+4)x)(1-x)μ+1dG0dx(x)+{-4μ(μ-1)x+4μ-4μx+(-13+14x)s2-1-λ2}(1-x)μG0(x)=0.
We want to divide the left-hand side of (4.21) by (1-x)μ+1. To do this, we take μ∈C so that μ satisfies (4.22)-4μ2+4μ-112s2-1-λ2=0.
The value of μ∈C is as follows. (1)
Ifs=2ν1-ν2,μ=(2±ν2)/4. So we take μ=(2+ν2)/4.(2)
Ifs=2ν2-ν1,μ=(2±ν1)/4. So we take μ=(2+ν1)/4.(3)
Ifs=-ν1-ν2,μ={2±(ν1-ν2)}/4. So we take μ=(2+ν1-ν2)/4.For thisμ, the left-hand side of (4.21) is divided by (1-x)μ+1, and the equation becomes (4.23)x(x-1)d2G0dx2(x)+(-1+(1+2μ)x)dG0dx+(μ2-116s2)G0(x)=0.
This is the Gaussian hypergeometric differential equation. Note that the Shintani function F(at) on A is regular at the origin (⇔x=0). Therefore, G0(x)=(1-x)-μF̃(x) is also regular at x=0. Equation (4.23) has just one solution which is regular around x=0 (up to constant multiples), and it is given by (4.24)G0(x)=F21(α,β;1;x),
where 2F1 is the Gaussian hypergeometric function and α,β∈C are defined by (4.25)1+α+β=1+2μ,αβ=μ2-116s2.
Explicitly, by solving these equations, α,β are given as follows. (1)
In case ofs=2ν1-ν2,(α,β)=((ν1+1)/2,(-ν1+ν2+1)/2).(2)
In case ofs=2ν2-ν1,(α,β)=((ν2+1)/2,(-ν2+ν1+1)/2).(3)
In case ofs=-ν1-ν2,(α,β)=((ν1+1)/2,(-ν2+1)/2).Finally, we consider the three conditions in (3.8). Condition (1) is equivalent to (-1)kF(at)=F(at). Condition (2) always holds. Condition (3) is equivalent to (-1)kF(1)=F(1), which holds if condition (1) is satisfied. Summing up, we have the following theorem.Theorem 4.8.
Letη=ηs,k be the unitary character of H defined by (3.1). Then, the necessary condition of the existence of the nontrivial elements in Cη,1∞(L)πσ0,ν is that
(4.26)k=0,s=2ν1-ν2,2ν2-ν1,-ν1-ν2.
Suppose that this condition is satisfied and nontrivial Shintani functions exist. If one puts x=tanh2(2t), F(at)=F̃(x)∈Cη,1∞(L)πσ0,ν|A is given as follows (up to constant multiples). (1)
In case ofs=2ν1-ν2, one has
(4.27)F̃(x)=(1-x)(2+ν2)/4F21(ν1+12,-ν1+ν2+12;1;x).(2)
In case ofs=2ν2-ν1, one has
(4.28)F̃(x)=(1-x)(2+ν1)/4F21(ν2+12,-ν2+ν1+12;1;x).(3)
In case ofs=-ν1-ν2, one has
(4.29)F̃(x)=(1-x)(2+ν1-ν2)/4F21(ν1+12,-ν2+12;1;x).
Especially, one has(4.30)dim⁡Cη,1∞(L)πσ0,ν≤1.


## 4.1. The Capelli Elements

LetZ(𝔤) be the center of the universal enveloping algebra U(𝔤) of 𝔤. Z(𝔤) has two independent generators, and they are obtained as the Capelli elements because 𝔤=𝔰𝔩3 is of type A2 (see [13]). For i=1,2,3, we put (4.1)Eii′=Eii-13(∑k=13Ekk).
The following proposition gives the explicit description of the independent generators of Z(𝔤) (see [11]).Proposition 4.1.
The independent generators{Cp2,Cp3} of Z(𝔤) are given as follows:
(4.2)Cp2=(E11′-1)E22′+E22′(E33′+1)+(E11′-1)(E33′+1)-E23E32-E13E31-E12E21,Cp3=(E11′-1)E22′(E33′+1)+E12E23E31+E13E21E32-(E11′-1)E23E32-E13E22′E31-E12E21(E33′+1).SinceCp2,Cp3 are the elements of Z(𝔤), they act on πσ0,ν as the scalar operators. And since the space of Shintani functions is the image of the (𝔤C,K)-homomorphism of πσ0,ν, they act on the space of Shintani functions as the same scalar operators, respectively.


## 4.2. Eigenvalues ofCp2, Cp3

In order to construct the partial differential equations satisfied by spherical functions attached to the spherical principal series, we have to compute the eigenvalues of the actions of the Capelli elementsCp2, Cp3. For the spherical principal series representation, σ=σ0 is the trivial character of M. Let f0 be the generator of the minimal K-type in Hσ0,ν normalized such that f0∣K≡1. The actions of Cp2,Cp3 on f0 are computed in [11], and the result is as follows.Proposition 4.2.
The Capelli elementsCp2,Cp3 act on f0 by scalar multiples, and the eigenvalues are given as follows:
(4.3)Cp2f0=-13(ν12-ν1ν2+ν22)f0,Cp3f0=-127(2ν1-ν2)(2ν2-ν1)(ν1+ν2)f0.


## 4.3. Construction of the Casimir Equations

Next, we compute the actions ofCp2, Cp3 on F(at)∈Cη,1∞(L)|A. Here, η=ηs,k is the unitary character of H.Lemma 4.3.
ForF(at)∈Cη,τ∞(L)|A, one has
(4.4)R(Ad(at-1)Yi)F(at)=sF(at)(i=1,4),R(Ad(at-1)Yi)F(at)=0(i=2,3),R(X13)F(at)=dFdt(at).Proof.
By definition, we have(4.5)R(Ad(at-1)Yi)F(at)=dduF(atexp⁡(uAd(at-1)Yi))|u=0=dduF(exp⁡(uYi)at)|u=0=dduη(exp⁡(uYi))|u=0F(at).
Since exp(uY1)=diag(eu,1,e-u), we have η((exp(uY1))=eku·e(s-k)u=esu.
Therefore, we have(4.6)R(Ad(at-1)Y1)F(at)=sF(at).
Next, since
(4.7)exp⁡⁡(uY2)=(cosh⁡⁡usinh⁡⁡u0sinh⁡⁡ucosh⁡⁡u0001),
η(exp(uY2))=1k·|1|s-k=1. Therefore, we have
(4.8)R(Ad(at-1)Y2)F(at)=0.
The computations of the actions of Ad(at-1)(Yi)(i=3,4) are similar. Finally, since exp(uX13)=au, we have
(4.9)R(X13)F(at)=dduF(at+u)|u=0=dFdt(at).By simple computations of matrices, we have the following expressions of the elements inM(3,R).Lemma 4.4.
One has(4.10)E11′=cosh⁡2t+13cosh⁡⁡(2t)Ad(at-1)Y1-13Ad(at-1)Y4-12tanh⁡(2t)K13,E22′=13(-H12+H23),E33′=sinh⁡2t-13cosh⁡⁡(2t)Ad(at-1)Y1-13Ad(at-1)Y4+12tanh⁡(2t)K13,E23=12sinh⁡⁡tAd(at-1)Y3+12tanhtK12+12K23,E13=12X13+12K13,E12=12cosh⁡⁡tAd(at-1)Y2-12tanh⁡tK23+12K12.To make use of Lemma3.4, we have to rewrite Cp2, Cp3 in the form of linear combinations of the elements in (Ad(at-1)𝔥)𝔞𝔨. To do this, we use the following formulas which can be obtained by direct computation.Lemma 4.5.
One has(4.11)[K13,Ad(at-1)Y1]=-2cosh⁡⁡(2t)X13,[K13,Ad(at-1)Y4]=-cosh⁡⁡(2t)X13,[K12,Ad(at-1)Y3]=sinh⁡⁡tX13,[K23,Ad(at-1)Y3]=-2sinh⁡3tcosh⁡⁡(2t)Ad(at-1)Y1+2sinh⁡⁡tAd(at-1)Y4+cosh⁡⁡tcosh⁡⁡(2t)K13,[K13,X13]=2cosh⁡⁡(2t)Ad(at-1)Y1-2tanh⁡(2t)K13,[K23,Ad(at-1)Y2]=-cosh⁡⁡tX13,[K12,Ad(at-1)Y2]=2cosh⁡3tcosh⁡⁡(2t)Ad(at-1)Y1-2cosh⁡⁡tAd(at-1)Y4-sinh⁡⁡tcosh⁡⁡(2t)K13,[K12,X13]=-1sinh⁡⁡tAd(at-1)Y3-1tanh⁡tK12,[K23,X13]=1cosh⁡⁡tAd(at-1)Y2-tanh⁡tK23,[X13,Ad(at-1)Y2]=-tanh⁡tAd(at-1)Y2-1cosh⁡⁡tK23,[K13,Ad(at-1)Y2]=-1tanh⁡tAd(at-1)Y3-cosh⁡⁡(2t)sinh⁡⁡tK12,[K13,Ad(at-1)Y3]=tanh⁡tAd(at-1)Y2-cosh⁡⁡(2t)cosh⁡⁡tK23,[X13,Ad(at-1)Y1]=-2tanh⁡(2t)Ad(at-1)Y1-2cosh⁡⁡(2t)K13,[X13,Ad(at-1)Y4]=-tanh⁡(2t)Ad(at-1)Y1-1cosh⁡⁡(2t)K13,[K12,Ad(at-1)Y1]=-cosh⁡⁡(2t)cosh⁡⁡tAd(at-1)Y2-tanh⁡tK23,[K12,Ad(at-1)Y4]=1-sinh⁡2tcosh⁡⁡tAd(at-1)Y2-2tanh⁡tK23,[K23,Ad(at-1)Y1]=-cosh⁡⁡(2t)sinh⁡⁡tAd(at-1)Y3-1tanh⁡tK12,[K23,Ad(at-1)Y4]=-1+cosh⁡2tsinh⁡⁡tAd(at-1)Y3-2tanh⁡tK12.
Here, [X,Y]:=XY-YX is the Lie bracket on 𝔤.By using Lemma4.5, we can rewrite Cp2, Cp3 as we wished. Now, since for all F(at)∈Cη,1∞(L)|A is annihilated by the action of U(𝔤)𝔨 and Ad(at-1)Y2U(𝔤)Ad(at-1)Y3U(𝔤), and the actions of Ad(at-1)Y1 and Ad(at-1)Y4 on F are the same (the multiplication by s), we may regard Cp2, Cp3 as the elements in U(𝔤)(mod𝔓), where 𝔓 is the subalgebra of U(𝔤) defined by (4.12)P=(Ad(at-1)Y1-Ad(at-1)Y4)U(g)+Ad(at-1)Y2U(g)+Ad(at-1)Y3U(g)+U(g)k.Lemma 4.6.
One has the congruences(4.13)Cp2≡-14X132-12(tanh⁡(2t)+1tanh⁡(2t))X13+(14tanh2(2t)-13)(Ad(at-1)Y1)2-1(mod⁡P),Cp3≡-112(Ad(at-1)Y1)X132-16(tanh⁡(2t)+1tanh⁡(2t))(Ad(at-1)Y1)X13+(-127+112tanh2(2t))(Ad(at-1)Y1)3-13(Ad(at-1)Y1)(mod⁡P).By combining Proposition4.2, and Lemmas 4.3 and 4.6, we have the following two differential equations.Theorem 4.7.
The Shintani functionF(at)∈Cη,1∞(L)πσ0,ν|A satisfies the following equations:
(1) the differential equation obtained from the action ofCp2:
(4.14)-14d2Fdt2(at)-12(tanh⁡(2t)+1tanh⁡(2t))dFdt(at)+{(-13+14tanh2(2t))s2-1}F(at)=λ2F(at);
(2) the differential equation obtained from the action ofCp3:
(4.15)-112sd2Fdt2(at)-16(tanh⁡(2t)+1tanh⁡(2t))sdFdt(at)+{(-227+112tanh2(2t))s3-13s}F(at)=λ3F(at).
Here,
(4.16)λ2=-13(ν12-ν1ν2+ν22),λ3=-127(2ν1-ν2)(2ν2-ν1)(ν1+ν2)
are the eigenvalues of the Capelli elements on principal series representations.
Equations (4.15)−(4.14) ×(1/3)s give
(4.17){127s3+13λ2s-λ3}F(at)=0.Therefore, ifF(at) is not identically zero, we have (4.18)s3-3(ν12-ν1ν2+ν22)s+(2ν1-ν2)(2ν2-ν1)(ν1+ν2)=0.
By solving this equation, we have (4.19)s=2ν1-ν2,2ν2-ν1,-ν1-ν2.
Therefore, one of the necessary conditions of the existence of nontrivial Shintani functions is that the parameter s is one of the above three values. Now, we assume that s satisfies this condition. We put x=tanh(2t),F̃(x):=F(at) in (4.14). Then, we have (4.20)-4x(1-x)2d2F̃dx2(x)-4(1-x)2dF̃dx(x)+{(-13+14x)s2-1-λ2}F̃(x)=0.
Next, we put F̃(x)=(1-x)μG0(x)(μ∈C). Then, G0 satisfies (4.21)-4x(1-x)μ+2d2G0dx2(x)+(-4+(8μ+4)x)(1-x)μ+1dG0dx(x)+{-4μ(μ-1)x+4μ-4μx+(-13+14x)s2-1-λ2}(1-x)μG0(x)=0.
We want to divide the left-hand side of (4.21) by (1-x)μ+1. To do this, we take μ∈C so that μ satisfies (4.22)-4μ2+4μ-112s2-1-λ2=0.
The value of μ∈C is as follows. (1)
Ifs=2ν1-ν2,μ=(2±ν2)/4. So we take μ=(2+ν2)/4.(2)
Ifs=2ν2-ν1,μ=(2±ν1)/4. So we take μ=(2+ν1)/4.(3)
Ifs=-ν1-ν2,μ={2±(ν1-ν2)}/4. So we take μ=(2+ν1-ν2)/4.For thisμ, the left-hand side of (4.21) is divided by (1-x)μ+1, and the equation becomes (4.23)x(x-1)d2G0dx2(x)+(-1+(1+2μ)x)dG0dx+(μ2-116s2)G0(x)=0.
This is the Gaussian hypergeometric differential equation. Note that the Shintani function F(at) on A is regular at the origin (⇔x=0). Therefore, G0(x)=(1-x)-μF̃(x) is also regular at x=0. Equation (4.23) has just one solution which is regular around x=0 (up to constant multiples), and it is given by (4.24)G0(x)=F21(α,β;1;x),
where 2F1 is the Gaussian hypergeometric function and α,β∈C are defined by (4.25)1+α+β=1+2μ,αβ=μ2-116s2.
Explicitly, by solving these equations, α,β are given as follows. (1)
In case ofs=2ν1-ν2,(α,β)=((ν1+1)/2,(-ν1+ν2+1)/2).(2)
In case ofs=2ν2-ν1,(α,β)=((ν2+1)/2,(-ν2+ν1+1)/2).(3)
In case ofs=-ν1-ν2,(α,β)=((ν1+1)/2,(-ν2+1)/2).Finally, we consider the three conditions in (3.8). Condition (1) is equivalent to (-1)kF(at)=F(at). Condition (2) always holds. Condition (3) is equivalent to (-1)kF(1)=F(1), which holds if condition (1) is satisfied. Summing up, we have the following theorem.Theorem 4.8.
Letη=ηs,k be the unitary character of H defined by (3.1). Then, the necessary condition of the existence of the nontrivial elements in Cη,1∞(L)πσ0,ν is that
(4.26)k=0,s=2ν1-ν2,2ν2-ν1,-ν1-ν2.
Suppose that this condition is satisfied and nontrivial Shintani functions exist. If one puts x=tanh2(2t), F(at)=F̃(x)∈Cη,1∞(L)πσ0,ν|A is given as follows (up to constant multiples). (1)
In case ofs=2ν1-ν2, one has
(4.27)F̃(x)=(1-x)(2+ν2)/4F21(ν1+12,-ν1+ν2+12;1;x).(2)
In case ofs=2ν2-ν1, one has
(4.28)F̃(x)=(1-x)(2+ν1)/4F21(ν2+12,-ν2+ν1+12;1;x).(3)
In case ofs=-ν1-ν2, one has
(4.29)F̃(x)=(1-x)(2+ν1-ν2)/4F21(ν1+12,-ν2+12;1;x).
Especially, one has(4.30)dim⁡Cη,1∞(L)πσ0,ν≤1.


## 5. Shintani Functions Attached to the Nonspherical Principal Series Representations

In this section, as a character ofM, we take a nontrivial character σ=σi(i=1,2,3). Then, the minimal K-type of πσi,ν is the three-dimensional representation of K which is isomorphic to the tautological representation τ2:K=SO(3,R)↪GL(3,R) which occurs of multiplicity one in πσi,ν|K. We take τ2* instead of τ2 as a minimal K-type of πσi,ν. The representation space of τ2 is denoted by Vτ2(=R3), and we take s1=t(1,0,0), s2=t(0,1,0), s3=t(0,0,1) as a basis of Vτ2. Let Ψ∈Cη,τ2∞(L)πσi,ν be the Shintani function. Then, Ψ is expressed by (5.1)Ψ(g)=F0(g)s1+G0(g)s2+H0(g)s3.
Ψ is characterized by its restriction to A. To investigate Ψ|A, we construct two kinds of differential equations. One is the Casimir equation of degree two and the other is the gradient equation (or the Dirac-Schmidt equation).


### 5.1. The Casimir Equation

Firstly, we construct the Casimir equation of degree two. Since the Capelli elementCp2 acts on the representation space of the principal series representation πσi,ν as a scalar operator (λ2-multiple) and the space of Shintani functions Cη,τ2∞(L)πσi,ν is the image of (𝔤C,K)-homomorphism of πσi,ν, Cp2 acts on this space as the same scalar operator. Since for all Ψ(at)∈Cη,τ2∞(L)|A is annihilated by the action of Ad(at-1)Y2U(𝔤),Ad(at-1)Y3U(𝔤) and the actions of Ad(at-1)Y1 and Ad(at-1)Y4 on F are the same (the multiplication by s), we may regard Cp2 as the element in U(𝔤)(mod𝔓′), where 𝔓′ is a subalgebra of U(𝔤) defined by (5.2)P′=(Ad(at-1)Y1-Ad(at-1)Y4)U(g)+Ad(at-1)Y2U(g)+Ad(at-1)Y3U(g).
By using Lemmas 4.4 and 4.5, we can rewrite Cp2 in Proposition 4.1 as follows.Lemma 5.1.
One has(5.3)Cp2≡-14X132-12(tanh⁡(2t)+1tanh⁡(2t))X13+(14tanh2(2t)-13)(Ad(at-1)Y1)2-1+tanh⁡(2t)2cosh⁡⁡(2t)Ad(at-1)Y1K13+(-14tanh2(2t)+14)K132+(-14tanh2t+14)K122+(14-14tanh2t)K232⋅(mod⁡(Ad(at-1)Y1-Ad(at-1)Y4)U(g)+Ad(at-1)Y2U(g)+Ad(at-1)Y3U(g)).By using this lemma, the action ofCp2 on Ψ(at)=F0(at)s1+G0(at)s2+H0(at)s3∈Cη,τ2∞(L)|A can be computed easily. We have (5.4)Cp2Ψ(at)=F′(at)s1+G′(at)s2+H′(at)s3,
where (5.5)F′(at)=-14d2F0dt2(at)-12(tanh⁡(2t)+1tanh⁡(2t))dF0dt(at)+{(14tanh2(2t)-13)s2+14tanh2(2t)+14tanh2t-32}F0(at)-tanh⁡(2t)2cosh⁡⁡(2t)sH0(at),G′(at)=-14d2G0dt2(at)-12(tanh⁡(2t)+1tanh⁡(2t))dG0dt(at)+{(14tanh2t-13)s2+14tanh2t+14tanh2t-32}G0(at),H′(at)=-14d2H0dt2(at)-12(tanh⁡(2t)+1tanh⁡(2t))dH0dt(at)+{(14tanh2(2t)-13)s2+14tanh2(2t)+14tanh2t-32}H0(at)+tanh⁡(2t)2cosh⁡⁡(2t)sF0(at).
Since Ψ(at) satisfies Cp2Ψ(at)=λ2Ψ(at), we have the following three differential equations.Theorem 5.2.
ForΨ(at)=F0(at)s1+G0(at)s2+H0(at)s3∈Cη,τ2∞(L)πσi,ν|A, the functions F0, G0, H0 satisfy the following equations:
(5.6)-14d2F0dt2(at)-12(tanh⁡(2t)+1tanh⁡(2t))dF0dt(at)+{(14tanh2(2t)-13)s2+14tanh2(2t)+14tanh2t-32}F0(at)-tanh⁡(2t)2cosh⁡⁡(2t)sH0(at)=λ2F0(at),(5.7)-14d2G0dt2(at)-12(tanh⁡(2t)+1tanh⁡(2t))dG0dt(at)+{(14tanh2t-13)s2+14tanh2t+14tanh2t-32}G0(at)=λ2G0(at),(5.8)-14d2H0dt2(at)-12(tanh(2t)+1tanh⁡(2t))dH0dt(at)+{(14tanh2(2t)-13)s2+14tanh2(2t)+14tanh2t-32}H0(at)+tanh⁡(2t)2cosh⁡⁡(2t)sF0(at)=λ2H0(at).


### 5.2. The Gradient Equation

For the spherical functionΨ(g)∈Cη,τ2∞(L), we define the right gradient operator ∇R as follows.Definition 5.3.
For the orthonormal basis{Xi}i=15 of 𝔭, the right gradient operator ∇R is defined by
(5.9)∇RΨ(g):=∑i=15R(Xi)Ψ⊗Xi*.
Here, Xi* is the dual basis of Xi with respect to the inner product (X,Y)∈𝔭×𝔭→Tr(XY)∈C.
The set{H12,H23,X12,X23,X13} becomes the orthonormal basis of 𝔭, and {(1/3)(2H12+H23),(1/3)(H12+2H23),(1/2)X12,(1/2)X23,(1/2)X13} is its dual basis. Therefore, the gradient operator ∇R is explicitly given by
(5.10)∇RΨ(g)=13R(H12)Ψ⊗(2H12+H23)+13R(H23)Ψ⊗(H12+2H23)+12∑i<jR(Xij)Ψ⊗Xij.
We rewrite this by using the basis of 𝔭C.Claim 1.
We define five elementswi(0≤i≤4)in𝔭Cby(5.11)w0:=-2(H23--1X23),w4:=-2(H23+-1X23),w2:=23(2H12+H23),w1:=X13+-1X12,w3:=-X13+-1X12.
Then, {wi∣0≤i≤4} becomes the basis of 𝔭C.With this basis, the gradient operator∇R is rewritten as (5.12)∇RΨ=116R(w4)Ψ⊗w0+116R(w0)Ψ⊗w4-14R(w3)Ψ⊗w1-14R(w1)Ψ⊗w3+38R(w2)Ψ⊗w2=14{14R(w4)Ψ⊗w0+14R(w0)Ψ⊗w4-R(w3)Ψ⊗w1-R(w1)Ψ⊗w3+32R(w2)Ψ⊗w2}.
The Lie algebra 𝔭C becomes the representation space of the adjoint action of K. We denote this representation by (τ4,W4). By the Clebsch-Gordan theorem, τ2⊗τ4 has the irreducible decomposition (5.13)τ2⊗τ4≅τ2⊕τ4⊕τ6.
Here, each τn is the (n+1)-dimensional irreducible representation of K. In this decomposition, the projector of K-modules (5.14)pr2:τ2⊗τ4↠τ2,si⊗wj⟼pr2(si⊗wj),
is described as in Table 2.Table 2w0w1w2w3w4s10-14(s3+-1s2)-13s114(s3--1s2)0s212(s2--1s3)--14s116s2--14s112(s2+-1s3)s3-12(s3+-1s2)-14s116s314s112(-s3+-1s2)∇RΨ is a τ2⊗(τ2⊗𝔭C)-valued function. Then, by mapping si⊗wk to pr2(si⊗wk), we have a K-homomorphism (5.15)pr2̃∘∇R:Cη,τ2∞(L)⟶Cη,τ2∞(L).
Since the minimal K-type τ2* occurs of multiplicity one, pr2̃∘∇R is a map of constant multiple. To compute the action of the gradient operator pr2̃∘∇R on the space of the Shintani functions Cη,τ2∞(L)πσi,ν, we have to decompose wi(i=0,1,2,3,4) along the decomposition 𝔤C=Ad(at-1)𝔥C⊕𝔞C⊕𝔨C.Lemma 5.4.
One has(5.16)w0=2sinh⁡2tcosh⁡⁡(2t)Ad(at-1)Y1-2Ad(at-1)Y4+tanh⁡(2t)K13+2-1sinh⁡⁡tAd(at-1)Y3+2-1tanh⁡tK12,w4=2sinh⁡2tcosh⁡⁡(2t)Ad(at-1)Y1-2Ad(at-1)Y4+tanh⁡(2t)K13-2-1sinh⁡⁡tAd(at-1)Y3-2-1tanh⁡tK12,w2=2(cosh⁡2t+1)3cosh⁡⁡(2t)Ad(at-1)Y1-23Ad(at-1)Y4-tanh(2t)K13,w1=X13+-1cosh⁡⁡tAd(at-1)Y2--1tanh⁡tK23,w3=-X13+-1cosh⁡⁡tAd(at-1)Y2--1tanh⁡tK23.By using Lemmas3.4 and 4.3 and the table of projections, we can compute the action of the gradient operator. For Ψ(at)=F0(at)s1+G0(at)s2+H0(at)s3∈Cη,τ2∞(L)|A, we have (5.17)pr2̃∘∇RΨ(at)=F′′(at)s1+G′′(at)s2+H′′(at)s3,
where (5.18)F′′(at)=-(12cosh⁡⁡(2t)-16)sF0(at)-12dH0dt(at)-12(tanh⁡(2t)+tanh⁡t)H0(at),G′′(at)=-13sG0(at),H′′(at)=(12cosh⁡⁡(2t)+16)sH0(at)-12dF0dt(at)-12(tanh⁡(2t)+1tanh⁡t)F0(at).
On the other hand, the eigenvalue of the gradient operator on the spherical functions of the principal series representation πσi,ν depends on the choice of σi, denoted by λσi(i=1,2,3). These values are computed in [11] and they are as follows: (5.19)λσ1=-13(2ν1-ν2),λσ2=-13(2ν2-ν1),λσ3=13(ν1+ν2).
Therefore, since Ψ(at)=F0(at)s1+G0(at)s2+H0(at)s3∈Cη,τ2∞(L)πσi,ν|A satisfies pr2̃∘∇RΨ(at)=λσiΨ(at), we have the following three differential equations.Theorem 5.5.
ForΨ(at)=F0(at)s1+G0(at)s2+H0(at)s3∈Cη,τ2∞(L)πσi,ν|A, one has
(5.20)-(12cosh⁡⁡(2t)-16)sF0(at)-12dH0dt(at)-12(tanh⁡(2t)+tanh⁡t)H0(at)=λσiF0(at),(5.21)-13sG0(at)=λσiG0(at),(5.22)(12cosh⁡⁡(2t)+16)sH0(at)-12dF0dt(at)-12(tanh⁡(2t)+1tanh⁡t)F0(at)=λσiH0(at).We consider the case ofσ=σ1. We have λσ1=-(1/3)(2ν1-ν2). By (5.21), if s≠-3λσ1=2ν1-ν2, we have G0(at)≡0. Suppose that s=2ν1-ν2. We put x=tanh2(2t), G0(at)=G1(x) in (5.21). Then, the equation becomes (5.23)-4x2(1-x)2d2G1dx2(x)-4x(1-x)2dG1dx(x)+{(14x2-13x)s2-(2+λ2)x+1}G1(x)=0.
We put G1(x)=xα(1-x)βG̃1(x)(α,β∈C). Then, the equation becomes (5.24)-4x2(1-x)2d2G̃1dx2(x)+4x(1-x){-2α-1+(2α+2β+1)x}dG̃1dx(x)+{(14x2-13x)-4α(α-1)(1-x)2-4β(β-1)x2+8αβx(1-x)-4α(1-x)2+4βx(1-x)+(14x2-13x)s2-(2+λ2)x+1}G̃1(x)=0.
We want to divide the left-hand side of (5.24) by x(1-x). For this purpose, α,β∈C must satisfy (5.25)-4α2+1=0,-4β(β-1)-112s2-λ2-1=0.
The solutions are α=±1/2,β=(2±ν2)/4. We choose α=1/2,β=(2+ν2)/4. Then the left-hand side of (5.24) can be divided by x(1-x), and the equation becomes (5.26)x(x-1)d2G̃1dx2(x)+{-2+(3+ν22)x}dG̃1dx(x)+14(-ν12+ν1ν2+2ν2+4)G̃1(x)=0.
This is a Gaussian hypergeometric differential equation, and its regular solution is given by (5.27)G̃1(x)=F21(12ν1+1,12ν2-12ν1+1;2;x)
up to constant multiples. (The other solutions are not regular around x=0, since they contain logx.) Therefore, we have (5.28)G0(at)=G0(x)=Cx1/2(1-x)(2+ν2)/4F21(12ν1+1,12ν2-12ν1+1;2;x),
where C is a constant number and x=tanh2(2t). Next, we consider the equations satisfied by F0 and H0. From (5.20) and (5.22), we have (5.29)dF0dt(at)=(1cosh⁡⁡(2t)+13)sH0(at)-(tanh⁡(2t)+1tanh⁡t)F0(at)-2λσ1H0(at),dH0dt(at)=-(1cosh⁡⁡(2t)-13)sF0(at)-(tanh⁡(2t)+tanh⁡t)H0(at)-2λσ1F0(at).
By differentiating both sides of (5.29) by t, we have
(5.30)d2F0dt2(at)={(1cosh⁡2(2t)-19)3tanh2(2t)+2tanh2t+2tanh⁡(2t)tanh⁡t-3-43sλσ1-(1cosh⁡2(2t)-19)s2+4λσ12}F0(at)+{-(4tanh⁡(2t)cosh⁡⁡(2t)+23tanh⁡(2t)+23tanh⁡(2t)+2sinh⁡⁡(2t))s+4(tanh⁡(2t)+1tanh⁡(2t))λσ1}H0(at),d2H0dt2(at)={(1cosh⁡2(2t)-19)3tanh2(2t)+2tanh2t+2tanh⁡ttanh⁡(2t)-3-43sλσ1-(1cosh⁡2(2t)-19)s2+4λσ12}H0(at)+{(4tanh⁡(2t)cosh⁡⁡(2t)-23tanh⁡(2t)-23tanh⁡(2t)+2sinh⁡⁡(2t))s+4(tanh⁡(2t)+1tanh⁡(2t))λσ1}F0(at).
By inserting (5.29) and (5.30) into the Casimir equation (5.6), (5.8) to eliminate the differential terms, we have (5.31)(13sλσ1-λσ12-19s2-λ2)F0(at)=0,(13sλσ1-λσ12-19s2-λ2)H0(at)=0.
Therefore, if the parameter s∈C satisfies (1/3)sλσ1-λσ12-(1/9)s2-λ2≠0, then F0(at),H0(at)≡0. Suppose that (1/3)sλσ1-λσ12-(1/9)s2-λ2=0. Since λσ1=-(1/3)(2ν1-ν2),λ2=-(1/3)(ν12-ν1ν2+ν22), we have (5.32)s2+(2ν1-ν2)s+(ν12-ν1ν2-2ν22)=0.
Therefore, we have (5.33)s=-ν1-ν2,-ν1+2ν2.
That is, (5.33) is the necessary condition of the existence of nontrivial F0(at),H0(at). We put (5.34)F0(at)=cosh⁡-1/2⁡(2t)(sinh⁡⁡t)-1F̃(t),H0(at)=cosh⁡-1/2⁡(2t)(cosh⁡⁡t)-1H̃(t)
and insert these into (5.20), (5.22). Then we have (5.35)-(12cosh⁡(2t)-16)s1tanh⁡tF̃(t)-12dH̃dt(t)=λσ11tanh⁡tF̃(t),(12cosh⁡⁡(2t)+16)stanh⁡tH̃(t)-12dF̃dt(t)=λσ1tanh⁡tH̃(t).
Next, we put tanh2t=u,F̃(t)=F̃1(u),H̃(t)=H̃1(u). Then, the above equations become (5.36)-(1-u2(1+u)-16)sF̃1(u)-u(1-u)dH̃1du(u)=λσ1F̃1(u),(5.37)-(1-u)dF̃1du(u)+(1-u2(1+u)+16)sH̃1(u)=λσ1H̃1(u).
For a while, we consider the case of s=0, that is, η=η0,k is a signature sgn(k) of H, where (5.38)sgn⁡(k)((h11h120h21h22000h1))=det⁡⁡(H1)k|det⁡⁡(H1)|-k,H1=(h11h12h21h22).
Then, by combining (5.36), (5.37), we have (5.39)u(1-u)2λσ1d2F̃1du2(u)-u(1-u)λσ1dF̃1du(u)-λσ13F̃1(u)=0.
Suppose that λσ1≠0. Then, the equation becomes (5.40)u(1-u)2d2F̃1du2(u)-u(1-u)dF̃1du(u)-λσ12F̃1(u)=0.
We put F̃1(u)=(1-u)λσ1F2(u). Then, F2(u) satisfies (5.41)u(1-u)d2F2du2(u)-(2λσ1+1)udF2du(u)-λσ12F2(u)=0.
Equation (5.41) is the Gaussian hypergeometric differential equation. Now, since F2(u)=(1-u)-λσ1cosh1/2(2t)sinhtF0(at) and F0(at) is regular at at=1(⇔t=0⇔u=0), F2(u) must be regular at u=0. The regular solution of (5.41) is given by (5.42)F2(u)=uF21(λσ1+1,λσ1+1;2;u)
up to constant multiples. Therefore, we have (5.43)F̃1(u)=C⋅u(1-u)λσ1F21(λσ1+1,λσ1+1;2;u)(C:constant).
Similarly, we have (5.44)H̃1(u)=C′⋅(1-u)λσ1F21(λσ1+1,λσ1;1;u)(C′:constant).
We want to find the relation between C and C′. By expanding F̃1(u) and H̃1(u) around u=0, we have (5.45)F̃1(u)=C⋅(u+λσ12+12u2+u3P1(u)),H̃1(u)=C′⋅(1+λσ12u+u2P2(u)),
where P1(u) and P2(u) are the analytic functions around u=0. By inserting these into (5.36), we have (5.46)-3u(1-u2)C′(λσ12+uP3(u))=3(1+u)λσ1C(u+u2P4(u)),
where P3(u) and P4(u) are also the analytic functions around u=0. By comparing the coefficients of u of both sides, we have (5.47)C=-λσ1C′.
Summing up, F0(at) and H0(at) are given by(5.48)F0(at)=-C′λσ1cosh⁡-1/2⁡(2t)(sinh⁡⁡t)-1u(1-u)λσ1F21(λσ1+1,λσ1+1;2;u),H0(at)=C′cosh⁡-1/2⁡(2t)(cosh⁡⁡t)-1(1-u)λσ1F21(λσ1+1,λσ1;1;u).(C′: some constant, u=tanh2t). In our computation, we assumed that λσ1≠0, but the result above holds without this assumption.Next, we consider conditions (1), (2), and (3) in (3.8). For Ψ=(F0,G0,H0)t∈Csgn(k),τ2∞(L), condition (1) is equivalent to (5.49)(-1)k⋅t(-F0(at),G0(at),-H0(at))=(F0(at),G0(at),H0(at))t.
This is equivalent to (5.50)k=0⟹F0≡0,H0≡0,k=1⟹G0≡0.
Condition (2) is equivalent to(5.51)t(-F0(at),-G0(at),H0(at))=(F0(at),G0(at),H0(at))t.
The solutions we have always satisfy this condition. Condition (3) is equivalent to (5.52)t(cos⁡⁡θF0(1)+sin⁡⁡θG0(1),-sin⁡⁡θF0(1)+cos⁡⁡θG0(1),H0(1))=(F0(1),G0(1),H0(1))t,t(cos⁡⁡θF0(1)+sin⁡⁡θG0(1),sin⁡θF0(1)-cos⁡⁡θG0(1),-H0(1))=(-1)k⋅t(F0(1),G0(1),H0(1)).
(for all θ∈R). Since F0(1)=G0(1)=0, (5.52) are equivalent to (5.53)k=0⟹H0≡0.
But this condition holds if condition (5.50) is satisfied. We have obtained a result about the Shintani functions attached to the nonspherical principal series representation πσ1,ν. Note that since the transform ν1↦ν2,ν2↦ν1 does not change the eigenvalue of Casimir operator λ2 and changes the eigenvalue of gradient operator λσ1 to λσ2, this transform gives the result in case of σ=σ2. Similarly, the transform ν1↦-ν1,ν2↦-ν1+ν2 gives the result in case of σ=σ3. Summing up these results, we have the following theorem.Theorem 5.6.
Letη=sgn(k)(k∈{0,1}) be a signature of H defined by (5.38) and τ2 a three-dimensional tautological representation of K, and let Ψ=(F0,G0,H0)t∈Csgn(k),τ2∞(L)πσ1,ν be a Shintani function corresponding to the nonspherical principal series representation πσ1,ν of G. Then, the restriction of Ψ to A is given as follows.
(1)
In case ofk=0,
(a)
if2ν1-ν2≠0, Ψ≡0;(b)
if2ν1-ν2=0, one has
(5.54)(F0(at)G0(at)H0(at))=C⋅(0x1/2(1-x)(2+ν2)/4F21(12ν1+1,12ν2-12ν1+1,;2;x)0).(x=tanh2(2t),C:someconstant).(2)
In case ofk=1,
(a)
if(-ν1-ν2)(-ν1+2ν2)≠0, Ψ≡0;(b)
if(-ν1-ν2)(-ν1+2ν2)=0, one has
(5.55)(F0(at)G0(at)H0(at))=C⋅(-λσ1cosh⁡-1/2⁡(2t)(sinh⁡t)-1u(1-u)λσ1F21(λσ1+1,λσ1+1;2;u)0cosh⁡-1/2⁡(2t)(cosh⁡⁡t)-1(1-u)λσ1F21(λσ1+1,λσ1;1;u)).(u=tanh2t,C:someconstant).
Especially, in any case, one has
(5.56)dim⁡⁡Csgn⁡(k),τ2∞(L)πσ1,ν≤1.
The transform ν1↦ν2,ν2↦ν1 gives the result in case of σ=σ2 and the transform ν1↦-ν1,ν2↦-ν1+ν2 gives the result in case of σ=σ3.Next, we compute the Shintani functionsΨ∈Cη,τ2∞(L)πσ1,ν for general unitary character η=ηs,k under the assumption that 1,ν1,ν2 are linearly independent over Q. We have already known that the necessary condition of the existence of non zero Ψ is that the parameter s is one of 2ν1-ν2,2ν2-ν1, or -ν1-ν2, and we have already solved the differential equations in case of s=2ν1-ν2. Hereafter, we suppose that the parameter s is either 2ν2-ν1 or -ν1-ν2. We put w=(1-u)/(1+u), F̃2(w)=F̃1(u), H̃2(w)=H̃1(u) in (5.36), (5.37). Then we have (5.57)-(w2-16)sF̃2(w)+w(1-w)dH̃2dw(w)=λσ1F̃2(w),w(1+w)dF̃2dw(w)+(w2+16)sH̃2(w)=λσ1H̃2(w).
We put (5.58)F̃2(w)=∑n=0∞anwn+α,H̃2(w)=∑n=0∞cnwn+γ
(α,γ∈C,a0,c0≠0) in (5.57) and compute the power series solutions. By inserting these series into (5.57), we have (5.59)s6a0wα+wα+1Q1(w)+γc0wγ+wγ+1Q2(w)=λσ1a0wα+wα+1Q3(w),αa0wα+wα+1Q4(w)+s6c0wγ+wγ+1Q5(w)=λσ1c0wγ+wγ+1Q6(w),
where Qi(w)(i=1,…,6) are the analytic functions around w=0. By comparing the lowest terms in power series, easily we have α=γ (in this argument, we use the fact that 1,ν1,ν2 are linearly independent over Q carefully). Therefore, from (5.59), we have (5.60)(s6-λσ1)+γc0=0,αa0+(s6-λσ1)c0=0.
Since (a0,c0)≠(0,0), we have (5.61)(s6-λσ1)2-αγ=0.
By combining this and α=γ, we have (5.62)α=γ=±(s6-λσ1).
Hereafter, we put A=A(ν1,ν2)=(s/6)-λσ1. Then, F̃2(w) (resp., H̃2(w)) are expressed by the linear combination of some power series ∑n=0∞anwn+A, ∑n=0∞an′wn-A (resp., ∑n=0∞cnwn+A, ∑n=0∞cn′wn-A). That is, there exist common constants C+, C- such that (5.63)F̃2(w)=C+∑n=0∞anwn+A+C-∑n=0∞an′wn-A,H̃2(w)=C+∑n=0∞cnwn+A+C-∑n=0∞cn′wn-A.
By inserting (5.64)F̃2(w)=∑n=0∞anwn+A,H̃2(w)=∑n=0∞cnwn+A
into (5.57) and picking up the coefficients of wn+A, we have the following recurrence relations: (5.65)-s2an-1+Aan-(n+A-1)cn-1+(n+A)cn=0,(5.66)(n+A-1)an-1+(n+A)an+s2cn-1+Acn=0,
for all n≥0. Here, we assume that al=cl=0 if l<0. From (5.65), (5.66), easily we have cn=(-1)n+1an for all n≥0 by induction. Therefore, by inserting cn=(-1)n+1an,cn-1=(-1)nan-1 into (5.65), we have (5.67)(A+(n+A)(-1)n+1)an=(s2+(n+A-1)(-1)n)an-1.
Thus, we have (5.68)an={∏k=1ns/2+(n+A-1)(-1)kA+(k+A)(-1)k+1}a0,(5.69)cn=(-1)n+1{∏k=1ns/2+(n+A-1)(-1)kA+(k+A)(-1)k+1}a0.
Similarly, if the characteristic roots are α=γ=-A, by inserting (5.70)F̃2(w)=∑n=0∞an′wn-A,H̃2(w)=∑n=0∞cn′wn-A
into (5.57), we have (5.71)an′={∏k=1ns/2+(n-A-1)(-1)kA+(k-A)(-1)k+1}a0,cn′=(-1)n{∏k=1ns/2+(n-A-1)(-1)kA+(k-A)(-1)k+1}a0.
(for all n≥1). From (5.68), a2n and a2n-1 are expressed by(5.72)a2n=(s/4+A/2+1/2)n(-s/4+A/2)nn!(A+1/2)na0,a2n-1=-(s/4+A/2+1/2)n-1(-s/4+A/2)n(n-1)!(A+1/2)na0.
Here, for α∈C,n∈Z>0, we define (α)n=α(α+1)(α+2)⋯(α+n-1) and (α)0:=1. Therefore, if we normalize a0=1, we have (5.73)∑n=0∞a2nw2n=∑n=0∞(s/4+A/2+1/2)n(-s/4+A/2)nn!(A+1/2)nw2n=F21(s4+A2+12,-s4+A2;A+12;w2),∑n=1∞a2n-1w2n-1=--s/4+A/2A+1/2∑n=1∞(s/4+A/2+1/2)n-1(-s/4+A/2+1)n-1(n-1)!(A+3/2)n-1w2n-2⋅w=s/4-A/2A+1/2w∑n=0∞(s/4+A/2+1/2)n(-s/4+A/2+1)nn!(A+3/2)nw2n=s/4-A/2A+1/2wF21(s4+A2+12,-s4+A2+1;A+32;w2).
Thus, we have (5.74)∑n=0∞anwn+A=wA{F21(s4+A2+12,-s4+A2;A+12;w2)+s/4-A/2A+1/2wF21(s4+A2+12,-s4+A2+1;A+32;w2)}.
And since cn=(-1)n+1an, we have (5.75)∑n=0∞cnwn+A=wA{-F21(s4+A2+12,-s4+A2;A+12;w2)+s/4-A/2A+1/2wF21(s4+A2+12,-s4+A2+1;A+32;w2)}.
Similarly, by using (5.76)a2n′=(-s/4-A/2+1/2)n(s/4-A/2)nn!(-A+1/2)na0′,a2n-1′=-(-s/4-A/2+1/2)n-1(s/4-A/2)n(n-1)!(-A+1/2)na0′,
and cn′=(-1)nan′, if we normalize a0′=1, we have (5.77)∑n=0∞an′wn-A=w-A{F21(-s4-A2+12,s4-A2;-A+12;w2)-s/4-A/2-A+1/2wF21(-s4-A2+12,s4-A2+1;-A+32;w2)},∑n=0∞cn′wn-A=w-A{F21(-s4-A2+12,s4-A2;-A+12;w2)+s/4-A/2-A+1/2wF21(-s4-A2+12,s4-A2+1;-A+32;w2)}.
Therefore, F̃2(w) and H̃2(w) are expressed as follows:
(5.78)F̃2(w)=C+wA{F21(s4+A2+12,-s4+A2;A+12;w2)+s/4-A/2A+1/2wF21(s4+A2+12,-s4+A2+1;A+32;w2)}+C-w-A{F21(-s4-A2+12,s4-A2;-A+12;w2)-s/4-A/2-A+1/2wF21(-s4-A2+12,s4-A2+1;-A+32;w2)},H̃2(w)=C+wA{-F21(s4+A2+12,-s4+A2;A+12;w2)+s/4-A/2A+1/2wF21(s4+A2+12,-s4+A2+1;A+32;w2)}+C-w-A{F21(-s4-A2+12,s4-A2;-A+12;w2)+s/4-A/2-A+1/2wF21(-s4-A2+12,s4-A2+1;-A+32;w2)}.
We want the relation between C+ and C-. We can find the relation by using the regularity of the Shintani function and the asymptotic formula of Gaussian hypergeometric function F12. Now, since F0(at) and H0(at) are regular around t=0(⇔w=1) and since (5.79)F̃(t)=cosh⁡1/2⁡(2t)sinh⁡⁡tF0(at),H̃(t)=cosh⁡1/2⁡(2t)cosh⁡⁡tH0(at),F̃2(w)=F̃(t) and H̃2(w)=H̃(t) must be regular around w=1 and F̃2(w) must satisfy F̃2(1)=0. Since all hypergeometric functions appearing in the right-hand sides of (5.78) are in the form of 2F1(a,b;a+b;w2), to investigate the behavior of the right-hand sides of (5.78) around w=1, we use the following asymptotic formula.Formula 1
We have(5.80)2F1(a,b;a+b;z)=-Γ(a+b)(log⁡⁡(1-z)+ψ(a)+ψ(b)+2γ)Γ(a)Γ(b)(1+O(z-1))(z⟶1).
Here, γ is the Euler constant and ψ is defined by
(5.81)ψ(z)=∑k=1∞(1k-1k+z-1)-γ.We apply this formula to the right-hand sides of (5.78). Firstly, the coefficient of log(1-w2) of F̃2(w) equals (5.82)C+{-Γ(A+1/2)Γ(s/4+A/2+1/2)Γ(-s/4+A/2)-s/4-A/2A+1/2Γ(A+3/2)Γ(s/4+A/2+1/2)Γ(-s/4+A/2+1)}+C-{-Γ(-A+1/2)Γ(-s/4-A/2+1/2)Γ(s/4-A/2)+s/4-A/2-A+1/2Γ(-A+3/2)Γ(-s/4-A/2+1/2)Γ(s/4-A/2+1)}=C+⋅0+C-⋅0=0.
Therefore, F̃2(w) is regular around w=1 regardless of the values of C+, C-. Next, the coefficient of log(1-w2) of H̃2(w) equals (5.83)C+{Γ(A+1/2)Γ(s/4+A/2+1/2)Γ(-s/4+A/2)-s/4-A/2A+1/2Γ(A+3/2)Γ(s/4+A/2+1/2)Γ(-s/4+A/2+1)}+C-{-Γ(-A+1/2)Γ(-s/4-A/2+1/2)Γ(s/4-A/2)-s/4-A/2-A+1/2Γ(-A+3/2)Γ(-s/4-A/2+1/2)Γ(s/4-A/2+1)}=2Γ(A+1/2)Γ(s/4+A/2+1/2)Γ(-s/4+A/2)C+-2Γ(-A+1/2)Γ(-s/4-A/2+1/2)Γ(s/4-A/2)C-.
Since this coefficient must be 0, we have (5.84)C+=Γ(-A+1/2)Γ(-s/4-A/2+1/2)Γ(s/4-A/2),C-=Γ(A+1/2)Γ(s/4+A/2+1/2)Γ(-s/4+A/2)
up to (the same) constant multiples. Note that we can easily verify that F̃2(1)=0 for these C+, C- by using formulas Γ(z+1)=zΓ(z),ψ(z+1)=ψ(z)+1/z. Now, we have completely determined F̃2(w),H̃2(w). We have
(5.85)F̃2(w)=Γ(-A+1/2)Γ(-s/4-A/2+1/2)Γ(s/4-A/2)wA×{F12(s4+A2+12,-s4+A2;A+12;w2)+s/4-A/2A+1/2wF12(s4+A2+12,-s4+A2+1;A+32;w2)}+Γ(A+1/2)Γ(s/4+A/2+1/2)Γ(-s/4+A/2)w-A×{F21(-s4-A2+12,s4-A2;-A+12;w2)-s/4-A/2-A+1/2wF12(-s4-A2+12,s4-A2+1;-A+32;w2)},H̃2(w)=Γ(-A+1/2)Γ(-s/4-A/2+1/2)Γ(s/4-A/2)wA×{-F12(s4+A2+12,-s4+A2;A+12;w2)+s/4-A/2A+1/2wF12(s4+A2+12,-s4+A2+1;A+32;w2)}+Γ(A+1/2)Γ(s/4+A/2+1/2)Γ(-s/4+A/2)w-A×{F12(-s4-A2+12,s4-A2;-A+12;w2)+s/4-A/2-A+1/2wF12(-s4-A2+12,s4-A2+1;-A+32;w2)}
up to (the same) constant multiples. Here, A=A(ν1,ν2)=s/6-λσ1, λσ1=-(1/3)(2ν1-ν2), w=(1-u)/(1+u)=(1-tanh2t)/(1+tanh2t), F̃2(w)=F̃(t)=cosh1/2(2t)sinhtF0(at), H̃2(w)=H̃(t)=cosh1/2(2t)coshtH0(at). By the same argument we have done when η is the signature, we can easily verify that conditions (1), (2), and (3) in (3.8) are equivalent to condition (5.50). We have already computed F0(at), G0(at), and H0(at) in any case. Summing up, we obtain the following theorem.Theorem 5.7.
Assume that1,ν1,ν2 are linearly independent over Q. Let η=ηs,k be a unitary character of H defined by (3.1) and σ=σ1 the character of M. Then, the necessary condition of the existence of the nontrivial Shintani functions attached to the nonspherical principal series representation πσ1,ν is that
(5.86)k=0,s=2ν1-ν2
or
(5.87)k=1,s=-ν1-ν2or-ν1+2ν2.
That is, if the condition above is not satisfied, one has
(5.88)Cη,τ2∞(L)πσ1,ν={0}.
Let Ψ(at)=(F0(at),G0(at),H0(at))t∈Cη,τ2∞(L)πσ1,ν|A and suppose that the condition above is satisfied. Then,(1)
ifk=0 and s=2ν1-ν2, one has(5.89)(F0(at)G0(at)H0(at))=C⋅(0x1/2(1-x)(2+ν2)/4F12(12ν1+1,12ν2-12ν1+1,;2;x)0).(x=tanh2(2t),C:someconstant);(2)
ifk=1 and s=-ν1-ν2 or -ν1+2ν2, one has(5.90)(F0(at)G0(at)H0(at))=C⋅(cosh⁡-1/2(2t)sinh⁡-1tF̃2(w)0cosh⁡-1/2(2t)cosh⁡-1tH̃2(w)),whereC is some constant and F̃2(w) and H̃2(w) are the functions given by (5.85). Especially, in any case, one has(5.91)dim⁡Cη,τ2∞(L)πσ1,ν≤1.
The transform ν1↦ν2,ν2↦ν1 gives the result in case of σ=σ2, and the transform ν1↦-ν1,ν2↦-ν1+ν2 gives the result in case of σ=σ3.Remark 5.8.
By using the relation1-x=w2 and the formulas of the hypergeometric function
(5.92)Γ(α)Γ(β)Γ(γ)F12(α,β;γ;z)=Γ(α)Γ(β-α)Γ(γ-α)(1-z)-αF12(α,γ-β;α-β+1;11-z)+Γ(β)Γ(α-β)Γ(γ-β)(1-z)-βF12(β,γ-α;β-α+1;11-z),F21(α,β,γ;z)=(1-z)-αF12(α,γ-β;γ;zz-1)=(1-z)γ-α-βF12(γ-α,γ-β;γ;z),
we can rewrite F̃2(w), H̃2(w) in Theorem 5.7 as functions in x=tanh2(2t). We put F̃2(w)=F3(x), H̃2(w)=H3(x). Then, we have
(5.93)F3(x)=(-s4+A2){-(1-x)A/2F12(s4+A+12,-s4+A2;1;x)+(1-x)(A+1)/2F12(s4+A+12,-s4+A2+1;1;x)},H3(x)=(-s4+A2){(1-x)A/2F12(s4+A+12,-s4+A2;1;x)+(1-x)(A+1)/2F12(s4+A+12,-s4+A2+1;1;x)}.
These computations are due to Professor T. Ishii.


## 5.1. The Casimir Equation

Firstly, we construct the Casimir equation of degree two. Since the Capelli elementCp2 acts on the representation space of the principal series representation πσi,ν as a scalar operator (λ2-multiple) and the space of Shintani functions Cη,τ2∞(L)πσi,ν is the image of (𝔤C,K)-homomorphism of πσi,ν, Cp2 acts on this space as the same scalar operator. Since for all Ψ(at)∈Cη,τ2∞(L)|A is annihilated by the action of Ad(at-1)Y2U(𝔤),Ad(at-1)Y3U(𝔤) and the actions of Ad(at-1)Y1 and Ad(at-1)Y4 on F are the same (the multiplication by s), we may regard Cp2 as the element in U(𝔤)(mod𝔓′), where 𝔓′ is a subalgebra of U(𝔤) defined by (5.2)P′=(Ad(at-1)Y1-Ad(at-1)Y4)U(g)+Ad(at-1)Y2U(g)+Ad(at-1)Y3U(g).
By using Lemmas 4.4 and 4.5, we can rewrite Cp2 in Proposition 4.1 as follows.Lemma 5.1.
One has(5.3)Cp2≡-14X132-12(tanh⁡(2t)+1tanh⁡(2t))X13+(14tanh2(2t)-13)(Ad(at-1)Y1)2-1+tanh⁡(2t)2cosh⁡⁡(2t)Ad(at-1)Y1K13+(-14tanh2(2t)+14)K132+(-14tanh2t+14)K122+(14-14tanh2t)K232⋅(mod⁡(Ad(at-1)Y1-Ad(at-1)Y4)U(g)+Ad(at-1)Y2U(g)+Ad(at-1)Y3U(g)).By using this lemma, the action ofCp2 on Ψ(at)=F0(at)s1+G0(at)s2+H0(at)s3∈Cη,τ2∞(L)|A can be computed easily. We have (5.4)Cp2Ψ(at)=F′(at)s1+G′(at)s2+H′(at)s3,
where (5.5)F′(at)=-14d2F0dt2(at)-12(tanh⁡(2t)+1tanh⁡(2t))dF0dt(at)+{(14tanh2(2t)-13)s2+14tanh2(2t)+14tanh2t-32}F0(at)-tanh⁡(2t)2cosh⁡⁡(2t)sH0(at),G′(at)=-14d2G0dt2(at)-12(tanh⁡(2t)+1tanh⁡(2t))dG0dt(at)+{(14tanh2t-13)s2+14tanh2t+14tanh2t-32}G0(at),H′(at)=-14d2H0dt2(at)-12(tanh⁡(2t)+1tanh⁡(2t))dH0dt(at)+{(14tanh2(2t)-13)s2+14tanh2(2t)+14tanh2t-32}H0(at)+tanh⁡(2t)2cosh⁡⁡(2t)sF0(at).
Since Ψ(at) satisfies Cp2Ψ(at)=λ2Ψ(at), we have the following three differential equations.Theorem 5.2.
ForΨ(at)=F0(at)s1+G0(at)s2+H0(at)s3∈Cη,τ2∞(L)πσi,ν|A, the functions F0, G0, H0 satisfy the following equations:
(5.6)-14d2F0dt2(at)-12(tanh⁡(2t)+1tanh⁡(2t))dF0dt(at)+{(14tanh2(2t)-13)s2+14tanh2(2t)+14tanh2t-32}F0(at)-tanh⁡(2t)2cosh⁡⁡(2t)sH0(at)=λ2F0(at),(5.7)-14d2G0dt2(at)-12(tanh⁡(2t)+1tanh⁡(2t))dG0dt(at)+{(14tanh2t-13)s2+14tanh2t+14tanh2t-32}G0(at)=λ2G0(at),(5.8)-14d2H0dt2(at)-12(tanh(2t)+1tanh⁡(2t))dH0dt(at)+{(14tanh2(2t)-13)s2+14tanh2(2t)+14tanh2t-32}H0(at)+tanh⁡(2t)2cosh⁡⁡(2t)sF0(at)=λ2H0(at).


## 5.2. The Gradient Equation

For the spherical functionΨ(g)∈Cη,τ2∞(L), we define the right gradient operator ∇R as follows.Definition 5.3.
For the orthonormal basis{Xi}i=15 of 𝔭, the right gradient operator ∇R is defined by
(5.9)∇RΨ(g):=∑i=15R(Xi)Ψ⊗Xi*.
Here, Xi* is the dual basis of Xi with respect to the inner product (X,Y)∈𝔭×𝔭→Tr(XY)∈C.
The set{H12,H23,X12,X23,X13} becomes the orthonormal basis of 𝔭, and {(1/3)(2H12+H23),(1/3)(H12+2H23),(1/2)X12,(1/2)X23,(1/2)X13} is its dual basis. Therefore, the gradient operator ∇R is explicitly given by
(5.10)∇RΨ(g)=13R(H12)Ψ⊗(2H12+H23)+13R(H23)Ψ⊗(H12+2H23)+12∑i<jR(Xij)Ψ⊗Xij.
We rewrite this by using the basis of 𝔭C.Claim 1.
We define five elementswi(0≤i≤4)in𝔭Cby(5.11)w0:=-2(H23--1X23),w4:=-2(H23+-1X23),w2:=23(2H12+H23),w1:=X13+-1X12,w3:=-X13+-1X12.
Then, {wi∣0≤i≤4} becomes the basis of 𝔭C.With this basis, the gradient operator∇R is rewritten as (5.12)∇RΨ=116R(w4)Ψ⊗w0+116R(w0)Ψ⊗w4-14R(w3)Ψ⊗w1-14R(w1)Ψ⊗w3+38R(w2)Ψ⊗w2=14{14R(w4)Ψ⊗w0+14R(w0)Ψ⊗w4-R(w3)Ψ⊗w1-R(w1)Ψ⊗w3+32R(w2)Ψ⊗w2}.
The Lie algebra 𝔭C becomes the representation space of the adjoint action of K. We denote this representation by (τ4,W4). By the Clebsch-Gordan theorem, τ2⊗τ4 has the irreducible decomposition (5.13)τ2⊗τ4≅τ2⊕τ4⊕τ6.
Here, each τn is the (n+1)-dimensional irreducible representation of K. In this decomposition, the projector of K-modules (5.14)pr2:τ2⊗τ4↠τ2,si⊗wj⟼pr2(si⊗wj),
is described as in Table 2.Table 2w0w1w2w3w4s10-14(s3+-1s2)-13s114(s3--1s2)0s212(s2--1s3)--14s116s2--14s112(s2+-1s3)s3-12(s3+-1s2)-14s116s314s112(-s3+-1s2)∇RΨ is a τ2⊗(τ2⊗𝔭C)-valued function. Then, by mapping si⊗wk to pr2(si⊗wk), we have a K-homomorphism (5.15)pr2̃∘∇R:Cη,τ2∞(L)⟶Cη,τ2∞(L).
Since the minimal K-type τ2* occurs of multiplicity one, pr2̃∘∇R is a map of constant multiple. To compute the action of the gradient operator pr2̃∘∇R on the space of the Shintani functions Cη,τ2∞(L)πσi,ν, we have to decompose wi(i=0,1,2,3,4) along the decomposition 𝔤C=Ad(at-1)𝔥C⊕𝔞C⊕𝔨C.Lemma 5.4.
One has(5.16)w0=2sinh⁡2tcosh⁡⁡(2t)Ad(at-1)Y1-2Ad(at-1)Y4+tanh⁡(2t)K13+2-1sinh⁡⁡tAd(at-1)Y3+2-1tanh⁡tK12,w4=2sinh⁡2tcosh⁡⁡(2t)Ad(at-1)Y1-2Ad(at-1)Y4+tanh⁡(2t)K13-2-1sinh⁡⁡tAd(at-1)Y3-2-1tanh⁡tK12,w2=2(cosh⁡2t+1)3cosh⁡⁡(2t)Ad(at-1)Y1-23Ad(at-1)Y4-tanh(2t)K13,w1=X13+-1cosh⁡⁡tAd(at-1)Y2--1tanh⁡tK23,w3=-X13+-1cosh⁡⁡tAd(at-1)Y2--1tanh⁡tK23.By using Lemmas3.4 and 4.3 and the table of projections, we can compute the action of the gradient operator. For Ψ(at)=F0(at)s1+G0(at)s2+H0(at)s3∈Cη,τ2∞(L)|A, we have (5.17)pr2̃∘∇RΨ(at)=F′′(at)s1+G′′(at)s2+H′′(at)s3,
where (5.18)F′′(at)=-(12cosh⁡⁡(2t)-16)sF0(at)-12dH0dt(at)-12(tanh⁡(2t)+tanh⁡t)H0(at),G′′(at)=-13sG0(at),H′′(at)=(12cosh⁡⁡(2t)+16)sH0(at)-12dF0dt(at)-12(tanh⁡(2t)+1tanh⁡t)F0(at).
On the other hand, the eigenvalue of the gradient operator on the spherical functions of the principal series representation πσi,ν depends on the choice of σi, denoted by λσi(i=1,2,3). These values are computed in [11] and they are as follows: (5.19)λσ1=-13(2ν1-ν2),λσ2=-13(2ν2-ν1),λσ3=13(ν1+ν2).
Therefore, since Ψ(at)=F0(at)s1+G0(at)s2+H0(at)s3∈Cη,τ2∞(L)πσi,ν|A satisfies pr2̃∘∇RΨ(at)=λσiΨ(at), we have the following three differential equations.Theorem 5.5.
ForΨ(at)=F0(at)s1+G0(at)s2+H0(at)s3∈Cη,τ2∞(L)πσi,ν|A, one has
(5.20)-(12cosh⁡⁡(2t)-16)sF0(at)-12dH0dt(at)-12(tanh⁡(2t)+tanh⁡t)H0(at)=λσiF0(at),(5.21)-13sG0(at)=λσiG0(at),(5.22)(12cosh⁡⁡(2t)+16)sH0(at)-12dF0dt(at)-12(tanh⁡(2t)+1tanh⁡t)F0(at)=λσiH0(at).We consider the case ofσ=σ1. We have λσ1=-(1/3)(2ν1-ν2). By (5.21), if s≠-3λσ1=2ν1-ν2, we have G0(at)≡0. Suppose that s=2ν1-ν2. We put x=tanh2(2t), G0(at)=G1(x) in (5.21). Then, the equation becomes (5.23)-4x2(1-x)2d2G1dx2(x)-4x(1-x)2dG1dx(x)+{(14x2-13x)s2-(2+λ2)x+1}G1(x)=0.
We put G1(x)=xα(1-x)βG̃1(x)(α,β∈C). Then, the equation becomes (5.24)-4x2(1-x)2d2G̃1dx2(x)+4x(1-x){-2α-1+(2α+2β+1)x}dG̃1dx(x)+{(14x2-13x)-4α(α-1)(1-x)2-4β(β-1)x2+8αβx(1-x)-4α(1-x)2+4βx(1-x)+(14x2-13x)s2-(2+λ2)x+1}G̃1(x)=0.
We want to divide the left-hand side of (5.24) by x(1-x). For this purpose, α,β∈C must satisfy (5.25)-4α2+1=0,-4β(β-1)-112s2-λ2-1=0.
The solutions are α=±1/2,β=(2±ν2)/4. We choose α=1/2,β=(2+ν2)/4. Then the left-hand side of (5.24) can be divided by x(1-x), and the equation becomes (5.26)x(x-1)d2G̃1dx2(x)+{-2+(3+ν22)x}dG̃1dx(x)+14(-ν12+ν1ν2+2ν2+4)G̃1(x)=0.
This is a Gaussian hypergeometric differential equation, and its regular solution is given by (5.27)G̃1(x)=F21(12ν1+1,12ν2-12ν1+1;2;x)
up to constant multiples. (The other solutions are not regular around x=0, since they contain logx.) Therefore, we have (5.28)G0(at)=G0(x)=Cx1/2(1-x)(2+ν2)/4F21(12ν1+1,12ν2-12ν1+1;2;x),
where C is a constant number and x=tanh2(2t). Next, we consider the equations satisfied by F0 and H0. From (5.20) and (5.22), we have (5.29)dF0dt(at)=(1cosh⁡⁡(2t)+13)sH0(at)-(tanh⁡(2t)+1tanh⁡t)F0(at)-2λσ1H0(at),dH0dt(at)=-(1cosh⁡⁡(2t)-13)sF0(at)-(tanh⁡(2t)+tanh⁡t)H0(at)-2λσ1F0(at).
By differentiating both sides of (5.29) by t, we have
(5.30)d2F0dt2(at)={(1cosh⁡2(2t)-19)3tanh2(2t)+2tanh2t+2tanh⁡(2t)tanh⁡t-3-43sλσ1-(1cosh⁡2(2t)-19)s2+4λσ12}F0(at)+{-(4tanh⁡(2t)cosh⁡⁡(2t)+23tanh⁡(2t)+23tanh⁡(2t)+2sinh⁡⁡(2t))s+4(tanh⁡(2t)+1tanh⁡(2t))λσ1}H0(at),d2H0dt2(at)={(1cosh⁡2(2t)-19)3tanh2(2t)+2tanh2t+2tanh⁡ttanh⁡(2t)-3-43sλσ1-(1cosh⁡2(2t)-19)s2+4λσ12}H0(at)+{(4tanh⁡(2t)cosh⁡⁡(2t)-23tanh⁡(2t)-23tanh⁡(2t)+2sinh⁡⁡(2t))s+4(tanh⁡(2t)+1tanh⁡(2t))λσ1}F0(at).
By inserting (5.29) and (5.30) into the Casimir equation (5.6), (5.8) to eliminate the differential terms, we have (5.31)(13sλσ1-λσ12-19s2-λ2)F0(at)=0,(13sλσ1-λσ12-19s2-λ2)H0(at)=0.
Therefore, if the parameter s∈C satisfies (1/3)sλσ1-λσ12-(1/9)s2-λ2≠0, then F0(at),H0(at)≡0. Suppose that (1/3)sλσ1-λσ12-(1/9)s2-λ2=0. Since λσ1=-(1/3)(2ν1-ν2),λ2=-(1/3)(ν12-ν1ν2+ν22), we have (5.32)s2+(2ν1-ν2)s+(ν12-ν1ν2-2ν22)=0.
Therefore, we have (5.33)s=-ν1-ν2,-ν1+2ν2.
That is, (5.33) is the necessary condition of the existence of nontrivial F0(at),H0(at). We put (5.34)F0(at)=cosh⁡-1/2⁡(2t)(sinh⁡⁡t)-1F̃(t),H0(at)=cosh⁡-1/2⁡(2t)(cosh⁡⁡t)-1H̃(t)
and insert these into (5.20), (5.22). Then we have (5.35)-(12cosh⁡(2t)-16)s1tanh⁡tF̃(t)-12dH̃dt(t)=λσ11tanh⁡tF̃(t),(12cosh⁡⁡(2t)+16)stanh⁡tH̃(t)-12dF̃dt(t)=λσ1tanh⁡tH̃(t).
Next, we put tanh2t=u,F̃(t)=F̃1(u),H̃(t)=H̃1(u). Then, the above equations become (5.36)-(1-u2(1+u)-16)sF̃1(u)-u(1-u)dH̃1du(u)=λσ1F̃1(u),(5.37)-(1-u)dF̃1du(u)+(1-u2(1+u)+16)sH̃1(u)=λσ1H̃1(u).
For a while, we consider the case of s=0, that is, η=η0,k is a signature sgn(k) of H, where (5.38)sgn⁡(k)((h11h120h21h22000h1))=det⁡⁡(H1)k|det⁡⁡(H1)|-k,H1=(h11h12h21h22).
Then, by combining (5.36), (5.37), we have (5.39)u(1-u)2λσ1d2F̃1du2(u)-u(1-u)λσ1dF̃1du(u)-λσ13F̃1(u)=0.
Suppose that λσ1≠0. Then, the equation becomes (5.40)u(1-u)2d2F̃1du2(u)-u(1-u)dF̃1du(u)-λσ12F̃1(u)=0.
We put F̃1(u)=(1-u)λσ1F2(u). Then, F2(u) satisfies (5.41)u(1-u)d2F2du2(u)-(2λσ1+1)udF2du(u)-λσ12F2(u)=0.
Equation (5.41) is the Gaussian hypergeometric differential equation. Now, since F2(u)=(1-u)-λσ1cosh1/2(2t)sinhtF0(at) and F0(at) is regular at at=1(⇔t=0⇔u=0), F2(u) must be regular at u=0. The regular solution of (5.41) is given by (5.42)F2(u)=uF21(λσ1+1,λσ1+1;2;u)
up to constant multiples. Therefore, we have (5.43)F̃1(u)=C⋅u(1-u)λσ1F21(λσ1+1,λσ1+1;2;u)(C:constant).
Similarly, we have (5.44)H̃1(u)=C′⋅(1-u)λσ1F21(λσ1+1,λσ1;1;u)(C′:constant).
We want to find the relation between C and C′. By expanding F̃1(u) and H̃1(u) around u=0, we have (5.45)F̃1(u)=C⋅(u+λσ12+12u2+u3P1(u)),H̃1(u)=C′⋅(1+λσ12u+u2P2(u)),
where P1(u) and P2(u) are the analytic functions around u=0. By inserting these into (5.36), we have (5.46)-3u(1-u2)C′(λσ12+uP3(u))=3(1+u)λσ1C(u+u2P4(u)),
where P3(u) and P4(u) are also the analytic functions around u=0. By comparing the coefficients of u of both sides, we have (5.47)C=-λσ1C′.
Summing up, F0(at) and H0(at) are given by(5.48)F0(at)=-C′λσ1cosh⁡-1/2⁡(2t)(sinh⁡⁡t)-1u(1-u)λσ1F21(λσ1+1,λσ1+1;2;u),H0(at)=C′cosh⁡-1/2⁡(2t)(cosh⁡⁡t)-1(1-u)λσ1F21(λσ1+1,λσ1;1;u).(C′: some constant, u=tanh2t). In our computation, we assumed that λσ1≠0, but the result above holds without this assumption.Next, we consider conditions (1), (2), and (3) in (3.8). For Ψ=(F0,G0,H0)t∈Csgn(k),τ2∞(L), condition (1) is equivalent to (5.49)(-1)k⋅t(-F0(at),G0(at),-H0(at))=(F0(at),G0(at),H0(at))t.
This is equivalent to (5.50)k=0⟹F0≡0,H0≡0,k=1⟹G0≡0.
Condition (2) is equivalent to(5.51)t(-F0(at),-G0(at),H0(at))=(F0(at),G0(at),H0(at))t.
The solutions we have always satisfy this condition. Condition (3) is equivalent to (5.52)t(cos⁡⁡θF0(1)+sin⁡⁡θG0(1),-sin⁡⁡θF0(1)+cos⁡⁡θG0(1),H0(1))=(F0(1),G0(1),H0(1))t,t(cos⁡⁡θF0(1)+sin⁡⁡θG0(1),sin⁡θF0(1)-cos⁡⁡θG0(1),-H0(1))=(-1)k⋅t(F0(1),G0(1),H0(1)).
(for all θ∈R). Since F0(1)=G0(1)=0, (5.52) are equivalent to (5.53)k=0⟹H0≡0.
But this condition holds if condition (5.50) is satisfied. We have obtained a result about the Shintani functions attached to the nonspherical principal series representation πσ1,ν. Note that since the transform ν1↦ν2,ν2↦ν1 does not change the eigenvalue of Casimir operator λ2 and changes the eigenvalue of gradient operator λσ1 to λσ2, this transform gives the result in case of σ=σ2. Similarly, the transform ν1↦-ν1,ν2↦-ν1+ν2 gives the result in case of σ=σ3. Summing up these results, we have the following theorem.Theorem 5.6.
Letη=sgn(k)(k∈{0,1}) be a signature of H defined by (5.38) and τ2 a three-dimensional tautological representation of K, and let Ψ=(F0,G0,H0)t∈Csgn(k),τ2∞(L)πσ1,ν be a Shintani function corresponding to the nonspherical principal series representation πσ1,ν of G. Then, the restriction of Ψ to A is given as follows.
(1)
In case ofk=0,
(a)
if2ν1-ν2≠0, Ψ≡0;(b)
if2ν1-ν2=0, one has
(5.54)(F0(at)G0(at)H0(at))=C⋅(0x1/2(1-x)(2+ν2)/4F21(12ν1+1,12ν2-12ν1+1,;2;x)0).(x=tanh2(2t),C:someconstant).(2)
In case ofk=1,
(a)
if(-ν1-ν2)(-ν1+2ν2)≠0, Ψ≡0;(b)
if(-ν1-ν2)(-ν1+2ν2)=0, one has
(5.55)(F0(at)G0(at)H0(at))=C⋅(-λσ1cosh⁡-1/2⁡(2t)(sinh⁡t)-1u(1-u)λσ1F21(λσ1+1,λσ1+1;2;u)0cosh⁡-1/2⁡(2t)(cosh⁡⁡t)-1(1-u)λσ1F21(λσ1+1,λσ1;1;u)).(u=tanh2t,C:someconstant).
Especially, in any case, one has
(5.56)dim⁡⁡Csgn⁡(k),τ2∞(L)πσ1,ν≤1.
The transform ν1↦ν2,ν2↦ν1 gives the result in case of σ=σ2 and the transform ν1↦-ν1,ν2↦-ν1+ν2 gives the result in case of σ=σ3.Next, we compute the Shintani functionsΨ∈Cη,τ2∞(L)πσ1,ν for general unitary character η=ηs,k under the assumption that 1,ν1,ν2 are linearly independent over Q. We have already known that the necessary condition of the existence of non zero Ψ is that the parameter s is one of 2ν1-ν2,2ν2-ν1, or -ν1-ν2, and we have already solved the differential equations in case of s=2ν1-ν2. Hereafter, we suppose that the parameter s is either 2ν2-ν1 or -ν1-ν2. We put w=(1-u)/(1+u), F̃2(w)=F̃1(u), H̃2(w)=H̃1(u) in (5.36), (5.37). Then we have (5.57)-(w2-16)sF̃2(w)+w(1-w)dH̃2dw(w)=λσ1F̃2(w),w(1+w)dF̃2dw(w)+(w2+16)sH̃2(w)=λσ1H̃2(w).
We put (5.58)F̃2(w)=∑n=0∞anwn+α,H̃2(w)=∑n=0∞cnwn+γ
(α,γ∈C,a0,c0≠0) in (5.57) and compute the power series solutions. By inserting these series into (5.57), we have (5.59)s6a0wα+wα+1Q1(w)+γc0wγ+wγ+1Q2(w)=λσ1a0wα+wα+1Q3(w),αa0wα+wα+1Q4(w)+s6c0wγ+wγ+1Q5(w)=λσ1c0wγ+wγ+1Q6(w),
where Qi(w)(i=1,…,6) are the analytic functions around w=0. By comparing the lowest terms in power series, easily we have α=γ (in this argument, we use the fact that 1,ν1,ν2 are linearly independent over Q carefully). Therefore, from (5.59), we have (5.60)(s6-λσ1)+γc0=0,αa0+(s6-λσ1)c0=0.
Since (a0,c0)≠(0,0), we have (5.61)(s6-λσ1)2-αγ=0.
By combining this and α=γ, we have (5.62)α=γ=±(s6-λσ1).
Hereafter, we put A=A(ν1,ν2)=(s/6)-λσ1. Then, F̃2(w) (resp., H̃2(w)) are expressed by the linear combination of some power series ∑n=0∞anwn+A, ∑n=0∞an′wn-A (resp., ∑n=0∞cnwn+A, ∑n=0∞cn′wn-A). That is, there exist common constants C+, C- such that (5.63)F̃2(w)=C+∑n=0∞anwn+A+C-∑n=0∞an′wn-A,H̃2(w)=C+∑n=0∞cnwn+A+C-∑n=0∞cn′wn-A.
By inserting (5.64)F̃2(w)=∑n=0∞anwn+A,H̃2(w)=∑n=0∞cnwn+A
into (5.57) and picking up the coefficients of wn+A, we have the following recurrence relations: (5.65)-s2an-1+Aan-(n+A-1)cn-1+(n+A)cn=0,(5.66)(n+A-1)an-1+(n+A)an+s2cn-1+Acn=0,
for all n≥0. Here, we assume that al=cl=0 if l<0. From (5.65), (5.66), easily we have cn=(-1)n+1an for all n≥0 by induction. Therefore, by inserting cn=(-1)n+1an,cn-1=(-1)nan-1 into (5.65), we have (5.67)(A+(n+A)(-1)n+1)an=(s2+(n+A-1)(-1)n)an-1.
Thus, we have (5.68)an={∏k=1ns/2+(n+A-1)(-1)kA+(k+A)(-1)k+1}a0,(5.69)cn=(-1)n+1{∏k=1ns/2+(n+A-1)(-1)kA+(k+A)(-1)k+1}a0.
Similarly, if the characteristic roots are α=γ=-A, by inserting (5.70)F̃2(w)=∑n=0∞an′wn-A,H̃2(w)=∑n=0∞cn′wn-A
into (5.57), we have (5.71)an′={∏k=1ns/2+(n-A-1)(-1)kA+(k-A)(-1)k+1}a0,cn′=(-1)n{∏k=1ns/2+(n-A-1)(-1)kA+(k-A)(-1)k+1}a0.
(for all n≥1). From (5.68), a2n and a2n-1 are expressed by(5.72)a2n=(s/4+A/2+1/2)n(-s/4+A/2)nn!(A+1/2)na0,a2n-1=-(s/4+A/2+1/2)n-1(-s/4+A/2)n(n-1)!(A+1/2)na0.
Here, for α∈C,n∈Z>0, we define (α)n=α(α+1)(α+2)⋯(α+n-1) and (α)0:=1. Therefore, if we normalize a0=1, we have (5.73)∑n=0∞a2nw2n=∑n=0∞(s/4+A/2+1/2)n(-s/4+A/2)nn!(A+1/2)nw2n=F21(s4+A2+12,-s4+A2;A+12;w2),∑n=1∞a2n-1w2n-1=--s/4+A/2A+1/2∑n=1∞(s/4+A/2+1/2)n-1(-s/4+A/2+1)n-1(n-1)!(A+3/2)n-1w2n-2⋅w=s/4-A/2A+1/2w∑n=0∞(s/4+A/2+1/2)n(-s/4+A/2+1)nn!(A+3/2)nw2n=s/4-A/2A+1/2wF21(s4+A2+12,-s4+A2+1;A+32;w2).
Thus, we have (5.74)∑n=0∞anwn+A=wA{F21(s4+A2+12,-s4+A2;A+12;w2)+s/4-A/2A+1/2wF21(s4+A2+12,-s4+A2+1;A+32;w2)}.
And since cn=(-1)n+1an, we have (5.75)∑n=0∞cnwn+A=wA{-F21(s4+A2+12,-s4+A2;A+12;w2)+s/4-A/2A+1/2wF21(s4+A2+12,-s4+A2+1;A+32;w2)}.
Similarly, by using (5.76)a2n′=(-s/4-A/2+1/2)n(s/4-A/2)nn!(-A+1/2)na0′,a2n-1′=-(-s/4-A/2+1/2)n-1(s/4-A/2)n(n-1)!(-A+1/2)na0′,
and cn′=(-1)nan′, if we normalize a0′=1, we have (5.77)∑n=0∞an′wn-A=w-A{F21(-s4-A2+12,s4-A2;-A+12;w2)-s/4-A/2-A+1/2wF21(-s4-A2+12,s4-A2+1;-A+32;w2)},∑n=0∞cn′wn-A=w-A{F21(-s4-A2+12,s4-A2;-A+12;w2)+s/4-A/2-A+1/2wF21(-s4-A2+12,s4-A2+1;-A+32;w2)}.
Therefore, F̃2(w) and H̃2(w) are expressed as follows:
(5.78)F̃2(w)=C+wA{F21(s4+A2+12,-s4+A2;A+12;w2)+s/4-A/2A+1/2wF21(s4+A2+12,-s4+A2+1;A+32;w2)}+C-w-A{F21(-s4-A2+12,s4-A2;-A+12;w2)-s/4-A/2-A+1/2wF21(-s4-A2+12,s4-A2+1;-A+32;w2)},H̃2(w)=C+wA{-F21(s4+A2+12,-s4+A2;A+12;w2)+s/4-A/2A+1/2wF21(s4+A2+12,-s4+A2+1;A+32;w2)}+C-w-A{F21(-s4-A2+12,s4-A2;-A+12;w2)+s/4-A/2-A+1/2wF21(-s4-A2+12,s4-A2+1;-A+32;w2)}.
We want the relation between C+ and C-. We can find the relation by using the regularity of the Shintani function and the asymptotic formula of Gaussian hypergeometric function F12. Now, since F0(at) and H0(at) are regular around t=0(⇔w=1) and since (5.79)F̃(t)=cosh⁡1/2⁡(2t)sinh⁡⁡tF0(at),H̃(t)=cosh⁡1/2⁡(2t)cosh⁡⁡tH0(at),F̃2(w)=F̃(t) and H̃2(w)=H̃(t) must be regular around w=1 and F̃2(w) must satisfy F̃2(1)=0. Since all hypergeometric functions appearing in the right-hand sides of (5.78) are in the form of 2F1(a,b;a+b;w2), to investigate the behavior of the right-hand sides of (5.78) around w=1, we use the following asymptotic formula.Formula 1
We have(5.80)2F1(a,b;a+b;z)=-Γ(a+b)(log⁡⁡(1-z)+ψ(a)+ψ(b)+2γ)Γ(a)Γ(b)(1+O(z-1))(z⟶1).
Here, γ is the Euler constant and ψ is defined by
(5.81)ψ(z)=∑k=1∞(1k-1k+z-1)-γ.We apply this formula to the right-hand sides of (5.78). Firstly, the coefficient of log(1-w2) of F̃2(w) equals (5.82)C+{-Γ(A+1/2)Γ(s/4+A/2+1/2)Γ(-s/4+A/2)-s/4-A/2A+1/2Γ(A+3/2)Γ(s/4+A/2+1/2)Γ(-s/4+A/2+1)}+C-{-Γ(-A+1/2)Γ(-s/4-A/2+1/2)Γ(s/4-A/2)+s/4-A/2-A+1/2Γ(-A+3/2)Γ(-s/4-A/2+1/2)Γ(s/4-A/2+1)}=C+⋅0+C-⋅0=0.
Therefore, F̃2(w) is regular around w=1 regardless of the values of C+, C-. Next, the coefficient of log(1-w2) of H̃2(w) equals (5.83)C+{Γ(A+1/2)Γ(s/4+A/2+1/2)Γ(-s/4+A/2)-s/4-A/2A+1/2Γ(A+3/2)Γ(s/4+A/2+1/2)Γ(-s/4+A/2+1)}+C-{-Γ(-A+1/2)Γ(-s/4-A/2+1/2)Γ(s/4-A/2)-s/4-A/2-A+1/2Γ(-A+3/2)Γ(-s/4-A/2+1/2)Γ(s/4-A/2+1)}=2Γ(A+1/2)Γ(s/4+A/2+1/2)Γ(-s/4+A/2)C+-2Γ(-A+1/2)Γ(-s/4-A/2+1/2)Γ(s/4-A/2)C-.
Since this coefficient must be 0, we have (5.84)C+=Γ(-A+1/2)Γ(-s/4-A/2+1/2)Γ(s/4-A/2),C-=Γ(A+1/2)Γ(s/4+A/2+1/2)Γ(-s/4+A/2)
up to (the same) constant multiples. Note that we can easily verify that F̃2(1)=0 for these C+, C- by using formulas Γ(z+1)=zΓ(z),ψ(z+1)=ψ(z)+1/z. Now, we have completely determined F̃2(w),H̃2(w). We have
(5.85)F̃2(w)=Γ(-A+1/2)Γ(-s/4-A/2+1/2)Γ(s/4-A/2)wA×{F12(s4+A2+12,-s4+A2;A+12;w2)+s/4-A/2A+1/2wF12(s4+A2+12,-s4+A2+1;A+32;w2)}+Γ(A+1/2)Γ(s/4+A/2+1/2)Γ(-s/4+A/2)w-A×{F21(-s4-A2+12,s4-A2;-A+12;w2)-s/4-A/2-A+1/2wF12(-s4-A2+12,s4-A2+1;-A+32;w2)},H̃2(w)=Γ(-A+1/2)Γ(-s/4-A/2+1/2)Γ(s/4-A/2)wA×{-F12(s4+A2+12,-s4+A2;A+12;w2)+s/4-A/2A+1/2wF12(s4+A2+12,-s4+A2+1;A+32;w2)}+Γ(A+1/2)Γ(s/4+A/2+1/2)Γ(-s/4+A/2)w-A×{F12(-s4-A2+12,s4-A2;-A+12;w2)+s/4-A/2-A+1/2wF12(-s4-A2+12,s4-A2+1;-A+32;w2)}
up to (the same) constant multiples. Here, A=A(ν1,ν2)=s/6-λσ1, λσ1=-(1/3)(2ν1-ν2), w=(1-u)/(1+u)=(1-tanh2t)/(1+tanh2t), F̃2(w)=F̃(t)=cosh1/2(2t)sinhtF0(at), H̃2(w)=H̃(t)=cosh1/2(2t)coshtH0(at). By the same argument we have done when η is the signature, we can easily verify that conditions (1), (2), and (3) in (3.8) are equivalent to condition (5.50). We have already computed F0(at), G0(at), and H0(at) in any case. Summing up, we obtain the following theorem.Theorem 5.7.
Assume that1,ν1,ν2 are linearly independent over Q. Let η=ηs,k be a unitary character of H defined by (3.1) and σ=σ1 the character of M. Then, the necessary condition of the existence of the nontrivial Shintani functions attached to the nonspherical principal series representation πσ1,ν is that
(5.86)k=0,s=2ν1-ν2
or
(5.87)k=1,s=-ν1-ν2or-ν1+2ν2.
That is, if the condition above is not satisfied, one has
(5.88)Cη,τ2∞(L)πσ1,ν={0}.
Let Ψ(at)=(F0(at),G0(at),H0(at))t∈Cη,τ2∞(L)πσ1,ν|A and suppose that the condition above is satisfied. Then,(1)
ifk=0 and s=2ν1-ν2, one has(5.89)(F0(at)G0(at)H0(at))=C⋅(0x1/2(1-x)(2+ν2)/4F12(12ν1+1,12ν2-12ν1+1,;2;x)0).(x=tanh2(2t),C:someconstant);(2)
ifk=1 and s=-ν1-ν2 or -ν1+2ν2, one has(5.90)(F0(at)G0(at)H0(at))=C⋅(cosh⁡-1/2(2t)sinh⁡-1tF̃2(w)0cosh⁡-1/2(2t)cosh⁡-1tH̃2(w)),whereC is some constant and F̃2(w) and H̃2(w) are the functions given by (5.85). Especially, in any case, one has(5.91)dim⁡Cη,τ2∞(L)πσ1,ν≤1.
The transform ν1↦ν2,ν2↦ν1 gives the result in case of σ=σ2, and the transform ν1↦-ν1,ν2↦-ν1+ν2 gives the result in case of σ=σ3.Remark 5.8.
By using the relation1-x=w2 and the formulas of the hypergeometric function
(5.92)Γ(α)Γ(β)Γ(γ)F12(α,β;γ;z)=Γ(α)Γ(β-α)Γ(γ-α)(1-z)-αF12(α,γ-β;α-β+1;11-z)+Γ(β)Γ(α-β)Γ(γ-β)(1-z)-βF12(β,γ-α;β-α+1;11-z),F21(α,β,γ;z)=(1-z)-αF12(α,γ-β;γ;zz-1)=(1-z)γ-α-βF12(γ-α,γ-β;γ;z),
we can rewrite F̃2(w), H̃2(w) in Theorem 5.7 as functions in x=tanh2(2t). We put F̃2(w)=F3(x), H̃2(w)=H3(x). Then, we have
(5.93)F3(x)=(-s4+A2){-(1-x)A/2F12(s4+A+12,-s4+A2;1;x)+(1-x)(A+1)/2F12(s4+A+12,-s4+A2+1;1;x)},H3(x)=(-s4+A2){(1-x)A/2F12(s4+A+12,-s4+A2;1;x)+(1-x)(A+1)/2F12(s4+A+12,-s4+A2+1;1;x)}.
These computations are due to Professor T. Ishii.



---

*Source: 842806-2011-12-26.xml*