On the restriction of Deligne-Lusztig characters

results of Hagedorn gave me the courage to attempt such calculations for general groups and to obtain closed multiplicity formulas for orthogonal groups. It is a pleasure to thank Dick Gross for initiating the work in [12] which led to this paper, for helpful remarks on an earlier version, and for aquainting me with Hagedorn's thesis. The referee read the original version of this paper with care and insight, made valuable comments and simplified some of the arguments. In particular, the proof of Lemma 3.1 given below is due to the referee and is much shorter than the original one. Some general notation: The cardinality of a finite set X is denoted by \X\. Equivalence classes are generally denoted by [ ], sometimes with ornamentation. If g is an element of a group G, we write Ad(g) for the conjugation map Ad(g) : x i? gxg~x, and also write 9T := gTg~x for a subgroup T C G. The center of G is denoted Z(G) and the centralizer of g G G is denoted CG(g). We write ( , )# for the pairing on the space of class functions on a finite group H, for which the irreducible characters of H are an orthonormal basis. l?G,Gf D H are finite overgroups of H and i?, ip' are class functions on G, G' respectively, then (^^')h is understood to mean (^\h^'\h)h^ where \h denotes restriction to H. 2. Remarks on maximal tori Let G be a connected reductive algebraic J-group. We assume G is defined over f and has Frobenius F. If T is a maximal torus in G we denote its normalizer in G by NG(T) and write WG(T) = NG(T)/T for the Weyl group of T in G. If T is F-stable, we have W(T)F = NG(T)F/TF, by the Lang-Steinberg theorem. The reduction formula for Deligne-Lusztig characters (recalled in section 4 below) involves a sum over the following kind of subset of GF. Fix an F-stable maximal torus T C G, and let s be a semisimple element in GF. We must sum over the set NG(s,T)F:={j GF: *" T}. Note that NG(s, T)F, if nonempty, is a union of GF x NG(T)F double cosets, where Gs := CG(s)? is the identity component of the centralizer CG(s) of s in G. To say that s^ G T is to say that 7TcGs, so determining the GF x NG(T)F double cosets in NG(s,T)F amounts to determining the Gf-conjugacy classes of F-stable maximal tori in Gs which are contained in a given GF-conjugacy class. Such classes of tori are parameterized by twisted conjugacy classes in Weyl groups of Gs and G. The aim of this section is to parameterize the GF x NG(T)F double cosets in NG(s, T)F in terms of the fiber of a natural map between twisted conjugacy classes in the Weyl groups of Gs and G. This parameterization will be fundamental to our later calculations with Deligne-Lusztig characters. We begin by recalling the classification of F-stable maximal tori in G. See [5, chap. 3] for more details in what follows. Fix an F-stable maximal torus To in G contained in an F-stable Borel subgroup of G, and abbreviate NG = Ng(Tq), Wg = WG(T0). This content downloaded from 207.46.13.149 on Mon, 03 Oct 2016 06:07:19 UTC All use subject to http://about.jstor.org/terms ON THE RESTRICTION OF DELIGNE-LUSZTIG CHARACTERS 577 Let T(G) denote the set of all F-stable maximal tori in G. Then T(G) is a finite union of GF-orbits. For any T G T(G), let [T\g := {7T : 7 G GF} denote the GF-orbit of T. There is g G G such that T = 9T0. Since T is F-stable, we have g~1F(g) G AfcThis gives an element w := g~1F(g)T0 G WG. The map Ad(#)t = <7?#_1 is an f-isomorphism Ad(g):{T0,wF)?+(T,F), where the second component denotes the action of Frobenius under an f-structure. For any finite group A with F-action, we let H1 (F, A) denote the set of F conjugacy classes in A. These are the orbits of the action of A on itself via (a, b) i? abF(a)~1. Let [b] G H1 (F, A) denote the F-conjugacy class of an element b G A. For g,T,w as above, the F-conjugacy class of w is independent of the choice of g. Hence we have a well-defined class c\(T,G):=[w]eH1(F,WG). For each u G H1 (F, WG), the set %(G):={TeT(G): cl(T,G)=u;} is a single GF-orbit in T(G), and all GF-orbits are of this form. Thus, the partition of the set of F-stable maximal tori into GF-orbits is given by T(G) = JJ %{G). Let s e GF be semisimple, and let Ts be an F-stable maximal torus of Gs contained in an F-stable Borel subgroup of Gs, and let WGs be the Weyl group of Ts in Gs. The partition of T(GS) into Gf-orbits is given, as above, by T(GS) = ?J %(GS). veH^F^Wos) If T e T(G), the set of F-stable maximal tori in Gs which are GF-conjugate to T is a finite union (possibly empty) of Gf-orbits. We want to describe this union in terms of F-conjugacy classes in WGs. That is, given u G H1 (F, WG), we have (2.1) TUJ(G)nT(Gs) = j} %(GS) veMw for some subset M^ ? Hl(F, WGs), and our task is to find Mw. The first point is that Ts is generally not contained in an F-stable Borel subgroup of G. Let g G G be such that 9TS = T0, and let ys := gF(g)~l have image ys G WG. Then d(Ts,G) = [ys}eH1(F,WG), and Ad(#) is an f-isomorphism Ad(g):(Ts,F)?+ (T0,ysF). Now To is also a maximal torus in Ad(^)Gs, whose Weyl group WG3 := Ad(g)WGa is a subgroup of WG, stable under Ad(ys) o F. This content downloaded from 207.46.13.149 on Mon, 03 Oct 2016 06:07:19 UTC All use subject to http://about.jstor.org/terms