# Comptes Rendus
Mathématique

Irene Spelta

Shimura subvarieties via endomorphisms

Volume 360 (2022), p. 1117-1124

https://doi.org/10.5802/crmath.369

<figure><img src="image_1.png" /></figure>

This article is licensed under the
CREATIVE COMMONS ATTRIBUTION 4.0 INTERNATIONAL LICENSE.
http://creativecommons.org/licenses/by/4.0/

<figure><img src="image_2.png" /></figure>

Les Comptes Rendus. Mathématique sont membres du
Centre Mersenne pour l'édition scientifique ouverte
www.centre-mersenne.org

e-ISSN : 1778-3569
Algebraic geometry / Géométrie algébrique

# Shimura subvarieties via endomorphisms

Irene Spelta<sup>a</sup>

<sup>a</sup> Università degli Studi di Pavia, Dipartimento di Matematica, Via Ferrata 5, 27100 Pavia, Italy

E-mail: irene.spelta@unipv.it

**Abstract.** We prove that there exist two families in $M_2, M_3$ of non-Galois covers of the projective line whose Jacobians trace out Shimura subvarieties of $\mathcal{A}_2, \mathcal{A}_3$. They provide the first two explicit examples of Shimura subvarieties obtained by means of Jacobians carrying non-trivial endomorphisms not directly induced by the automorphisms of the curves. We also obtain a new example of a positive dimensional family of special Pryms in $\mathcal{A}_4^0$.

**2020 Mathematics Subject Classification.** 14H10, 14H30, 14H40, 14G35.

**Funding.** The author was partially supported by MIUR PRIN 2017 "Moduli spaces and Lie Theory", by MIUR "Programma Dipartimenti di Eccellenza (2018-2022) - Dipartimento di Matematica F. Casorati Università degli Studi di Pavia" and by INdAM (GNSAGA). In particular, the post-doc position of the author was funded by INdAM (GNSAGA).

Manuscript received 3 December 2021, revised 1 February 2022 and 16 March 2022, accepted 22 April 2022.

# 1. Introduction

Let $M_g$ be the coarse moduli space of smooth projective genus $g$ curves and $\mathcal{A}_g$ be that of principally polarized $g$-dimensional abelian varieties. The Torelli map $t: M_g \to \mathcal{A}_g$ sends the class $[C] \in M_g$ to the class of the Jacobian variety JC endowed with the polarization induced by the cup product. By the Torelli theorem, $t$ is injective. Moreover, it is an immersion outside the hyperelliptic locus (see [25]). For this reason, it is natural to investigate the Zariski closure of the image $t(M_g)$, usually referred to as the *Torelli locus*.

The study of the Torelli locus has been addressed from different perspectives in the literature: one among them concerns Shimura (often called “special”) subvarieties $S \subseteq \mathcal{A}_g$ which are generically contained in it, i.e. $S \subseteq \bar{t}(M_g)$ and $S \cap t(M_g) \neq \emptyset$. Shimura varieties are by definition Hodge loci for the natural variation of Hodge structure on $\mathcal{A}_g$. They are totally geodesic subvarieties of $\mathcal{A}_g$, i.e. they are images of totally geodesic submanifolds of the Siegel space $\mathcal{S}_g$ ([23]). We focus on special subvarieties of PEL type. Given $J_o \in \mathcal{S}_g$, set $F := \text{End}_{\mathbb{Q}}(A_{J_o}) = \{f \in \text{End}(\mathbb{Q})^{2g} : J_o f = f J_o\}$. We have:

**Definition 1.** The PEL type special subvariety $S(F)$ is defined as the image in $\mathcal{A}_g$ of the connected component containing $J_o$ of the set $\{J \in \mathcal{S}_g : F \subseteq \text{End}_{\mathbb{Q}}(A_J)\}$.

As the genus $g$ grows, one expects the Torelli embedding to be more and more curved with respect to the locally symmetric geometry induced by the Siegel space on $\mathcal{A}_g$. In terms of special varieties, this expectation is formulated as follows.
**Conjecture (Coleman-Oort).** For $g$ large enough, there do not exist positive dimensional special subvarieties of $\mathcal{A}_g$ generically contained in the Torelli locus.

Up to genus 7, there are examples of positive dimensional (PEL) special varieties generically contained in the Torelli locus (see [6, 8–10, 13, 16, 17, 20–22]). They were found by considering families $Z$ of Galois covers $C \to C/G$ and requiring that $\dim Z = \dim(\mathrm{Sym}^2 H^0(C, \omega_C))^G$ (see condition (*) of [9, Theorem 3.9] and of [13, Theorem 3.7]).

In [5], the authors formulated an analogue of the Coleman-Oort conjecture for the Prym loci, namely they asked if there exist positive dimensional special subvarieties generically contained in $\mathcal{P}(\mathcal{R}_{g,0})$ and $\mathcal{P}(\mathcal{R}_{g,2})$. Further generalizations are discussed in [11], [12], and [14]. As usual, $\mathcal{R}_{g,r}$ denotes the moduli space parametrizing double covers $\pi : C' \to C$ of genus $g$ curves $C$ ramified in $r \ge 0$ points and $\mathcal{P}$ is the corresponding Prym map. This type of problem naturally arose since the second fundamental form of the Prym map has very similar behaviour to that of the Torelli map. Many examples of positive dimensional families of (PEL) special Pryms were found under a certain (again sufficient) numerical condition (condition B of [5], [11], [12], [14], see Definition 15).

We stress that all these examples of special varieties have been obtained considering families of Jacobians and Pryms whose automorphisms are induced by those of the underlying curves.

Very little is known about the existence of families of curves whose Jacobians are acted on by a large ring of endomorphisms not induced by automorphisms of the curves. The first examples are due to Tautz-Top-Verberkmoes, Mestre, and Brumer. Then Ellenberg generalized these works (see [7] and the references therein). In particular, for every odd prime number $p \ge 7$, he found a 3-dimensional family of curves of genus $(p-1)/2$ whose endomorphism algebra contains the real cyclotomic field $\mathbb{Q}(\xi_p + \xi_p^{-1})$, with $\xi_p^p = 1$.

This is our starting point. Indeed, in this note, we provide the first two explicit examples of special subvarieties of $\mathcal{A}_g$ which are generically contained in the Torelli locus and whose Jacobians have extra automorphisms. Our examples occur when $g = 2, 3$ and have dimensions $2, 3$, respectively. Actually, when $g = 2, 3$, we have $t(\mathcal{M}_g) = \mathcal{A}_g$, thus all (infinitely many) special subvarieties of $\mathcal{A}_g$ are generically contained in the Torelli locus. Furthermore, we know that all of them are of PEL type, i.e. they are given by the existence of extra automorphisms. But this is only an abstract approach. Our contribution consists of explicitly writing down two families of curves whose Jacobians trace out the first two concrete examples.

We exploit the families already studied by Albano and Pirola in [1] to disprove a conjecture of Xiao on the relative irregularity of a fibration. In details, they are families of cyclic étale covers of hyperelliptic genus $g$ curves $\pi : C \to H := C/\langle\sigma\rangle$ of prime odd degree $p$. By construction, the curves $C$ are dihedral covers of $\mathbb{P}^1$ and the lift of the hyperelliptic involution of $H$ to $C$ gives an intermediate quotient $C \xrightarrow{2:1} C_0$ of genus $g(C_0) = g_{C_0} = \frac{1}{2}(g-1)(p-1)$. Moreover, the Prym variety $P(C, H)$ is isomorphic to $JC_0 \times JC_0$. The curves $C_0$ map $p : 1$ to $\mathbb{P}^1$, but the cover is non-Galois. Finally, these curves are special in moduli. Indeed the automorphism $\sigma$ induces a non-trivial automorphism of $JC_0$ (not preserving the polarization on $JC_0$). It follows that $\mathrm{End}_0(JC_0)$ contains $F_p := \mathbb{Q}(\xi_p + \xi_p^{-1})$. For details on such covers, we refer to [27].

This construction gives a map

$$ \psi : \mathcal{R}\mathcal{H}_g(p) \to \mathcal{M}_{g_{C_0}} $$

from the moduli space $\mathcal{R}\mathcal{H}_g(p)$ parametrizing covers $\pi : C \to H$, to that of curves of genus $g_{C_0}$.

Our results are the following:

**Theorem 2.** For $g=2$ and $p=5$, resp. $p=7$, the closure of the image of $t \circ \psi$ is the special subvariety $S(F_5) \subset \mathcal{A}_2$, resp. $S(F_7) \subset \mathcal{A}_3$, containing it. It has dimension 2, resp. 3.

**Corollary 3.** The special subvarieties $S(F_5) \subset \mathcal{A}_2$ and $S(F_7) \subset \mathcal{A}_3$ do not arise from Galois covers $C \to C/G$ satisfying the condition (*).
Notice that they are the first two explicit examples of this type, namely they are the first examples of special subvarieties traced out without using the condition (*).

There are also interesting consequences on the image in the Prym locus of the families analysed in Theorem 2. Let $\mathcal{A}_g^\delta$ be the moduli space of $g$-dimensional abelian varieties with polarization of type $\delta$. We have the following:

**Theorem 4.** *The Prym varieties of the two families of Theorem 2 yield Shimura subvarieties of $\mathcal{A}_4^\delta$, resp. $\mathcal{A}_6^\delta$, of dimension 2, resp. 3.*

Finally, we show that the family with $g = 2$ and $p = 5$ does not satisfy the aforementioned condition B. Therefore, it is not one of the examples found in [14]. In particular, we have the following

**Corollary 5.** *The condition B of [5], [11], [12], [14] is sufficient, but not necessary, to construct positive dimensional families of special Pryms.*

### Acknowledgements

The author thanks G.P. Pirola for his inspiring question, A. Ghigi for the right reference at the right time and P. Frediani for her many valuable comments. Moreover, she would like to thank B. Moonen for a brilliant observation that refined the exposition. Finally, she is indebted to the anonymous referee for the very careful reading and the significant improvements.

## 2. Hyperelliptic covers and endomorphisms of Jacobians

In this section, we overview known results concerning cyclic covers of hyperelliptic curves and their associated Prym varieties. Moreover, we describe the endomorphism algebras of the Jacobians of the curves occurring in this construction.

Let $H$ be a smooth hyperelliptic genus $g$ curve and $\pi : C \to H$ be an unramified cyclic cover of odd prime order $p$. By the general theory of cyclic étale covers, $\pi$ corresponds to a pair $(H, L)$, where $L \in \text{Pic}^0(H)$ satisfies $L^p = \mathcal{O}_H$. Let $\mathcal{RH}_g(p)$ be the coarse moduli space of pairs $(H, L)$. We will call such elements *hyperelliptic covers*. By the Riemann–Hurwitz formula we get $g(C) = g_C = p(g-1) + 1$. Let us denote by $\sigma$ a fixed generator of the cyclic group $\mathbb{Z}/p$ acting on $C$. We borrow from [27] in what follows.

**Proposition 6.** *Let $(H, L)$ be an element of $\mathcal{RH}_g(p)$. The hyperelliptic involution of $H$ lifts to an involution $\tau$ on $C$ and the cover $C \to \mathbb{P}^1$ is Galois with Galois group $G = D_p$ generated by $\sigma$ and $\tau$ (here $D_p$ stands for the dihedral group of order $2p$).*

It is well-known that if $C \to H$ is an unramified abelian cover of a hyperelliptic curve, then the hyperelliptic involution lifts to an involution $\tau$ on $C$. In our situation, $\sigma$ and $\tau$ generate the automorphism group $G \cong D_p$ of $C$. This means that we have the following diagram:

$$ \begin{array}{ccc} C & \xrightarrow{p:1} & H \\ \bigdownarrow_{2:1} & & \bigdownarrow_{2:1} \\ C_0 := C/\tau & \xrightarrow{p:1} & \mathbb{P}^1 \cong C/D_p. \end{array} \qquad (1) $$

The cover $C_0 \xrightarrow{p:1} \mathbb{P}^1$ is non-Galois: over every branch point of $H \to \mathbb{P}^1$ the map $C_0 \xrightarrow{p:1} \mathbb{P}^1$ has $1 + (p-1)/2$ points. One of these points is étale while the others have order 2. The Galois closure of such a map is $C \to C_0 \to \mathbb{P}^1$. By the Riemann–Hurwitz formula, we have $g(C_0) = g_{C_0} = (g-1)(p-1)/2$.
Every cover $(H, L) \in \mathcal{RH}_g(p)$ determines an abelian variety $P(H, L)$ defined as the connected component containing the origin of the kernel of the Norm map $Nm: JC \to JH$. It has dimension $(g-1)(p-1)$ and it carries a polarization $\Xi$, induced by that of $JC$, of type $\delta = (1, \dots, 1, p, \dots, p)$, with 1 repeated $(p-2)(g-1)$ times and $p$ repeated $(g-1)$ times. Usually, it is referred to as the *Prym variety* of the cover $C \to H$. In the case of hyperelliptic covers, we have the following:

**Theorem 7 ([27, §6, Lemma, Theorem 1]).** There is an isomorphism of polarized abelian varieties

$$P(H, L) \cong JC_0 \times JC_0. \qquad (2)$$

In particular, $JC$ is isogenous to $JH \times JC_0 \times JC_0$.

Furthermore, the endomorphism $\sigma + \sigma^{-1}$ of $JC$ gives a non-trivial endomorphism of $JC_0$ for $p > 3$ and $\sigma^{\frac{p+1}{2}} + (\sigma^{-1})^{\frac{p+1}{2}}$ is an automorphism of $JC_0$ which does not preserve the polarization on $JC_0$.

**Definition 8.** Let $F$ be a totally real number field, i.e. $F$ is generated over $\mathbb{Q}$ by a root of a polynomial with integral coefficients, whose roots are all real. A curve $X$ has real multiplication by $F$ if $g(X) = [F : \mathbb{Q}]$ and if $F \to \text{End}_0(JX)$.

**Corollary 9.** When $g = (p-1)/2$ and $p > 3$ the curve $C_0$ has real multiplication by $\mathbb{Q}(\xi_p + \xi_p^{-1})$, where $\xi_p^p = 1, \xi_p \neq 1$.

Using Proposition 6, we define the morphism

$$\psi : \mathcal{RH}_g(p) \to \mathcal{M}_{gC_0}$$

which sends the isomorphism class of a cover $C \to H$ to the isomorphism class of the associated quotient curve $C_0$. This is well-defined since any two lifts of the hyperelliptic involution are conjugated in Aut($C$). As in [1, Remark 2.8], composing with $t$, we get a map $T : \mathcal{RH}_g(p) \to \mathcal{A}_{(g-1)(p-1)}$ sending $(H, L)$ to $JC_0 \times JC_0$ with the product polarization.

On the other hand we can consider the Prym map

$$\mathcal{P} : \mathcal{RH}_g(p) \to \mathcal{A}_{(g-1)(p-1)}^\delta, \quad (H, L) \mapsto (P(H, L), \Xi). \qquad (3)$$

By Theorem 7, $P(H, L) \cong JC_0 \times JC_0$ and by Torelli Theorem $t$ is injective. Since an abelian variety has at most a countable number of polarizations, we can observe that the fibres of $\mathcal{P}$ have the same dimension of those of $\psi$. They were described by the following

**Proposition 10 (Albano–Pirola, [1]).** The map $\psi$ has finite fibres if and only if $p \ge 7$ or $p = 5$ and $g \ge 3$ or $p = 3$ and $g \ge 5$.

**Remark 11.** In [1], the authors used Proposition 10 to look for positive dimensional fibres of $\psi$. They showed that there are three cases for $(g, p)$ where an irreducible component of the fibre $\psi^{-1}(C_0)$ gives families of curves $C$ (as in diagram (1)) disproving Xiao's conjecture on the relative irregularity of a fibration. Actually, the first Xiao fibration, namely the first fibration which violates Xiao's conjecture, was constructed by Pirola in [26]. This family is very interesting. Indeed, in [13] it is shown that it yields a Shimura 3fold in $\mathcal{A}_4$ and in [10] that, through its Prym map, it is fibred in infinitely many totally geodesic curves, countably many of which are Shimura. The families of [26] and of [1] were also obtained in [15] where the authors study Xiao fibrations considering Galois covers $C \to H$ under less restrictive assumptions (i.e. any degree and any value for $r$).

Although we no longer deal with Xiao's conjecture, the families described in [1] turn out to be suitable for our purposes. Indeed up to now we have shown that, when $p > 3$, the construction of diagram (1) produces families of curves $C_0$ of genus $(g-1)(p-1)/2$ with $\mathbb{Q}(\xi_p + \xi_p^{-1}) \to \text{End}_0(JC_0)$.

When the fibres of $\psi$ are finite, these families have dimension $2g-1$, i.e. the dimension of $\mathcal{RH}_g(p)$. In particular, when $g = 2$ and $p \ge 7$ we recover the 3-dimensional families of curves
whose Jacobians are acted on by a large ring of endomorphisms found in case 1 of the main Theorem of [7]. Instead, when the fibres of $\psi$ are positive dimensional, we consider the family of Albano and Pirola with data $g = 2$, $p = 5$ because it has non-trivial endomorphism algebra. As observed in [1, Proposition 2.7], this family is 2-dimensional. Indeed, we have $\dim \mathcal{R}\mathcal{H}_2(5) = \dim \mathcal{M}_2 = 3$. Since, by Theorem 7, $\mathcal{J}\mathcal{C}_0$ has a non-trivial endomorphism, the curve $C_0$ is not general in moduli. Therefore the image of $\psi$, namely the family we are interested in, has dimension at most 2.

In the next section, we prove that there are two cases where the Jacobians of the curves $C_0$ of diagram (1) yield special subvarieties of $\mathcal{A}_{gC_0}$ generically contained in the Torelli locus. Moreover, thanks to Theorem 7, we obtain interesting consequences on Prym loci too.

### 3. Shimura Subvarieties

Let $\Lambda$ be a rank 2g lattice and $Q: \Lambda \times \Lambda \to \mathbb{Z}$ an alternating form of type $(1, 1, \dots, 1)$. Let $K$ be a field with $Q \subseteq K \subseteq \mathbb{C}$ and set $\Lambda_K := \Lambda \otimes_{\mathbb{Z}} K$. Denote by $A_J$ the quotient $\Lambda_{\mathbb{R}}/\Lambda$ provided with the complex structure $J$ and the polarization $Q$. It yields an element in $\mathcal{A}_g$. As usual, we will interpret $\mathcal{A}_g$ as the orbit space of the action of the symplectic group $Sp(\Lambda, Q)$ on the Siegel space $\mathcal{S}_g$. There is a natural variation of rational Hodge structure on $\mathcal{A}_g$: it corresponds to the decomposition of $\Lambda_C$ into $\pm i$ eigenspaces for $J$.

**Definition 12.** A Shimura (special) subvariety of $S \subseteq \mathcal{A}_g$ is by definition a Hodge locus of the natural variation of Hodge structure on $\mathcal{A}_g$ described above.

As already said, we will focus on special subvarieties of PEL type; the name comes from the fact that they can be described in terms of abelian varieties with a polarization, given endomorphisms and a level structure. From Definition 1: fix $J_0 \in \mathcal{S}_g$ and set $F := \text{End}_Q(A_{J_0})$. Then the PEL type special subvariety $S(F)$ is the image in $\mathcal{A}_g$ of the connected component containing $J_0$ of the set $\{J \in \mathcal{S}_g : F \subseteq \text{End}_Q(A_J)\}$. To be more precise, one should have to write $S_{J_0}(F)$. To lighten the notation, we avoid the subscript: from our context it will be clear which PEL we want to study.

In view of construction (1) and of Corollary 9, it is natural to investigate the subvarieties of $\mathcal{A}_{gC_0}$ which are described by the Jacobians $\mathcal{J}\mathcal{C}_0$. Our goal is to compare them with the PEL type special subvarieties $S(F_p)$ with $F_p = \mathbb{Q}(\xi_p + \xi_p^{-1})$, $p \ge 5$, in order to verify when they coincide.

**Proof of Theorem 2.** Let us start with $g = 2$, $p = 5$ (precisely with one of the families studied in [1]). The genus 2 curves $C_0$ have real multiplication by $F_5 = \mathbb{Q}(\xi_5 + \xi_5^{-1})$. Indeed $F_5$ is a totally real field with $e := [F_5 : \mathbb{Q}] = 2 = g_{C_0}$. Let $S(F_5)$ be the special subvariety of $\mathcal{A}_2$ of PEL type defined by $F_5$ as in Definition 1. It arises as the quotient of a product of $g_{C_0}$ copies of the Siegel space $\mathcal{S}_{g_{C_0}/e} = \mathcal{S}_1$, as shown in [3, Chapter 9, §2]. Therefore

$$ \dim S(F_5) = \frac{g_{C_0}(g_{C_0} + e)}{2e} = 2. $$

Now let $X$ be the closure of the locus of $\mathcal{J}\mathcal{C}_0$ for $C_0$ varying in the family of Albano and Pirola with $g = 2$ and $p = 5$. Clearly $X$ is an irreducible subvariety of $S(F_5)$ (the irreducibility follows from that of $\mathcal{R}\mathcal{H}_g(p)$ claimed in [1, Proposition 2.7]). Moreover $\dim X = \dim S(F_5) = 2$ thanks to [1, Proposition 4.2]. Thus $X$ must coincide with the PEL subvariety of $\mathcal{A}_2$ identified by $F_5$. In particular, it is special.

Now take $g = 2$, $p = 7$. In this case we have

$$ F_7 = \mathbb{Q}(\xi_7 + \xi_7^{-1}), \quad e = g_{C_0} = 3. $$

The same computation as in the previous situation gives $\dim S(F_7) = 3$. By Proposition 10, $\psi$ has finite fibres when $p = 7$. Thus $\mathcal{J}\mathcal{C}_0$ varies in a 3-dimensional family and so we get the PEL subvariety of $\mathcal{A}_3$ identified by $F_7$. $\square$
**Proof of Corollary 3.** We need to show that the special subvarieties $S(F_5)$ and $S(F_7)$ are not obtained using families of Galois covers of $\mathbb{P}^1$ satisfying the sufficient condition $(*)$. In other words, we need to check that our family (1), resp. (2), is different from those obtained in [9] and in [13]. Genus and dimension tell us that they may coincide only with family (26), resp. family (27), of [9]. Family (26) is the bielliptic locus in genus 2 and family (27) is strictly contained in the bielliptic locus in genus 3. The generic curve of these two families carries a completely decomposable Jacobian. Using Mumford's list of non-trivial endomorphism rings of abelian varieties ([24]) and comparing the endomorphism algebras of families (26), (27) with the ones of our families (1) and (2), we conclude. $\square$

**Remark 13.** [4, Theorem 1.1] states that families (26) and (1) are the unique irreducible 2-dimensional subvarieties of $\mathcal{A}_2$ whose generic point has non-trivial endomorphisms, namely $\mathbb{Z} \subsetneq \text{End}_0(JC)$.

**Remark 14.** For other values of $g$ and $p$, the image of $\psi$ still produces subvarieties of $t(\mathcal{M}_{g_{c_0}})$. In these cases, such subvarieties are strictly contained in the special subvarieties of PEL type arising from the corresponding totally real field $F_p = \mathbb{Q}(\xi_p + \xi_p^{-1})$. The same occurs for all the families constructed in [7, Main Theorem] (except for the one corresponding to our family (2)). This agrees with the results discussed in [18] for $g > 4$ and in [2] for $g = 4$. Together these two works state that if $g \ge 4$ and $S \subseteq \mathcal{A}_g$ is a special subvariety of PEL type obtained from a totally real field of degree $g$, then $S$ is not generically contained in the Torelli locus. Therefore, letting $g$ and $p$ vary, we should not expect the existence of other families as (1) and (2) yielding special varieties of PEL type generically contained in higher genus Torelli locus. Notice that our examples prove that the bound $g \ge 4$ is sharp.

Let us now look at the loci described by the Prym varieties of the two families studied in Theorem 2, namely at the families of $P(C, H)$ in $\mathcal{P}(\mathcal{RH}_g(p)) \subset \mathcal{A}_{(g-1)(p-1)}^\delta$.

**Proof of Theorem 4.** Just using the isomorphism (2), the families (1) and (2) of Theorem 2 yield two special subvarieties of $\mathcal{A}_4^\delta$, resp. $\mathcal{A}_6^\delta$ (generically contained in $\mathcal{P}(\mathcal{RH}_2(5))$, resp. $\mathcal{P}(\mathcal{RH}_2(7))$. These varieties have dimension 2, 3 respectively. $\square$

Let us now interpret families (1) and (2) as Prym data $(D_p, \theta, \sigma)$ (we borrow the notation from [5], [11], [12]): $D_p$ is the Galois group acting on $C$ with quotient $\mathbb{P}^1$, $\sigma$ gives the intermediate quotient $C \to H = C/\langle\sigma\rangle$ and $\theta$ is the monodromy map. Setting $V := H^0(C, \omega_C)$ and letting $\sigma$ act on $V$, we can decompose

$$V = V_+ \oplus V_- \quad \text{where} \quad V_+ \cong H^0(C, \omega_C)^{\langle\sigma\rangle} = H^0(H, \omega_H) \quad \text{and} \quad V_- \cong \bigoplus_{i=1}^{p-1} H^0(H, \omega_H \otimes L^i).$$

By [19, Proposition 4.1], the codifferential of the Prym map (3) at a point $(H, L) \in \mathcal{RH}_g(p)$ can be identified with the composition

$$\text{Sym}^2 V_- \xrightarrow{m} H^0(C, \omega_C^{\otimes 2}) \to H^0(H, \omega_H^{\otimes 2}),$$

where $m$ is the multiplication map while the second map is the projection to the $\sigma$-invariant part.

Actually, we are interested in the restriction of $m$ to $\text{Sym}^2 V^{\langle\sigma\rangle}$. Indeed $\langle\sigma\rangle \subseteq \text{Aut}(JC)$ acts on $JC$ and thus on $P(C, H)$. Therefore the image of $\mathcal{P}$ is contained in the locus of $(g-1)(p-1)$-abelian varieties with polarization of type $\delta$ and with an order $p$ automorphism. Hence the codifferential of $\mathcal{P}$ is given by

$$m: \left( \text{Sym}^2 \bigoplus_{i=1}^{p-1} H^0(H, \omega_H \otimes L^i) \right)^{\langle\sigma\rangle} \to H^0(H, \omega_H^{\otimes 2}). \quad (4)$$
Theorems 3.2 of [5], [11], [12] use the codifferential of the Prym map to give a sufficient criterion for a family of Galois covers $C \xrightarrow{2:1} C/\langle\sigma\rangle \to \mathbb{P}^1$ to yield special subvarieties of $\mathcal{A}_{g-1+\frac{r}{2}}^\delta$ generically contained in the Prym loci $\mathcal{P}(\mathcal{R}_{g,r})$. An analogous statement is provided in case of higher degree Galois covers $C \to C/\langle\sigma\rangle$ in [14, Theorem 4.1]. Indeed, all these authors start with a Prym datum $(G, \theta, \sigma)$. By the Riemann existence Theorem, it identifies a family of Galois covers $C \to C' := C/\langle\sigma\rangle \to C/G \cong \mathbb{P}^1$ with Galois group $G$ and monodromy $\theta$. Then, considering the corresponding family of Pryms $P(C, C') \in \mathcal{A}_{gC-gC'}^\delta$, they give the following:

**Definition 15.** A family of Galois covers $(G, \theta, \sigma)$ is said to satisfy condition B if the codifferential of the Prym map at the generic point of the family is an isomorphism.

Thus the following is shown.

**Theorem 16 ([5, 11, 12, 14]).** If condition B holds, then the closure in $\mathcal{A}_{gC-gC'}^\delta$ of the locus of the Prym varieties $P(C, C')$ of the family $(G, \theta, \sigma)$ yields a special subvariety generically contained in the Prym locus.

Hence the authors use this criterion to explicitly construct examples of such special varieties ([5, Theorem 1.3], [11, Theorem 1.1], [12, Theorem 1.1], [14, Table p. 21]).

Since our families (1) and (2) fit in the setting of [5, 11, 12, 14] and yield special varieties, it seems natural to check if they satisfy or not the aforementioned condition B. Indeed, as already observed, it is only known to be a sufficient criterion. We have the following:

**Proposition 17.** The multiplication map (4) of family (1) is not an isomorphism while that of family (2) is.

**Proof.** The proof is very easy and follows directly from the fact that the fibres of $\mathcal{P}$ have the same dimension of that of $\psi$. In case of family (1), i.e. if $g=2, p=5$, $\psi$ has fibres of dimension 1 (see [1, Proposition 4.2]). Therefore

$$d\mathcal{P}^* = m: (H^0(H, \omega_H \otimes L) \otimes H^0(H, \omega_H \otimes L^4)) \oplus (H^0(H, \omega_H \otimes L^2) \otimes H^0(H, \omega_H \otimes L^3)) \to H^0(H, \omega_H^2)$$

cannot be surjective. In case of family (2) we have that $d\psi$ is injective and so $d\mathcal{P}$ is too (Proposition 10). Therefore $m$ is surjective. Furthermore $\dim \text{Sym}^2(V_-)^{(\sigma)} = 3 = \dim H^0(H, \omega_H^{\otimes 2})$, since

$$ \begin{aligned} \text{Sym}^2(V_-)^{(\sigma)} = H^0(H, \omega_H \otimes L) \otimes H^0(H, \omega_H \otimes L^6) \oplus H^0(H, \omega_H \otimes L^2) \otimes H^0(H, \omega_H \otimes L^5) \\ \qquad \oplus H^0(H, \omega_H \otimes L^3) \otimes H^0(H, \omega_H \otimes L^4) \end{aligned} $$

and $h^0(H, \omega_H \otimes L^i) = 1$ for $i=1, 2, \dots, 6$. Hence we conclude. $\square$

**Proof of Corollary 5.** It follows from what is stated in Proposition 17 for family (1). $\square$

## References

[1] A. Albano, G. P. Pirola, “Dihedral monodromy and Xiao fibrations”, *Ann. Mat. Pura Appl.* **195** (2016), no. 4, p. 1255-1268.

[2] M. Bainbridge, M. Möller, “The locus of real multiplication and the Schottky locus”, *J. Reine Angew. Math.* **686** (2014), p. 167-186.

[3] C. Birkenhake, H. Lange, *Complex abelian varieties*, 2nd ed., Grundlehren der Mathematischen Wissenschaften, vol. 302, Springer, 2004.

[4] C. Ciliberto, G. van der Geer, M. Teixidor i Bigas, “On the number of parameters of curves whose Jacobians possess nontrivial endomorphisms”, *J. Algebr. Geom.* **1** (1992), no. 2, p. 215-229.

[5] E. Colombo, P. Frediani, A. Ghigi, M. Penegini, “Shimura curves in the Prym locus”, *Commun. Contemp. Math.* **21** (2019), no. 2, article no. 1850009 (34 pages).

[6] D. Conti, A. Ghigi, R. Pignatelli, “Some evidence for the Coleman-Oort Conjecture”, *Rev. R. Acad. Cienc. Exactas Fis. Nat., Ser. A Mat., RACSAM* **116** (2022), no. 1, article no. 50 (19 pages).
[7] J. S. Ellenberg, "Endomorphism algebras of Jacobians", Adv. Math. **162** (2001), no. 2, p. 243-271.

[8] P. Frediani, "Abelian covers and Second Fundamental Form", https://arxiv.org/abs/2105.07947, 2021.

[9] P. Frediani, A. Ghigi, M. Penegini, "Shimura varieties in the Torelli locus via Galois coverings", Int. Math. Res. Not. **2015** (2015), no. 20, p. 10595-10623.

[10] P. Frediani, A. Ghigi, I. Spelta, "Infinitely many Shimura varieties in the Jacobian locus for $g \le 4$", Ann. Sc. Norm. Super. Pisa, Cl. Sci. **22** (2021), no. 4, p. 1597-1619.

[11] P. Frediani, G. P. Grosselli, "Shimura curves in the Prym loci of ramified double covers", Int. J. Math. **32** (2021), no. 14, article no. 2150110 (23 pages).

[12] P. Frediani, G. P. Grosselli, A. Mohajer, "Higher dimensional Shimura varieties in the Prym loci of ramified double covers", https://arxiv.org/abs/2101.09016, to appear in Math. Nachr., 2021.

[13] P. Frediani, M. Penegini, P. Porru, "Shimura varieties in the Torelli locus via Galois coverings of elliptic curves", Geom. Dedicata **181** (2016), p. 177-192.

[14] G. P. Grosselli, A. Mohajer, "Shimura subvarieties in the Prym locus of ramified Galois coverings", https://arxiv.org/abs/2106.05704, 2021.

[15] G. P. Grosselli, I. Spelta, "Explicit analysis of positive dimensional fibres of $\mathcal{P}_{g,r}$ and Xiao conjecture", https://arxiv.org/abs/2110.03938, to appear in Rend. Ist. Mat. Univ. Trieste, 2021.

[16] S. Grushevsky, M. Möller, "Shimura curves within the locus of hyperelliptic Jacobians in genus 3", Int. Math. Res. Not. **2016** (2016), no. 6, p. 1603-1639.

[17] J. de Jong, R. Noot, "Jacobians with complex multiplication", in Arithmetic algebraic geometry, Progress in Mathematics, vol. 89, Birkhäuser, 1991, p. 177-192.

[18] J. de Jong, S.-W. Zhang, "Generic abelian varieties with real multiplication are not Jacobians", in Diophantine geometry, Centro di Ricerca Matematica Ennio De Giorgi (CRM) Series (Nuova Serie), vol. 4, Edizioni della Normale, 2007, p. 165-172.

[19] H. Lange, A. Ortega, "Prym varieties of cyclic coverings", Geom. Dedicata **150** (2011), p. 391-403.

[20] A. Mohajer, K. Zuo, "On Shimura subvarieties generated by families of abelian covers of $\mathbb{P}^1$", J. Pure Appl. Algebra **222** (2018), no. 4, p. 931-949.

[21] B. Moonen, "Linearity properties of Shimura varieties. I", J. Algebr. Geom. **7** (1998), no. 3, p. 539-567.

[22] —, "Special subvarieties arising from families of cyclic covers of the projective line", Doc. Math. **15** (2010), p. 793-819.

[23] B. Moonen, F. Oort, "The Torelli locus and special subvarieties", in Handbook of moduli. Volume II, Advanced Lectures in Mathematics, vol. 25, International Press, 2013, p. 549-594.

[24] D. Mumford, Abelian Varieties, Tata Institute of Fundamental Research Studies in Mathematics, vol. 5, Oxford University Press, 1974.

[25] F. Oort, J. Steenbrink, "The local Torelli problem for algebraic curves", in Journées de Géometrie Algébrique d'Angers, Sijthoff & Noordhoff, Alphenaan den Rijn, 1980, p. 157-204.

[26] G. P. Pirola, "On a conjecture of Xiao", J. Reine Angew. Math. **431** (1992), p. 75-89.

[27] J. Ries, "The Prym variety for a cyclic unramified cover of a hyperelliptic Riemann surface", J. Reine Angew. Math. **340** (1983), p. 59-69.